Affine term structure model

Similarity learning is one type of a supervised machine learning task in artificial intelligence. It is closely related to regression and classification, but the goal is to learn from examples a function that measure how similar or related two objects are. It has applications in ranking and in recommendation systems.

Learning setup

There are three common setups for similarity and metric distance learning.

• Regression similarity learning. In this setup, pairs of objects are given $(x_i^1,x_i^2)$ together with a measure of their similarity $y_i\in R$. The goal is to learn a function that approximates $f(x_i^1,x_i^2)\sim y_i$ for every new labeled triplet example $(x_i^1,x_i^2,y_i)$. This is typically achieved by minimizing a regularized loss $mi{n}_W\sum _{i}loss(w;x_i^1,x_i^2,y_i)+reg(w)$.
• Classification similarity learning. Given are pairs of similar objects $(x_i,x_i^{+})$ and non similar objects $(x_i,x_i^{-})$. An equivalent formulation is that every pair $(x_i^1,x_i^2)$ is given together with a binary label $y_i\in \{0,1\}$ that determines if the two objects are similar or not. The goal is again to learn a classifier that can decide if a new pair of objects is similar or not.
• Ranking similarity learning. Given are triplets of objects $(x_i,x_i^{+},x_i^{-})$ whose relative similarity obey a predefined order: $x_i$ is known to be more similar to $x_i^{+}$ than to $x_i^{-}$. The goal is to learn a function $f$ such that for any new triplet of objects $(x,x^{+},x^{-})$, it obeys $f(x,x^{+})>f(x,x^{-})$. This setup assumes a weaker form of supervision than in regression, because instead of providing an exact measure of similarity, one only has to provide the relative order of similarity. For this reason, ranking-based similarity learning is easier to apply in real large scale applications.

A common approach for learning similarity, is to model the similarity function as a bilinear form. For example, in the case of ranking similarity learning, one aims to learn a matrix W that parametrizes the similarity function $f_W(x,z)=x^{T}Wz$.

Metric learning

Similarity learning is closely related to distance metric learning. Metric learning is the task of learning a distance function over objects. A metric or distance function has to obey three axioms: non-negativity, symmetry and subadditivity / triangle inequality.

When the objects $x_i$ are vectors in $R^d$, then any positive definite matrix $W>0$ defines a distance function of the space of x through the bilinear form $f(x_1,x_2)=x_1^{T}W{x}_2$. Some well known approaches for metric learning include Large margin nearest neighbor , Information theoretic metric learning (ITML).

In statistics, the covariance matrix of the data is sometimes used to define a distance metric called Mahalanobis distance.

Applications

Similarity learning is used in information retrieval for learning to rank, and in recommendation systems. Also, many machine learning approaches rely on some metric. This include unsupervised learning such as clustering, which groups together close or similar objects. It also includes supervised approaches like K-nearest neighbor algorithm which rely on labels of nearby objects to decide on the label of a new object. Metric learning has been proposed as a preprocessing step for many of these approaches

Further reading

For further information on this topic, see the survey on metric and similarity learning by Bellet et al.

References

43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.