A kernel semi-supervised distance metric learning with relative distance: Integration with a MOO approach

Abstract

Metric learning, which aims to determine an appropriate distance function to measure the similarity and dissimilarity between data points accurately, is one of the most popular methods to enhance the performance of many machine learning methods such as K-means clustering and K nearest neighbor classifier algorithms. These algorithms may not perform well because of the use of normal Euclidean distance function that ignores any statistical regularities that might be estimated from a large training set of labeled examples. In many real-world applications, the Euclidean distance may not be fit to capture the intrinsic similarity and dissimilarity between the data points. Compared to existing metric learning algorithms, which use large amount of labeled data in the form of must-link (ML) and cannot-link constraints as side information where the granularity of true clustering is unknown, our proposed approach uses few labeled data in the form of relative-distance constraints such as equality constraints, Ceq, and inequality constraints, Cneq. For satisfying such constraints, we need to project the initial Euclidean distance matrix by using Bregman projection on the convex subset of constraints in such a way that all the constraints are satisfied. Since Bregman projection is not orthogonal, means while satisfying the current constraint previously satisfied constraints may get unsatisfied, we need to select a proper subset of constraints for learning better distance function. The multi-objective framework is utilized for selecting a good subset of constraints which can help in getting the proper labeling of the data set. The selected subset of constraints is used for adjusting the initial kernel-matrix. K-means clustering technique is applied to the adjusted kernel matrix to label the data set. In order to evaluate the quality of obtained labeling, different external and internal cluster validity indices are deployed. The values of these indices are simultaneously optimized using the search capability of MOO with the aim of selecting the appropriate subset of constraints. The proposed approach is evaluated on UCI Human Activity Recognition using Smartphone Dataset v1.0 along with nine other popular data sets. Results show that our approach outperforms the state of the art semi-supervised metric learning algorithms with respect to different internal and external cluster validity indices.