Large margin projection-based multi-metric learning for classification

Abstract

Metric learning has been a promising technology to improve classification performance, which aims to learn a data-dependent distance metric such that the similarity between samples can be more effectively evaluated. Metric plays a significant role in the description of similarity between samples, however, learning a single distance metric is usually inadequate, especially when dealing with the heterogeneously distributed data. Traditional metric learning only considers a global metric, while the local metric, which is critical for heterogeneous data, is ignored. In this paper, we formulate a novel large margin projection-based multi-metric learning (LMML) for the binary classification of heterogeneous data, which constructs a unified framework based on global metric and local metrics, where two local distance metrics are learned, one for each class, so that the covariance of samples is as small as possible, and the sample of another class is as far away as possible from the mean of the sample. Moreover, a global distance metric is introduced to capture the common structure between the two classes, which requires that the distance metric in each class should be as close as possible to the global one. An efficient iterative algorithm is designed to optimize the LMML. We also conduct some insightful analyses on the computational complexity and the convergence of the proposed algorithm. Experiments are conducted on artificial datasets, UCI benchmark datasets and handwritten digit datasets to evaluate the proposed method. Compared with the state-of-the-art approaches, the experiment results demonstrate the feasibility and effectiveness of the proposed method.