Distance Metric Learning Research Articles

Optical Remote Sensing Ship Image Classification Based on Deep Feature Combined Distance Metric Learning

Zhao, S.; Xu, Y.; Li, W., and Lang, H., 2020. Optical remote sensing ship image classification based on deep feature combined distance metric learning. In: Jung, H.-S.; Lee, S.; Ryu, J.-H., and Cui, T. (eds.), Advances in Geospatial Research of Coastal Environments. Journal of Coastal Research, Special Issue No. 102, pp. 82-87. Coconut Creek (Florida), ISSN 0749-0208.Image classification of ships has become an active research topic in the field of remote sensing. Effectively distinguishing specific category of a ship is crucial for many maritime matters. At present, there are two major difficulties in ship classification tasks. Firstly, the construction of traditional artificial features not only requires the assistance of professional knowledge, but also fails to represent the rich semantic information of ship images. Secondly, the classification of ships requires the identification of subcategory, the problem of intra-class diversity and inter-class similarity has become more serious and requires more careful handling. In order to solve the above problems, this paper proposes a method combining the deep feature and the distance metric learning (DML) algorithm to implement the classification of ships in the optical remote sensing images. In this study, an improved fine-tuned AlexNet convolution neural network (CNN) is adopted to extract the deep feature of ship images. The DML algorithm is applied to learn a good distance metric to reveal the similarity and dissimilarity between ships which are represented by deep feature. Comprehensive experiments show that the classification performance of deep feature combined DML algorithm significantly outperforms the comparative methods.

Learning Distance Metrics from Probabilistic Information

The goal of metric learning is to learn a good distance metric that can capture the relationships among instances, and its importance has long been recognized in many fields. An implicit assumption in the traditional settings of metric learning is that the associated labels of the instances are deterministic. However, in many real-world applications, the associated labels come naturally with probabilities instead of deterministic values, which makes it difficult for the existing metric-learning methods to work well in these applications. To address this challenge, in this article, we study how to effectively learn the distance metric from datasets that contain probabilistic information, and then propose several novel metric-learning mechanisms for two types of probabilistic labels, i.e., the instance-wise probabilistic label and the group-wise probabilistic label. Compared with the existing metric-learning methods, our proposed mechanisms are capable of learning distance metrics directly from the probabilistic labels with high accuracy. We also theoretically analyze the proposed mechanisms and conduct extensive experiments on real-world datasets to verify the desirable properties of these mechanisms.

XTML: A Unified Heterogeneous Transfer Metric Learning Framework for Multimedia Applications [Application Notes

Owing to the continual growth of multimodal data (or feature spaces), we have seen a rising interest in multimedia applications (e.g., object classification and searching) over these heterogeneous data. However, the accuracy of classification and searching tasks is highly dependent on the distance estimation between data samples, and simple Euclidean (EU) distance has been proven to be inadequate. Previous research has focused on learning a robust distance metric to quantify the relationships among data samples. In this context, existing distance metric learning (DML) algorithms mainly leverage on label information in the target domain for model training and may fail when the label information is scarce. As an improvement, transfer metric learning (TML) approaches are proposed to leverage information from other related domains. However, current TML algorithms assume that different domains explore the same representation; thus, they are not applicable in heterogeneous settings where the data representations of different domains vary. In this research, we propose xTML, a novel unified heterogeneous transfer metric learning framework, to improve the distance estimation of the domains of interest (i.e., the target domains in classification and searching tasks) when limited label information, complementary with extensive unlabeled data, is provisioned for model training. We further illustrate how our proposed framework can be applied to a selected list of multimedia applications, including opinion mining, deception detection and online product searching.

Unsupervised Metric Learning with Synthetic Examples

Distance Metric Learning (DML) involves learning an embedding that brings similar examples closer while moving away dissimilar ones. Existing DML approaches make use of class labels to generate constraints for metric learning. In this paper, we address the less-studied problem of learning a metric in an unsupervised manner. We do not make use of class labels, but use unlabeled data to generate adversarial, synthetic constraints for learning a metric inducing embedding. Being a measure of uncertainty, we minimize the entropy of a conditional probability to learn the metric. Our stochastic formulation scales well to large datasets, and performs competitive to existing metric learning methods.

Secure Metric Learning via Differential Pairwise Privacy

Distance Metric Learning (DML) has drawn much attention over the last two decades. A number of previous works have shown that it performs well in measuring the similarities of individuals given a set of correctly labeled pairwise data by domain experts. These important and precisely-labeled pairwise data are often highly sensitive in real world (e.g., patients similarity). This paper studies, for the first time, how pairwise information can be leaked to attackers during distance metric learning, and develops differential pairwise privacy (DPP), generalizing the definition of standard differential privacy, for secure metric learning. Unlike traditional differential privacy which only applies to independent samples, thus cannot be used for pairwise data, DPP successfully deals with this problem by reformulating the worst case. Specifically, given the pairwise data, we reveal all the involved correlations among pairs in the constructed undirected graph. DPP is then formalized that defines what kind of DML algorithm is private to preserve pairwise data. After that, a case study employing the contrastive loss is exhibited to clarify the details of implementing a DPP- DML algorithm. Particularly, the sensitivity reduction technique is proposed to enhance the utility of the output distance metric. Experiments both on a toy dataset and benchmarks demonstrate that the proposed scheme achieves pairwise data privacy without compromising the output performance much (Accuracy declines less than 0.01 throughout all benchmark datasets when the privacy budget is set at 4).

Distribution Shift Metric Learning for Fine-Grained Ship Classification in SAR Images

Fine-grained ship classification in synthetic aperture radar (SAR) images is a challenging task, since SAR images can only provide limited discriminative information due to the limitation of SAR imaging mechanism. Distance metric learning (DML) methods have the ability to improve the discriminative ability of the feature representations through preserving the supervisory information of ship samples. In this article, we proposed a novel DML method, termed as distribution shift metric learning (DML-ds), which improves the original Laplacian regularized metric learning by adding an interclass distribution shift regularization term. Extensive experiments and in-depth analysis demonstrate that the proposed DML-ds can effectively increase the interclass separability and the intraclass compactness, thereby improving the fine-grained ship classification performance in SAR images, and outperforms most of state-of-the-art methods.

Recurrent matching networks of spatial alignment learning for person re-identification

Person re-identification (re-id) usually refers to matching people across disjoint camera views. Many existing methods focus on extracting discriminative features or learning distance metrics to make the intraclass distance smaller than interclass distances. These methods subconsciously assume that pedestrian images are well aligned. However, one major challenge in person re-id is the unconstrained spatial misalignment between image pairs due to view angle changes and pedestrian pose variations. To address this problem, in this paper, we propose Recurrent Matching Network of Spatial Alignment Learning (RMN-SAL) to simulate the human vision perception. Reinforcement learning is introduced to locate attention regions, since it provides a flexible learning strategy for sequential decision-making. A linear mapping is employed to convert the environment state into spatial constraint, comprising spatial alignment into feature learning. And recurrent models are used to extract information from a sequence of corresponding regions. Finally, person re-id is performed based on the global features and the features from the learned alignment regions. Our contributions are: 1) the recurrent matching network, which can subtly combine local feature learning and sequential spatial correspondence learning into an end-to-end framework; 2) the design of a location network, which is based on reinforcement learning and aims to learn task-specific sequential spatial correspondences for different image pairs through the local pairwise internal representation interactions. The proposed model is evaluated on three benchmarks, including Market-1501, DukeMTMC-reID and CUHK03, and achieves better performances than other methods.

Transforming Device Fingerprinting for Wireless Security via Online Multitask Metric Learning

Device fingerprinting is a crucial part in the Internet of Things applications. Existing device-fingerprinting solutions either ignore the influence of the type of network traffic or separately learn a fingerprinting model for each traffic type. This often leads to suboptimal solutions, especially when training data are limited. Considering that the data distributions of different traffic types may be different but related, we propose a novel multitask learning method to learn the fingerprinting models for several traffic types simultaneously. Specifically, we first design a system for device fingerprinting using the popular k-nearest neighbor (KNN) approach. Then, a novel distance metric learning (DML) algorithm termed online multitask metric learning (OMTML) is developed to improve the distance estimation in our system. OMTML enables the models describing different traffic types to help each other during the metric learning procedure, and thus improving their respective accuracies. OMTML can also be updated adaptively, and the updating process is efficient. The experimental results show that the proposed KNN-based system outperforms the artificial neural network (ANN)-based counterpart significantly. Besides, the comparisons of our OMTML and other representative online and multitask DML approaches demonstrate both effectiveness and efficiency of the proposed metric learning method.

Significance of Softmax-Based Features in Comparison to Distance Metric Learning-Based Features.

End-to-end distance metric learning (DML) has been applied to obtain features useful in many computer vision tasks. However, these DML studies have not provided equitable comparisons between features extracted from DML-based networks and softmax-based networks. In this paper, we present objective comparisons between these two approaches under the same network architecture.

Learning distance to subspace for the nearest subspace methods in high-dimensional data classification

The nearest subspace methods (NSM) are a category of classification methods widely applied to classify high-dimensional data. In this paper, we propose to improve the classification performance of NSM through learning tailored distance metrics from samples to class subspaces. The learned distance metric is termed as ‘learned distance to subspace’ (LD2S). Using LD2S in the classification rule of NSM can make the samples closer to their correct class subspaces while farther away from their wrong class subspaces. In this way, the classification task becomes easier and the classification performance of NSM can be improved. The superior classification performance of using LD2S for NSM is demonstrated on three real-world high-dimensional spectral datasets.

Gaussian process approach for metric learning

Learning appropriate distance metric from data can significantly improve the performance of machine learning tasks under investigation. In terms of the distance metric representation forms in the models, distance metric learning (DML) approaches can be generally divided into two categories: parametric and non-parametric. The first category needs to make parametric assumption on the distance metric and learns the parameters, easily leading to overfitting and limiting model flexibility. The second category abandons the above assumption and instead, directly learns a non-parametric distance metric whose complexity can be adjusted according to the number of available training data, and makes the model representation relatively flexible. In this paper we follow the idea of the latter category and develop a non-parametric DML approach. The main challenge of our work concerns the formulation and learning of non-parametric distance metric. To meet this, we use Gaussian Process (GP) to extend the bilinear similarity into a non-parametric metric (here we abuse the concept of metric) and then learn this metric for specific task. As a result, our approach learns not only nonlinear metric that inherits the flexibility of GP but also representative features for the follow-up tasks. Compared with the existing GP-based feature learning approaches, our approach can provide accurate similarity prediction in the new feature space. To the best of our knowledge, this is the first work that directly uses GP as non-parametric metric. In the experiments, we compare our approach with related GP-based feature learning approaches and DML approaches respectively. The results demonstrate the superior performance of our approach.

Guest Editorial Introduction to the Special Issue on Large Scale and Nonlinear Similarity Learning for Intelligent Video Analysis

Learning similarity and distance measures has become increasingly important for the analysis, matching, retrieval, recognition, and categorization of video and multimedia data. With the ubiquitous use of digital imaging devices, mobile terminals and social networks, there are massive volumes of heterogeneous and homogeneous video and multimedia data from multiple sources, views, and domains, e.g., news media websites, microblog, mobile phone, social networking, etc. Similarity and distance-based constraints can also be extended and incorporated to boost classification and relationship learning. Moreover, the spatio-temporal coherence among video data can also be utilized for self-supervised learning of similarity and distance metrics. This trend has brought several challenging issues for developing similarity and metric learning methods for large scale and weakly annotated data, where outliers and incorrectly annotated data are inevitable. Recently, scalability has been investigated to cope with lightweight and large scale metric learning, while nonlinear similarity models have shown their great potentials in learning invariant representation and nonlinear measures of video and multimedia data.

Kernel Distance Metric Learning Using Pairwise Constraints for Person Re-Identification.

Person re-identification is a fundamental task in many computer vision and image understanding systems. Due to appearance variations from different camera views, person re-identification still poses an important challenge. In the literature, KISSME has already been introduced as an effective distance metric learning method using pairwise constraints to improve the re-identification performance. Computationally, it only requires two inverse covariance matrix estimations. However, the linear transformation induced by KISSME is not powerful enough for more complex problems. We show that KISSME can be kernelized, resulting in a nonlinear transformation, which is suitable for many real-world applications. Moreover, the proposed kernel method can be used for learning distance metrics from structured objects without having a vectorial representation. The effectiveness of our method is validated on five publicly available data sets. To further apply the proposed kernel method efficiently when data are collected sequentially, we introduce a fast incremental version that learns a dissimilarity function in the feature space without estimating the inverse covariance matrices. The experiments show that the latter variant can obtain competitive results in a computationally efficient manner.

Toward Intelligent Product Retrieval for TV-to-Online (T2O) Application: A Transfer Metric Learning Approach

It is desired (especially for young people) to shop for the same or similar products shown in the multimedia contents (such as online TV programs). This indicates an urgent demand for improving the experience of TV-to-Online (T2O). In this paper, a transfer learning approach as well as a prototype system for effortless T2O experience is developed. In the system, a key component is high-precision product search, which is to fulfill exact matching between a query item and the database ones. The matching performance primarily relies on distance estimation, but the data characteristics cannot be well modeled and exploited by a simple Euclidean distance. This motivates us to introduce distance metric learning (DML) for improving the distance estimation. However, in traditional DML methods, the side information (such as the similar/dissimilar constraints or relevance/irrelevance judgements) in the target domain is leveraged. These methods may fail due to limited side information. Fortunately, this issue can be alleviated by utilizing transfer metric learning (TML) to exploit information from other related domains. In this paper, a novel manifold regularized heterogeneous multitask metric learning framework is proposed, in which each domain is treated equally. The proposed approach allows us to simultaneously exploit the information from other domains and the unlabeled information. Furthermore, the ranking-based loss is adopted to make our model more appropriate for search. Experiments on two challenging real-world datasets demonstrate the effectiveness of the proposed method. This TML approach is expected to impact the transformation of the emerging T2O trend in both TV and online video domains.

Learning deep compact similarity metric for kinship verification from face images

Recent advances in kinship verification have shown that learning an appropriate kinship similarity metric on human faces plays a critical role in this problem. However, most of existing distance metric learning (DML) based solutions rely on linearity assumption of the kinship metric model, and the domain knowledge of large cross-generation discrepancy (e.g., large age span and gender difference between parent and child images) has not been considered in metric learning, leading to degraded performance for genetic similarity measure on human faces. To address these limitations, we propose in this work a new kinship metric learning (KML) method with a coupled deep neural network (DNN) model. KML explicitly models the cross-generation discrepancy inherent on parent-child pairs, and learns a coupled deep similarity metric such that the image pairs with kinship relation are pulled close, while those without kinship relation (but with high appearance similarity) are pushed as far away as possible. Moreover, by imposing the intra-connection diversity and inter-connection consistency over the coupled DNN, we introduce the property of hierarchical compactness into the coupled network to facilitate deep metric learning with limited amount of kinship training data. Empirically, we evaluate our algorithm on several kinship benchmarks against the state-of-the-art DML alternatives, and the results demonstrate the superiority of our method.

Content-based image retrieval model based on cost sensitive learning

How to retrieve the desired images quickly and accurately from the large scale image database has become a hot topic in the field of multimedia research. Many content-based image retrieval (CBIR) technologies already exist, but they are not always satisfactory. In many applications, the CBIR model based on machine learning relies heavily on the distance metric between samples. Although the traditional distance metric methods are simple and convenient, it is not always appropriate for CBIR tasks. In this paper, a novel distance metric learning (DML) method based on cost sensitive learning (CSL) is studied, and then it is used in a large margin distribution learning machine (LDM) to replace the traditional kernel functions. The improved LDM also takes into account CSL, and which is called CS-DLDM. Finally, CS-DLDM model is applied to CBIR tasks for implementation classification. We compare the proposed CS-DLDM model with other classifiers based on CSL. The experimental results show that the proposed CS-DLDM model not only has satisfactory classification performance but also the lowest misclassification cost, can effectively avoid the class imbalance of sample.

Local Large-Margin Multi-Metric Learning for Face and Kinship Verification

Metric learning has attracted wide attention in face and kinship verification, and a number of such algorithms have been presented over the past few years. However, most existing metric learning methods learn only one Mahalanobis distance metric from a single feature representation for each face image and cannot make use of multiple feature representations directly. In many face-related tasks, we can easily extract multiple features for a face image to extract more complementary information, and it is desirable to learn distance metrics from these multiple features, so that more discriminative information can be exploited than those learned from individual features. To achieve this, we present a large-margin multi-metric learning (LM3L) method for face and kinship verification, which jointly learns multiple global distance metrics under which the correlations of different feature representations of each sample are maximized, and the distance of each positive pair is less than a low threshold and that of each negative pair is greater than a high threshold. To better exploit the local structures of face images, we also propose a local metric learning and local LM3Lmethods to learn a set of local metrics. Experimental results on three face data sets show that the proposed methods achieve very competitive results compared with the state-of-the-art methods.

Multi-Level Common Space Learning for Person Re-Identification

Matching people in different cameras, commonly referred to as person re-identification, is a challenging task. The challenges come from the drastic appearance variation across different camera views caused by changes in pose, lighting condition, occlusion, background, and so on. Instead of matching images in the original feature space, many existing methods learn distance metrics or feature transformations to improve the matching accuracy. For example, data from different camera views can be projected onto their common space to mitigate the feature gap caused by view discrepancy. However, a single-step mapping may not be sufficient to close this gap as the view difference can be significant between different cameras. To overcome this limitation, we propose a multi-level common space learning framework, which can gradually minimize the data discrepancy between different views in an iterative manner. At each intermediate level, synthetic data are generated using the automatically discovered grouping information, and these synthetic data can be viewed as a transitional state between the original camera views. The matching is performed by mapping the probe and gallery onto their common space in multiple steps. We evaluate the proposed method on two widely used person re-identification data sets. The results show that the proposed multi-level scheme significantly outperforms single-level mapping. In addition, competitive results are achieved as compared with over 20 state-of-the-art techniques.

Fast generalization rates for distance metric learning

Distance metric learning (DML) aims to find a suitable measure to compute a distance between instances. Facilitated by side information, the learned metric can often improve the performance of similarity or distance based methods such as kNN. Theoretical analyses of DML focus on the learning effectiveness for squared Mahalanobis distance. Specifically, whether the Mahalanobis metric learned from the empirically sampled pairwise constraints is in accordance with the optimal metric optimized over the paired samples generated from the true distribution, and the sample complexity of this process. The excess risk could measure the quality of the generalization, i.e., the gap between the expected objective of empirical metric learned from a regularized objective with convex loss function and the one with the optimal metric. Given N training examples, existing analyses over this non-i.i.d. learning problem have proved the excess risk of DML converges to zero at a rate of $${\mathcal {O}}\left( \frac{1}{\sqrt{N}}\right) $$ . In this paper, we obtain a faster convergence rate of DML, $${\mathcal {O}}\left( \frac{1}{N}\right) $$ , when learning the distance metric with a smooth loss function and a strongly convex objective. In addition, when the problem is relatively easy, and the number of training samples is large enough, this rate can be further improved to $${\mathcal {O}}\left( \frac{1}{N^2}\right) $$ . Synthetic experiments validate that DML can achieve the specified faster generalization rate, and results under various settings help explore the theoretical properties of DML a lot.

Multi-View Ground-Based Cloud Recognition by Transferring Deep Visual Information

Since cloud images captured from different views possess extreme variations, multi-view ground-based cloud recognition is a very challenging task. In this paper, a study of view shift is presented in this field. We focus both on designing proper feature representation and learning distance metrics from sample pairs. Correspondingly, we propose transfer deep local binary patterns (TDLBP) and weighted metric learning (WML). On one hand, to deal with view shift, like variations of illuminations, locations, resolutions and occlusions, we first utilize cloud images to train a convolutional neural network (CNN), and then extract local features from the part summing maps (PSMs) based on feature maps. Finally, we maximize the occurrences of regions for the final feature representation. On the other hand, the number of cloud images in each category varies greatly, leading to the unbalanced similar pairs. Hence, we propose a weighted strategy for metric learning. We validate the proposed method on three cloud datasets (the MOC_e, IAP_e, and CAMS_e) that are collected by different meteorological organizations in China, and the experimental results show the effectiveness of the proposed method.