Information Theoretic Metric Learning Research Articles

Symmetric Information–Theoretic Metric Learning for Target Detection in Hyperspectral Imagery

Metric learning-based methods, which yield great performance and show considerable potential to improve the performance of hyperspectral image processing, aim to calculate the Mahalanobis distance metric matrix. In this article, we proposed a symmetric information-theoretic metric learning (SITML) method for hyperspectral target detection. The SITML algorithm is designed based on the classical information-theoretic metric learning (ITML) and, minimizes the differential Kullback–Leibler (KL) divergence. To enhance both of the detection performance and the generalization ability, we build metric spaces from the neighborhood of training samples to preserve the local discriminative information. Then, we conduct local pairwise constraints to maximize the Jeffery divergence (also named the symmetric KL divergence) of two multivariate Gaussian distributions to solve the problem of an asymmetric KL divergence. Finally, we use a closed-form solution to solve the optimization problem. Intensive experiments on three hyperspectral datasets indicate that SITML outperforms the classical ITML algorithm and other state-of-the-art target detection methods.

Music Online Education Reform and Wireless Network Optimization Using Artificial Intelligence Piano Teaching

The purpose is to realize the intelligent reform of piano online teaching and the intelligent optimization of wireless networks. Empirical research is realized with quantitative research and algorithm simulation as the starting point. First, regression fitting algorithm and Relief F weight algorithm are adopted to extract the effectiveness of each characteristic variable. Next, under the guidance of metric learning theory, K-Nearest Neighbors (KNN) in Projected Feature Space (P-KNN) algorithm is proposed to complete the hierarchical recognition of piano teaching influence features. Metric Learning With Support Vector Machine (ML-SVM) classification algorithm is employed to identify the feature performance affecting piano teaching. Finally, the performance of P-KNN algorithm and ML-SVM algorithm is compared with KNN algorithm and Information-Theoretic-Metric-Learning (ITML) algorithm. It is concluded that the recognition accuracies of P-KNN and ML-SVM are 82.78% and 83.97%, respectively. Based on the quantitative research on the characteristics affecting piano teaching, artificial intelligence and wireless network optimization are combined to explore the implementation path of intelligent technology in piano teaching reform, reflect the use value of modern science and technology in piano teaching, and innovate the process of music online education reform of piano teaching.

Accelerating drug repurposing for COVID-19 treatment by modeling mechanisms of action using cell image features and machine learning.

The novel coronavirus disease, COVID-19, has rapidly spread worldwide. Developing methods to identify the therapeutic activity of drugs based on phenotypic data can improve the efficiency of drug development. Here, a state-of-the-art machine-learning method was used to identify drug mechanism of actions (MoAs) based on the cell image features of 1105 drugs in the LINCS database. As the multi-dimensional features of cell images are affected by non-experimental factors, the characteristics of similar drugs vary considerably, and it is difficult to effectively identify the MoA of drugs as there is substantial noise. By applying the supervised information theoretic metric-learning (ITML) algorithm, a linear transformation made drugs with the same MoA aggregate. By clustering drugs to communities and performing enrichment analysis, we found that transferred image features were more conducive to the recognition of drug MoAs. Image features analysis showed that different features play important roles in identifying different drug functions. Drugs that significantly affect cell survival or proliferation, such as cyclin-dependent kinase inhibitors, were more likely to be enriched in communities, whereas other drugs might be decentralized. Chloroquine and clomiphene, which block the entry of virus, were clustered into the same community, indicating that similar MoA could be reflected by the cell image. Overall, the findings of the present study laid the foundation for the discovery of MoAs of new drugs, based on image data. In addition, it provided a new method of drug repurposing for COVID-19.Supplementary InformationThe online version contains supplementary material available at 10.1007/s11571-021-09727-5.

Bearing performance degradation assessment based on information-theoretic metric learning and fuzzy c-means clustering

Bearing performance degradation assessment (PDA) is used to construct a degradation index (DI) to track the degradation process effectively and it plays an important role in the intelligent maintenance of rotating mechanical equipment. In order to construct a high quality DI of rolling bearing PDA processes, a bearing PDA model based on information-theoretic metric learning (ITML) and fuzzy C-means (FCM) clustering is proposed, which is called ITML-FCM in this paper. Firstly, the acquired vibration acceleration signals are decomposed into a set of intrinsic mode functions (IMFs) by variational mode decomposition (VMD) and fault features, such as singular value and relative energy of each IMF, are extracted to construct a high-dimensional feature space. Then, since the traditional distance metrics (i.e. Euclidean distance and Mahalanobis distance) ignore the label information of the samples, a distance metric matrix is learned using the ITML algorithm in the feature space based on the pairwise constraint from labeled training samples with normal and failure states. After that, the cluster centers of both the normal and the failure states are obtained using the FCM clustering algorithm, which is trained with the learned distance metric matrix. Finally, the DI is constructed using test samples' membership of a normal state. To evaluate the DI quality comprehensively, a weighted criterion using four metrics (i.e. trendability, monotonicty, robustness and discreteness) is proposed. Results of the analysis of the rolling bearing experimental data show that the distance metric matrix learned by ITML is conuctive to the classification of bearing degradation severity. Furthermore, the DI constructed by the ITML-FCM model shows superior comprehensive quality and, on average, the weighted criterion value is 8.45% higher than the FCM model using Euclidean distance and 19.71% higher than the FCM model using Mahalanobis distance.

Mathematical Analysis on Information-Theoretic Metric Learning With Application to Supervised Learning

This article presents a concrete mathematical analysis on Information-Theoretic Metric Learning (ITML). The analysis provides a theoretical foundation for ITML, by supplying well-posedness, strong duality, and convergence. Our analysis suggests the correction of a typo in the original ITML article that may lead to the loss of accuracy in the metric learning. The necessity of this correction is confirmed by several numerical experiments on supervised learning.

Hyperspectral Target Detection via Adaptive Information—Theoretic Metric Learning with Local Constraints

By using the high spectral resolution, hyperspectral images (HSIs) provide significant information for target detection, which is of great interest in HSI processing. However, most classical target detection methods may only perform well based on certain assumptions. Simultaneously, using limited numbers of target samples and preserving the discriminative information is also a challenging problem in hyperspectral target detection. To overcome these shortcomings, this paper proposes a novel adaptive information-theoretic metric learning with local constraints (ITML-ALC) for hyperspectral target detection. The proposed method firstly uses the information-theoretic metric learning (ITML) method as the objective function for learning a Mahalanobis distance to separate similar and dissimilar point-pairs without certain assumptions, needing fewer adjusted parameters. Then, adaptively local constraints are applied to shrink the distances between samples of similar pairs and expand the distances between samples of dissimilar pairs. Finally, target detection decision can be made by considering both the threshold and the changes between the distances before and after metric learning. Experimental results demonstrate that the proposed method can obviously separate target samples from background ones and outperform both the state-of-the-art target detection algorithms and the other classical metric learning methods.

Hyperspectral Image Classification Using Metric Learning in One-Dimensional Embedding Framework

Hyperspectral image (HSI) classification has become an active research area in the remote sensing field. In order to construct a simple and reliable classifier, learning an adequate distance metric from a given HSI dataset is still a critical and challenging task in many HSI applications. In this paper, a novel distance metric learning (DML) framework based on 1-D manifold embedding (1DME), named DL1DME, is proposed for HSI classification. The 1DME framework was developed by using the recently developed smooth ordering technique. This framework enables us to elaborately exploit the benefits of DML in the development of the 1DME algorithm. The core of the state-of-the-art DML is to learn a Mahalanobis matrix from the given dataset that better describes the similarity between pixels. Largest margin nearest neighbors (LMNN) and information theoretic metric learning (ITML) are employed for the Mahalanobis matrix learning. Then, based on the affinity defined by the Mahalanobis matrix, the preclassifiers are constructed using the simple 1-D regularization on 1DME; and they predict the labels of the test data. By a voting rule, the pixels labeled in the same class by most of the preclassifiers are voted into the confidently predicted set, which are then merged with the current labeled set. The labeled set enlargement process is repeated if the original one has a very small size. The final classifier is then constructed in the 1DME framework again, but based on the enlarged labeled set. According to the aforementioned strategy, two novel DML-based 1DME classification algorithms, DL1DME-LMNN and DL1DME-ITML, are developed in this paper. Experimental results on three popular real-world HSIs demonstrate that the classification performance of the proposed DL1DME is superior to other most popular SSL methods in terms of classification accuracies.

Data pair selection for accurate classification based on information-theoretic metric learning

Data classification is one of the main technique in data analysis which has become more and more important in various fields of business. Automatic classification is the problem that classification category label is learned from training data. One of the effective approaches for automatic classification is the k-nearest neighbour (kNN) method based on distances between data pairs, combining with the well-known distance metric learning. In this study, we focus on information-theoretic metric learning (ITML) method. In ITML, the optimisation problem is formulated as learning metric matrix so that the distance between each pair of data belonging to the same class becomes smaller than a constant, while the distance between each pair of data belonging to different classes becomes larger than the other constant. In this study, we propose an improved procedure by choosing the data-pairs which affect clarifying the boundaries effectively. We verify the effectiveness of our proposed method by conducting the simulation experiment with benchmark dataset.

Data pair selection for accurate classification based on information-theoretic metric learning

Data classification is one of the main technique in data analysis which has become more and more important in various fields of business. Automatic classification is the problem that classification category label is learned from training data. One of the effective approaches for automatic classification is the k-nearest neighbour (kNN) method based on distances between data pairs, combining with the well-known distance metric learning. In this study, we focus on information-theoretic metric learning (ITML) method. In ITML, the optimisation problem is formulated as learning metric matrix so that the distance between each pair of data belonging to the same class becomes smaller than a constant, while the distance between each pair of data belonging to different classes becomes larger than the other constant. In this study, we propose an improved procedure by choosing the data-pairs which affect clarifying the boundaries effectively. We verify the effectiveness of our proposed method by conducting the simulation experiment with benchmark dataset.

Distance Metric Learning Using Privileged Information for Face Verification and Person Re-Identification.

In this paper, we propose a new approach to improve face verification and person re-identification in the RGB images by leveraging a set of RGB-D data, in which we have additional depth images in the training data captured using depth cameras such as Kinect. In particular, we extract visual features and depth features from the RGB images and depth images, respectively. As the depth features are available only in the training data, we treat the depth features as privileged information, and we formulate this task as a distance metric learning with privileged information problem. Unlike the traditional face verification and person re-identification tasks that only use visual features, we further employ the extra depth features in the training data to improve the learning of distance metric in the training process. Based on the information-theoretic metric learning (ITML) method, we propose a new formulation called ITML with privileged information (ITML+) for this task. We also present an efficient algorithm based on the cyclic projection method for solving the proposed ITML+ formulation. Extensive experiments on the challenging faces data sets EUROCOM and CurtinFaces for face verification as well as the BIWI RGBD-ID data set for person re-identification demonstrate the effectiveness of our proposed approach.

A Kernel Classification Framework for Metric Learning.

Learning a distance metric from the given training samples plays a crucial role in many machine learning tasks, and various models and optimization algorithms have been proposed in the past decade. In this paper, we generalize several state-of-the-art metric learning methods, such as large margin nearest neighbor (LMNN) and information theoretic metric learning (ITML), into a kernel classification framework. First, doublets and triplets are constructed from the training samples, and a family of degree-2 polynomial kernel functions is proposed for pairs of doublets or triplets. Then, a kernel classification framework is established to generalize many popular metric learning methods such as LMNN and ITML. The proposed framework can also suggest new metric learning methods, which can be efficiently implemented, interestingly, using the standard support vector machine (SVM) solvers. Two novel metric learning methods, namely, doublet-SVM and triplet-SVM, are then developed under the proposed framework. Experimental results show that doublet-SVM and triplet-SVM achieve competitive classification accuracies with state-of-the-art metric learning methods but with significantly less training time.