Deep Metric Learning Research Articles

Transforming crocodile traceability: Deep metric learning for identifying Siamese crocodiles

This study introduces a novel method for identifying individual Siamese crocodiles (Crocodylus siamensis), which is a crucial requirement for conservation and sustainable industry practices. Although deep metric learning (DML) has improved identification model robustness and reduced dependency on large datasets, comprehensive field studies and long-term deployments are lacking. To address this, DML combined with convolutional neural network (CNN) was applied for enhancing accuracy using a limited and imbalanced number of images per class and distinguishing dissimilar scale patterns of the head and ventral regions. Individual crocodiles were identified using the k-nearest neighbor (KNN) and support vector machine (SVM) classifiers based on the extracted features. Data were collected from 30 individuals on a crocodile farm using photographs taken over two consecutive years. Two identification types, Type 1, based on a model trained on images collected over two years; and Type 2, based on a model trained exclusively on images from the first year, were implemented. Type 1 identification, which used a CNN combined with the KNN and SVM classifiers, exhibited an accuracy exceeding 99.75 and 92.93% for the ventral and head regions, respectively. Type 2 identification exhibited a reduced accuracy because of a comparatively smaller amount of learning information; the proposed CNN achieved 83.99% accuracy for ventral identification and 67.14 and 65.61% for head identification with KNN and SVM, respectively. This study underscores the efficacy of DML and CNN for handling small, imbalanced datasets in identifying individual crocodiles, and has significant implications for traceability and conservation initiatives in the crocodile industry.

Attack detection model for BCoT based on contrastive variational autoencoder and metric learning

With development of blockchain technology, clouding computing and Internet of Things (IoT), blockchain and cloud of things (BCoT) has become development tendency. But the security has become the most development hinder of BCoT. Attack detection model is a crucial part of attack revelation mechanism for BCoT. As a consequence, attack detection model has received more concerned. Due to the great diversity and variation of network attacks aiming to BCoT, tradition attack detection models are not suitable for BCoT. In this paper, we propose a novel attack detection model for BCoT, denoted as cVAE-DML. The novel model is based on contrastive variational autoencoder (cVAE) and deep metric learning (DML). By training the cVAE, the proposed model generates private features for attack traffic information as well as shared features between attack traffic information and normal traffic information. Based on those generated features, the proposed model can generate representative new samples to balance the training dataset. At last, the decoder of cVAE is connected to the deep metric learning network to detect attack aiming to BCoT. The efficiency of cVAE-DML is verified using the CIC-IDS 2017 dataset and CSE-CIC-IDS 2018 dataset. The results show that cVAE-DML can improve attack detection efficiency even under the condition of unbalanced samples.

Improving deep metric learning via self-distillation and online batch diffusion process

Currently, deep metric learning (DML) methods typically rely on class labels to keep positive samples as closely clustered as possible, while distancing them from negative samples. However, this approach tends to disregard some vital information inherent in the data, such as intra-class information variation, which can hinder the generalization of the trained model. To address this issue, we propose an online batch diffusion-based self-distillation method (OBD-SD), which consists of a progressive self-distillation (PSD) technique and an online batch diffusion process (OBDP). Specifically, PSD is a simple yet effective self-distillation technique that is beneficial for the diversity of embedding representations. OBDP uses the diffusion process to reveal the intrinsic relationships among samples in the mini-batch and produce better soft distance targets. Our approach combines PSD with OBDP, which can extract richer relational information among samples by capturing the local geometric structure of manifolds in each mini-batch. OBD-SD is a highly flexible framework that can be integrated into state-of-the-art DML methods. Our experimental results on CUB200, CARS196, and Stanford Online Products datasets demonstrate that our OBD-SD consistently enhances the performance of existing DML methods, without requiring additional training time, resulting in competitive results.

Proxy-based robust deep metric learning in the presence of label noise

Real-world datasets contain label noise data that can deteriorate the performance of a deep learning model. Cleaning annotations manually requires substantial efforts from experts and is not practical in large datasets. Therefore, many methods are proposed to enhance the robustness of deep models in the presence of label noise. However, these methods are primarily developed for classification tasks and cannot be directly applied to Deep Metric Learning (DML) applications. To bridge this gap, we present a proxy-based deep metric learning method to detect wrong labels through the estimation of the posterior distribution for observed and true labels. Specifically, we model the distribution of the observed annotations as a mixture of two components: one component represents the clean label distribution and the other denotes the noisy labels. Besides, we present an EM-like algorithm that precisely identifies label noise data jointly with the similarity learning method. We also exploit information of the identified noisy labeled data by utilizing state-of-the-art semi-supervised learning (SSL) techniques. The results of the extensive experiments on datasets with real or synthetic label noise indicate that our method consistently outperforms the state-of-the-art methods by a large margin. Moreover, the proposed method accurately detects noisy labeled data and generates correct pseudo labels for them after only a few epochs.

A novel approach for identification of zoonotic trypanosome utilizing deep metric learning and vector database-based image retrieval system

Trypanosomiasis, a significant health concern in South America, South Asia, and Southeast Asia, requires active surveys to effectively control the disease. To address this, we have developed a hybrid model that combines deep metric learning (DML) and image retrieval. This model is proficient at identifying Trypanosoma species in microscopic images of thin-blood film examinations. Utilizing the ResNet50 backbone neural network, a trained-model has demonstrated outstanding performance, achieving an accuracy exceeding 99.71 % and up to 96 % in recall. Acknowledging the necessity for automated tools in field scenarios, we demonstrated the potential of our model as an autonomous screening approach. This was achieved by using prevailing convolutional neural network (CNN) applications, and vector database based-images returned by the KNN algorithm. This achievement is primarily attributed to the implementation of the Triplet Margin Loss function as 98 % of precision. The robustness of the model demonstrated in five-fold cross-validation highlights the ResNet50 neural network, based on DML, as a state-of-the-art CNN model as AUC >98 %. The adoption of DML significantly improves the performance of the model, remaining unaffected by variations in the dataset and rendering it a useful tool for fieldwork studies. DML offers several advantages over conventional classification model to manage large-scale datasets with a high volume of classes, enhancing scalability. The model has the capacity to generalize to novel classes that were not encountered during training, proving particularly advantageous in scenarios where new classes may consistently emerge. It is also well suited for applications requiring precise recognition, especially in discriminating between closely related classes. Furthermore, the DML exhibits greater resilience to issues related to class imbalance, as it concentrates on learning distances or similarities, which are more tolerant to such imbalances. These contributions significantly make the effectiveness and practicality of DML model, particularly in in fieldwork research.

Towards Improved Proxy-Based Deep Metric Learning via Data-Augmented Domain Adaptation

Deep Metric Learning (DML) plays an important role in modern computer vision research, where we learn a distance metric for a set of image representations. Recent DML techniques utilize the proxy to interact with the corresponding image samples in the embedding space. However, existing proxy-based DML methods focus on learning individual proxy-to-sample distance, while the overall distribution of samples and proxies lacks attention. In this paper, we present a novel proxy-based DML framework that focuses on aligning the sample and proxy distributions to improve the efficiency of proxy-based DML losses. Specifically, we propose the Data-Augmented Domain Adaptation (DADA) method to adapt the domain gap between the group of samples and proxies. To the best of our knowledge, we are the first to leverage domain adaptation to boost the performance of proxy-based DML. We show that our method can be easily plugged into existing proxy-based DML losses. Our experiments on benchmarks, including the popular CUB-200-2011, CARS196, Stanford Online Products, and In-Shop Clothes Retrieval, show that our learning algorithm significantly improves the existing proxy losses and achieves superior results compared to the existing methods. The code and Appendix are available at: https://github.com/Noahsark/DADA

Deep Metric Learning With Locality Sensitive Mining for Self-Correcting Source Separation of Neural Spiking Signals.

Automated source separation algorithms have become a central tool in neuroengineering and neuroscience, where they are used to decompose neurophysiological signal into its constituent spiking sources. However, in noisy or highly multivariate recordings these decomposition techniques often make a large number of errors. Such mistakes degrade online human-machine interfacing methods and require costly post-hoc manual cleaning in the offline setting. In this article we propose an automated error correction methodology using a deep metric learning (DML) framework, generating embedding spaces in which spiking events can be both identified and assigned to their respective sources. Furthermore, we investigate the relative ability of different DML techniques to preserve the intraclass semantic structure needed to identify incorrect class labels in neurophysiological time series. Motivated by this analysis, we propose locality sensitive mining, an easily implemented sampling-based augmentation to typical DML losses which substantially improves the local semantic structure of the embedding space. We demonstrate the utility of this method to generate embedding spaces which can be used to automatically identify incorrectly labeled spiking events with high accuracy.

Enhancing Sample Utilization in Noise-robust Deep Metric Learning with Subgroup-based Positive-pair Selection.

The existence of noisy labels in real-world data negatively impacts the performance of deep learning models. Although much research effort has been devoted to improving the robustness towards noisy labels in classification tasks, the problem of noisy labels in deep metric learning (DML) remains under-explored. Existing noisy label learning methods designed for DML mainly discard suspicious noisy samples, resulting in a waste of the training data. To address this issue, we propose a noise-robust DML framework with SubGroup-based Positive-pair Selection (SGPS), which constructs reliable positive pairs for noisy samples to enhance the sample utilization. Specifically, SGPS first effectively identifies clean and noisy samples by a probability-based clean sample selectionstrategy. To further utilize the remaining noisy samples, we discover their potential similar samples based on the subgroup information given by a subgroup generation module and then aggregate them into informative positive prototypes for each noisy sample via a positive prototype generation module. Afterward, a new contrastive loss is tailored for the noisy samples with their selected positive pairs. SGPS can be easily integrated into the training process of existing pair-wise DML tasks, like image retrieval and face recognition. Extensive experiments on multiple synthetic and real-world large-scale label noise datasets demonstrate the effectiveness of our proposed method. Without any bells and whistles, our SGPS framework outperforms the state-of-the-art noisy label DML methods.

An Adaptive Deep Metric Learning Loss Function for Class-Imbalance Learning via Intraclass Diversity and Interclass Distillation.

Deep metric learning (DML) has been widely applied in various tasks (e.g., medical diagnosis and face recognition) due to the effective extraction of discriminant features via reducing data overlapping. However, in practice, these tasks also easily suffer from two class-imbalance learning (CIL) problems: data scarcity and data density, causing misclassification. Existing DML losses rarely consider these two issues, while CIL losses cannot reduce data overlapping and data density. In fact, it is a great challenge for a loss function to mitigate the impact of these three issues simultaneously, which is the objective of our proposed intraclass diversity and interclass distillation (IDID) loss with adaptive weight in this article. IDID-loss generates diverse features within classes regardless of the class sample size (to alleviate the issues of data scarcity and data density) and simultaneously preserves the semantic correlations between classes using learnable similarity when pushing different classes away from each other (to reduce overlapping). In summary, our IDID-loss provides three advantages: 1) it can simultaneously mitigate all the three issues while DML and CIL losses cannot; 2) it generates more diverse and discriminant feature representations with higher generalization ability, compared with DML losses; and 3) it provides a larger improvement on the classes of data scarcity and density with a smaller sacrifice on easy class accuracy, compared with CIL losses. Experimental results on seven public real-world datasets show that our IDID-loss achieves the best performances in terms of G-mean, F1-score, and accuracy when compared with both state-of-the-art (SOTA) DML and CIL losses. In addition, it gets rid of the time-consuming fine-tuning process over the hyperparameters of loss function.

Deep Metric Learning Based on Meta-Mining Strategy With Semiglobal Information.

Recently, deep metric learning (DML) has achieved great success. Some existing DML methods propose adaptive sample mining strategies, which learn to weight the samples, leading to interesting performance. However, these methods suffer from a small memory (e.g., one training batch), limiting their efficacy. In this work, we introduce a data-driven method, meta-mining strategy with semiglobal information (MMSI), to apply meta-learning to learn to weight samples during the whole training, leading to an adaptive mining strategy. To introduce richer information than one training batch only, we elaborately take advantage of the validation set of meta-learning by implicitly adding additional validation sample information to training. Furthermore, motivated by the latest self-supervised learning, we introduce a dictionary (memory) that maintains very large and diverse information. Together with the validation set, this dictionary presents much richer information to the training, leading to promising performance. In addition, we propose a new theoretical framework that can formulate pairwise and tripletwise metric learning loss functions in a unified framework. This framework brings new insights to society and facilitates us to generalize our MMSI to many existing DML methods. We conduct extensive experiments on three public datasets, CUB200-2011, Cars-196, and Stanford Online Products (SOP). Results show that our method can achieve the state of the art or very competitive performance. Our source codes have been made available at https://github.com/NUST-Machine-Intelligence-Laboratory/MMSI.

Visual Embedding Augmentation in Fourier Domain for Deep Metric Learning

Deep Metric Learning (DML) is very effective for many computer vision applications such as image retrieval or cross-modal matching. The common paradigm for DML is to seek metric spaces that can encode semantically similar objects close while locating the dissimilar ones far away from each other. To make features more discriminative, the mainstream methods usually design various specific loss functions to seek the help of hard negatives through complex hard mining strategies or hard synthesizing with additional networks. In spite of their fruitfulness, these approaches ignore the impact of low-level information in images on the performance, which may degrade the discerning ability of learned embedding. To alleviate these problems, we introduce a simple yet effective augmentation method to generate more hard negatives by swapping the low-frequency spectra of negative instances with anchors in the Fourier domain. Specifically, unlike previous methods, our proposed approach does not involve any complex design strategies but enriches hard negatives by manipulating the low-level variability of images only with simple Fourier transforms. In addition, our method is treated as a universal plug-in, which can be incorporated into different models for performance improvement. In the end, we conduct extensive experiments to evaluate our method on the widely-used datasets including CUB-200-2011, CARS-196, and Stanford Online Products. Our quantitative results demonstrate that the proposed plug-in outperforms previous approaches consistently and significantly across different datasets and evaluation metrics.

Denoising Multi-Similarity Formulation: A Self-Paced Curriculum-Driven Approach for Robust Metric Learning

Deep Metric Learning (DML) is a group of techniques that aim to measure the similarity between objects through the neural network. Although the number of DML methods has rapidly increased in recent years, most previous studies cannot effectively handle noisy data, which commonly exists in practical applications and often leads to serious performance deterioration. To overcome this limitation, in this paper, we build a connection between noisy samples and hard samples in the framework of self-paced learning, and propose a Balanced Self-Paced Metric Learning (BSPML) algorithm with a denoising multi-similarity formulation, where noisy samples are treated as extremely hard samples and adaptively excluded from the model training by sample weighting. Especially, due to the pairwise relationship and a new balance regularization term, the sub-problem w.r.t. sample weights is a nonconvex quadratic function. To efficiently solve this nonconvex quadratic problem, we propose a doubly stochastic projection coordinate gradient algorithm. Importantly, we theoretically prove the convergence not only for the doubly stochastic projection coordinate gradient algorithm, but also for our BSPML algorithm. Experimental results on several standard data sets demonstrate that our BSPML algorithm has better generalization ability and robustness than the state-of-the-art robust DML approaches.

MES-Loss: Mutually equidistant separation metric learning loss function

Deep metric learning has attracted much attention in recent years due to its extensive applications, such as clustering and image retrieval. Thanks to the success of deep learning (DL), many deep metric learning (DML) methods have been proposed. Neural networks (NNs) utilize DML loss functions to learn a mapping function that maps samples into a highly discriminative low-dimensional feature space, facilitating measuring similarities between pairs of samples in such a manifold. Most existing methods usually try to boost the discriminatory power of NN by enhancing intra-class compactness in the high-level feature space. However, they do not explicitly imply constraints to improve inter-class separation. We propose in this paper a new composite DML loss function that, in addition to the intra-class compactness, explicitly implies regulations to enforce the best inter-class separation by mutually equidistantly distributing the centers of the classes. The proposed DML loss function achieved state-of-the-art results for clustering and image-retrieval tasks on two real-world data sets.

Multi-proxy based deep metric learning

Deep metric learning (DML) achieves excellent performance in many open-set scenarios. However, the current multi-proxy methods rely on a classification framework and the performances of these methods rely on large batch size due to the representation limitation of mini-batch for neighbor distribution of proxies. This study proposes a multi-proxy method aiming to enhance the generalization ability of DML on unseen classes by enhancing the data representation ability. As a sample may not share all fine-grained semantic information of a class, our model does not require full association between samples and proxies, and is updated only according to the associated pairs. Moreover, a mini-batch may not contain sufficient information for learning. So our model also considers the proxy-proxy relation in order to provide a global view of the data structure which improves the learning on the intra-class distance. This study also investigates how the positive and negative margins, i.e., the parameters of our model which control the required similarity for a class and different classes, affect our performance, and finds that less demanding on the negative margin avoids over-training and improves generalization ability to unseen classes. The outstanding performance of our model is confirmed by experimental results compared with the state-of-the-art DML methods in the benchmark image retrieval datasets.

Deep metric learning based approach for network intrusion detection

Today, intrusion detection systems are the way to defend network intrusion flows. In this paper, to categorize network traffic data, we proposed a novel method for detecting network intrusions. It builds intrusion detection models using a deep metric learning (DML) strategy that incorporates two multi-scale convolutional neural networks (MSCNN) and a Triplet network. During the phase of training MSCNN networks, the network traffic data are divided into attack network traffic data and normal data, and we train two distinct MSCNN networks on the basis of these two datasets. To determine the distance between the network traffic data, a Triplet network is trained to learn a mapping space which preserves the relationships of attacks and normal network flows. Soft-margin triplet loss is used as a loss function to train the Triplet network. In the prediction stage, each new flow passes through two MSCNN networks, which reconstruct the network flows, and then the Triplet network measures the distance between the network flow and the reconstructed flow. To verify the model’s advantages in intrusion detection, we compare the performance of the MSCNN+Triplet network, the CNN+LSTM, and the shallow neural network. Experimental results reveal the proposed scheme has superior detection performance compared to other intrusion detection schemes. The code will be available at https://gitee.com/wbsk/mscnn_triplet/tree/master/.

Deep Metric Learning for Scalable Gait-Based Person Re-Identification Using Force Platform Data

Walking gait data acquired with force platforms may be used for person re-identification (re-ID) in various authentication, surveillance, and forensics applications. Current force platform-based re-ID systems classify a fixed set of identities (IDs), which presents a problem when IDs are added or removed from the database. We formulated force platform-based re-ID as a deep metric learning (DML) task, whereby a deep neural network learns a feature representation that can be compared between inputs using a distance metric. The force platform dataset used in this study is one of the largest and the most comprehensive of its kind, containing 193 IDs with significant variations in clothing, footwear, walking speed, and time between trials. Several DML model architectures were evaluated in a challenging setting where none of the IDs were seen during training (i.e., zero-shot re-ID) and there was only one prior sample per ID to compare with each query sample. The best architecture was 85% accurate in this setting, though an analysis of changes in walking speed and footwear between measurement instances revealed that accuracy was 28% higher on same-speed, same-footwear comparisons, compared to cross-speed, cross-footwear comparisons. These results demonstrate the potential of DML algorithms for zero-shot re-ID using force platform data, and highlight challenging cases.

Multi proxy anchor family loss for several types of gradients

The deep metric learning (DML) objective is to learn a neural network that maps into an embedding space where similar data are near and dissimilar data are far. However, conventional proxy-based losses for DML have two problems: gradient problem and application of the real-world dataset with multiple local centers. Additionally, the performance metrics of DML also have some issues with stability and flexibility. This paper proposes three multi-proxies anchor (MPA) family losses and a normalized discounted cumulative gain (nDCG@k) metric. This paper makes three contributions. (1) MPA-family losses can learn using a real-world dataset with multi-local centers. (2) MPA-family losses improve the training capacity of a neural network owing to solving the gradient problem. (3) MPA-family losses have data-wise or class-wise characteristics with respect to gradient generation. Finally, we demonstrate the effectiveness of MPA-family losses, and MPA-family losses achieves higher accuracy on two datasets for fine-grained images.

Adaptive Hierarchical Similarity Metric Learning With Noisy Labels.

Deep Metric Learning (DML) plays a critical role in various machine learning tasks. However, most existing deep metric learning methods with binary similarity are sensitive to noisy labels, which are widely present in real-world data. Since these noisy labels often cause a severe performance degradation, it is crucial to enhance the robustness and generalization ability of DML. In this paper, we propose an Adaptive Hierarchical Similarity Metric Learning method. It considers two noise-insensitive information, i.e., class-wise divergence and sample-wise consistency. Specifically, class-wise divergence can effectively excavate richer similarity information beyond binary in modeling by taking advantage of Hyperbolic metric learning, while sample-wise consistency can further improve the generalization ability of the model using contrastive augmentation. More importantly, we design an adaptive strategy to integrate this information in a unified view. It is noteworthy that the new method can be extended to any pair-based metric loss. Extensive experimental results on benchmark datasets demonstrate that our method achieves state-of-the-art performance compared with current deep metric learning approaches.

Hybrid Deep Learning-Improved BAT Optimization Algorithm for Soil Classification Using Hyperspectral Features

Now a days, Remote Sensing (RS) techniques are used for earth observation and for detection of soil types with high accuracy and better reliability. This technique provides perspective view of spatial resolution and aids in instantaneous measurement of soil’s minerals and its characteristics. There are a few challenges that is present in soil classification using image enhancement such as, locating and plotting soil boundaries, slopes, hazardous areas, drainage condition, land use, vegetation etc. There are some traditional approaches which involves few drawbacks such as, manual involvement which results in inaccuracy due to human interference, time consuming, inconsistent prediction etc. To overcome these draw backs and to improve the predictive analysis of soil characteristics, we propose a Hybrid Deep Learning improved BAT optimization algorithm (HDIB) for soil classification using remote sensing hyperspectral features. In HDIB, we propose a spontaneous BAT optimization algorithm for feature extraction of both spectral-spatial features by choosing pure pixels from the Hyper Spectral (HS) image. Spectral-spatial vector as training illustrations is attained by merging spatial and spectral vector by means of priority stacking methodology. Then, a recurring Deep Learning (DL) Neural Network (NN) is used for classifying the HS images, considering the datasets of Pavia University, Salinas and Tamil Nadu Hill Scene, which in turn improves the reliability of classification. Finally, the performance of the proposed HDIB based soil classifier is compared and analyzed with existing methodologies like Single Layer Perceptron (SLP), Convolutional Neural Networks (CNN) and Deep Metric Learning (DML) and it shows an improved classification accuracy of 99.87%, 98.34% and 99.9% for Tamil Nadu Hills dataset, Pavia University and Salinas scene datasets respectively.

Metrics Space and Norm: Taxonomy to Distance Metrics

A lot of machine learning algorithms, including clustering methods such as K-nearest neighbor (KNN), highly depend on the distance metrics to understand the data pattern well and to make the right decision based on the data. In recent years, studies show that distance metrics can significantly improve the performance of the machine learning or deep learning model in clustering, classification, data recovery tasks, etc. In this article, we provide a survey on widely used distance metrics and the challenges associated with this field. The most current studies conducted in this area are commonly influenced by Siamese and triplet networks utilized to make associations between samples while employing mutual weights in deep metric learning (DML). They are successful because of their ability to recognize the relationships among samples that show a similarity. Furthermore, the sampling strategy, suitable distance metric, and network structure are complex and difficult factors for researchers to improve network model performance. So, this article is significant because it is the most recent detailed survey in which these components are comprehensively examined and valued as a whole, evidenced by assessing the numerical findings of the techniques.