Margin-based Loss Function Research Articles

HAMFace: Hardness adaptive margin loss for face recognition with various intra-class variations

The boost of convolutional neural networks (CNNs) has promoted the development of face recognition. Recently, the emergence of margin-based loss functions has further significantly improved the performance of face recognition. However, these methods sharply degrade in performance when dealing with large intra-class variations, including age, pose, illumination, resolution, and occlusion. Unlike most methods that target specific variations, our proposed approach, HAMFace, addresses the problems uniformly from the perspective of hard positive examples. To mitigate the intra-class variance, we argue that hard positive examples prefer larger margins, which can push them closer to their corresponding class centers. First, we design a hardness adaptive margin function to adjust the margin according to the hardness of the hard positive examples. Then, to enhance performance for unconstrained face recognition with various intra-class variations, we introduce a novel loss function named Hardness Adaptive Margin (HAM) Softmax Loss. This loss function allocates larger margins to hard positive examples during training based on their level of hardness. The proposed HAMFace is evaluated on nine challenging face recognition benchmarks and exhibits its superiority compared with other state-of-the-arts.

JAMsFace: joint adaptive margins loss for deep face recognition

Deep feature learning has become crucial in large-scale face recognition, and margin-based loss functions have demonstrated impressive success in this field. These methods aim to enhance the discriminative power of the softmax loss by increasing the feature margin between different classes. These methods assume class balance, where a fixed margin is sufficient to squeeze intra-class variation equally. However, real-face datasets often exhibit imbalanced classes, where the fixed margin is suboptimal, limiting the discriminative power and generalizability of the face recognition model. Furthermore, margin-based approaches typically focus on enhancing discrimination either in the angle or cosine space, emphasizing one boundary while disregarding the other. To overcome these limitations, we propose a joint adaptive margins loss function (JAMsFace) that learns class-related margins for both angular and cosine spaces. This approach allows adaptive margin penalties to adjust adaptively for different classes. We explain and analyze the proposed JAMsFace geometrically and present comprehensive experiments on multiple face recognition benchmarks. The results show that JAMsFace outperforms existing face recognition losses in mainstream face recognition tasks. Specifically, JAMsFace advances the state-of-the-art face recognition performance on LFW, CPLFW, and CFP-FP and achieves comparable results on CALFW and AgeDB-30. Furthermore, for the challenging IJB-B and IJB-C benchmarks, JAMsFace achieves impressive true acceptance rates (TARs) of 89.09% and 91.81% at a false acceptance rate (FAR) of 1e-4, respectively.

Contactless Palm Vein Recognition Based on Attention-Gated Residual U-Net and ECA-ResNet

Palm vein recognition has received some considerable attention regarding its use in biometric identification. Palm vein characteristics offer a superior level of security and reliability in personal identification compared to extrinsic methods such as fingerprint, face, and palm print recognition, as vein patterns are difficult to duplicate and do not change throughout one’s lifetime. This study proposes both segmentation and recognition methods to enhance the authentication performance and achieve correct identification using palm vein features. First, we propose a segmentation model based on the U-Net model, enhanced with an attention gate, to effectively segment palm vein patterns. The incorporation of both the attention gate and residual block allows the segmentation model for the learning of the essential features required for specific segmentation tasks. The Hessian-based Jerman filtering method is used for ground-truth labeling. The segmentation model extracts the palm vein patterns and filters out the irrelevant and noisy pixels for the purpose of recognition. The efficient channel attention residual network is trained to learn discriminative features for personal identification using combined margin-based loss functions for palm vein recognition. The channel attention module enhances the useful information and suppresses irrelevant features in the feature maps, which overcomes the problem of rotation, position translation, and scale transformation, as well as improves the recognition rate. The combined loss function used in this study increases the similarity between the intra-class samples and the diversity between inter-class samples. The proposed recognition model achieved 100% accuracy for palm vein recognition and an equal error rate of 0.018 for palm vein verification.

HeadPose-Softmax: Head pose adaptive curriculum learning loss for deep face recognition

Face recognition has been one of the most popular applications in the field of target detection. Currently, frontal faces can be easily detected, but multi-view face detection remains a difficult task because of various factors such as illumination, various poses, occlusions, and facial expressions. Margin-based loss functions are used to increase the feature margins between different classes, thus enhancing the discriminability of face recognition models, but the performance in face detection in complex scenes (e.g., high pitch angle face detection in surveillance environments) can be significantly degraded. Recently, the idea of a mining-based strategy to emphasize hard samples has been used to achieve good results in multi-view face detection. However, most of the existing methods do not explicitly emphasize samples based on their importance, resulting in the underutilization of hard samples. In this paper, we propose a curriculum learning loss function (HeadPose-Softmax) to classify the difficulty of a sample based on its facial pose, and embed the concept of curriculum learning into the loss function to implement a novel training strategy for deep face recognition. The loss function explicitly emphasizes the importance of the samples according to the different difficulty of each sample, which allows the model to make fuller use of hard samples, focus on learning pose invariant features, and improve the accuracy of the model in multi-view face detection tasks. Specifically, our HeadPose-Softmax dynamically adjusts the relative importance of the hard samples according to the pose angle of the face in the hard samples during the training phase. At each stage, different samples are assigned different importance according to their corresponding difficulty. Extensive experimental results under popular benchmarks show that our HeadPose-Softmax can enhance the accuracy of the model in multi-view face detection and outperform the state-of-the-art competitors.

UnifiedFace: A Uniform Margin Loss Function for Face Recognition

Face recognition has achieved great success due to the development of deep convolutional neural networks (DCNNs) and loss functions based on margin. However, complex DCNNs bring a large number of parameters as well as computational effort, which pose a significant challenge to resource-constrained embedded devices. Meanwhile, the popular margin-based loss functions all introduce only one type of margin and cannot further introduce a larger margin to achieve tighter classification boundary. In contrast to the common approach, we believe that additive and multiplicative margins should be used jointly to introduce larger margins from the margin perspective. Therefore, we propose a new margin-based loss function called UnifiedFace. First, we introduce an additive margin in the target angle activation function. Second, we add a multiplicative margin in the non-target angle. UnifiedFace introduces both additive and multiplicative margins, allowing for the introduction of large margins to achieve more compact intra-class variance and closer separated inter-class variance. In addition, we specifically design efficient face recognition models called GhostFaceNet for resource-constrained embedded devices. Experimental results demonstrate that UnifiedFace achieves state-of-the-art performance or performance competed with popular methods in training datasets of different sizes. UnifiedFace achieves optimal performance in models of varying complexity. Moreover, competitive results are achieved in the large-scale test set IJBB/C, especially the state-of-the-art performance achieved in TAR (FAR=1e−6). GhostFaceNet can significantly improve operational efficiency without significantly degrading recognition performance, making it ideal for embedded devices with limited resources.

A deep learning loss based on additive cosine margin: Application to fashion style and face recognition

Recently, loss functions based on angular spans improved the performance of deep visual recognition. These losses converted Euclidean cross entropy to angular cross entropy loss. Fashion style recognition deals with the problem of assigning a person’s outfit to a fashion style category. Due to the high similarity between different clothing items and the use of softmax-based loss functions, many of the current methods that address this problem show relatively poor performance and cannot guarantee sufficient inter-class margins in the fashion domain. In this work, we propose an end-to-end method for deep visual recognition by combining a standard CNN architecture with a novel loss function, which we call Additive Cosine Margin Loss (ACML). The proposed function not only projects feature vectors of different classes into different regions of the embedding, but also enforces compactness of the projections within each class. Our experiments were conducted on two public and well-known fashion style recognition datasets FashionStyle14 and HipsterWars, and on the face verification and identification datasets LFW, YTF, and MegaFace. These experiments demonstrate the superiority of the proposed loss function over: (1) existing angular margin-based loss functions (2) state-of-the-art methods for clothing style recognition as well as face analysis tasks.

LocalFace: Learning significant local features for deep face recognition

As a widely mentioned topic in face recognition, the margin-based loss function enhances the discriminability of face recognition models by applying margin between class decision boundaries. However, there is still room to improve the representation of face features. Local face feature extraction has been employed in traditional face recognition methods, but with the increase of network depth in deep learning, the traditional approach requires a large number of computational resources. In this paper, we propose a novel face recognition architecture called LocalFace to extract local face features. First, by analyzing the distribution of significant features in face images, we propose an efficient face fixed-point local feature extraction approach and improve this method to propose a more effective face dynamic local feature extraction scheme. Subsequently, we propose a block-based random occlusion method for the limitations of the random face occlusion method to better simulate the occlusion situation in real scenes. In the end, we present a detailed discussion on the channel attention method that is more appropriate for face recognition and classification tasks. Our method enhances the representation of face features by ensembling local features into global features without extra parameters, which is efficient and easy to implement. Extensive experiments on various benchmarks demonstrate the superiority of our LocalFace, and part of the experimental results achieve SOTA results.

Link Prediction in Knowledge Graphs: A Hierarchy-Constrained Approach

Link prediction over a knowledge graph aims to predict the missing head entities <inline-formula><tex-math notation="LaTeX">$h$</tex-math></inline-formula> or tail entities <inline-formula><tex-math notation="LaTeX">$t$</tex-math></inline-formula> and missing relations <inline-formula><tex-math notation="LaTeX">$r$</tex-math></inline-formula> for a triple <inline-formula><tex-math notation="LaTeX">$(h,r,t)$</tex-math></inline-formula> . Recent years have witnessed great advance of knowledge graph embedding based link prediction methods, which represent entities and relations as elements of a continuous vector space. Most methods learn the embedding vectors by optimizing a margin-based loss function, where the margin is used to separate negative and positive triples in the loss function. The loss function utilizes the general structures of knowledge graphs, e.g., the vector of <inline-formula><tex-math notation="LaTeX">$r$</tex-math></inline-formula> is the translation of the vector of <inline-formula><tex-math notation="LaTeX">$h$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$t$</tex-math></inline-formula> , and the vector of <inline-formula><tex-math notation="LaTeX">$t$</tex-math></inline-formula> should be the nearest neighbor of the vector of <inline-formula><tex-math notation="LaTeX">$h+r$</tex-math></inline-formula> . However, there are many particular structures, and can be employed to promote the performance of link prediction. One typical structure in knowledge graphs is hierarchical structure, which existing methods have much unexplored. We argue that the hierarchical structures also contain rich inference patterns, and can further enhance the link prediction performance. In this paper, we propose a hierarchy-constrained link prediction method, called hTransM, on the basis of the translation-based knowledge graph embedding methods. It can adaptively determine the optimal margin by detecting the single-step and multi-step hierarchical structures. Moreover, we prove the effectiveness of hTransM theoretically, and experiments over three benchmark datasets and two sub-tasks of link prediction demonstrate the superiority of hTransM.

Boosting Iris Recognition by Margin-Based Loss Functions

In recent years, the topic of contactless biometric identification has gained considerable traction due to the COVID-19 pandemic. One of the most well-known identification technologies is iris recognition. Determining the classification threshold for large datasets of iris images remains challenging. To solve this issue, it is essential to extract more discriminatory features from iris images. Choosing the appropriate loss function to enhance discrimination power is one of the most significant factors in deep learning networks. This paper proposes a novel iris identification framework that integrates the light-weight MobileNet architecture with customized ArcFace and Triplet loss functions. By combining two loss functions, it is possible to improve the compactness within a class and the discrepancies between classes. To reduce the amount of preprocessing, the normalization step is omitted and segmented iris images are used directly. In contrast to the original SoftMax loss, the EER for the combined loss from ArcFace and Triplet is decreased from 1.11% to 0.45%, and the TPR is increased from 99.77% to 100%. In CASIA-Iris-Thousand, EER decreased from 4.8% to 1.87%, while TPR improved from 97.42% to 99.66%. Experiments have demonstrated that the proposed approach with customized loss using ArcFace and Triplet can significantly improve state-of-the-art and achieve outstanding results.

DeltaVLAD: An efficient optimization algorithm to discriminate speaker embedding for text-independent speaker verification

&lt;abstract&gt; &lt;p&gt;Text-independent speaker verification aims to determine whether two given utterances in open-set task originate from the same speaker or not. In this paper, some ways are explored to enhance the discrimination of embeddings in speaker verification. Firstly, difference is used in the coding layer to process speaker features to form the DeltaVLAD layer. The frame-level speaker representation is extracted by the deep neural network with differential operations to calculate the dynamic changes between frames, which is more conducive to capturing insignificant changes in the voiceprint. Meanwhile, NeXtVLAD is adopted to split the frame-level features into multiple word spaces before aggregating, and subsequently perform VLAD operations in each subspace, which can significantly reduce the number of parameters and improve performance. Secondly, the margin-based softmax loss function and the few-shot learning-based loss function are proposed to be combined for more discriminative speaker embeddings. Finally, for a fair comparison, the experimental results are performed on Voxceleb-1 showing superior performance of speaker verification system and can obtain new state-of-the-art results.&lt;/p&gt; &lt;/abstract&gt;

Text-independent speaker recognition based on adaptive course learning loss and deep residual network

Text-independent speaker recognition is widely used in identity recognition that has a wide spectrum of applications, such as criminal investigation, payment certification, and interest-based customer services. In order to improve the recognition ability of log filter bank feature vectors, a method of text-independent speaker recognition based on deep residual networks model was proposed in this paper. The deep residual network was composed of a residual network (ResNet) and a convolutional attention statistics pooling (CASP) layer. The CASP layer could aggregate frame-level features from the ResNet into an utterance-level features. Extracting speech features for each speaker using deep residual networks was a promising direction to explore, and a straightforward solution was to train the discriminative feature extraction network by using a margin-based loss function. However, a margin-based loss function often has certain limitations, such as the margins between different categories were set to be the same and fixed. Thus, we used an adaptive curriculum learning loss (ACLL) to address the problem and introduce two different margin-based losses for this problem, i.e., AM-Softmax and AAM-Softmax. The proposed method was applied to a large-scale VoxCeleb2 dataset for extensive text-independent speaker recognition experiments, and average equal error rate (EER) could achieve 1.76% on VoxCeleb1 test dataset, 1.91% on VoxCeleb1-E test dataset, and 3.24% on VoxCeleb1-H test dataset. Compared with related speaker recognition methods, EER was improved by 1.11% on VoxCeleb1 test dataset, 1.04% on VoxCeleb1-E test dataset, and 1.69% on VoxCeleb1-H test dataset.

A directional margin paradigm for noise suppression in face recognition

Convolutional neural networks (CNN) have achieved outstanding face recognition (FR) performance with increasing large-scale face datasets. With face dataset size grown, noisy data will inevitably increase, undoubtedly bringing difficulties to data cleaning. In this paper, the probability that the sample belongs to noise can be determined based on the cosine distance (cosθ) of normalized angle center and face feature vector in the margin-based loss functions. According to this finding, we propose a two-step learning method integrated into the loss function. The new proposed directional margin loss function combines the noise probability with the label as the supervision information. Experiments show that our method can tolerate noisy data and get high FR accuracy when the training datasets mix with more than 30% noise. Our approach can also achieve a great result of 79.33% in MegaFace challenge one using a noisy training dataset.

Error-Corrected Margin-Based Deep Cross-Modal Hashing for Facial Image Retrieval

Cross-modal hashing facilitates mapping of heterogeneous multimedia data into a common Hamming space, which can be utilized for fast and flexible retrieval across different modalities. In this paper, we propose a novel cross-modal hashing architecture-deep neural decoder cross-modal hashing (DNDCMH), which uses a binary vector specifying the presence of certain facial attributes as an input query to retrieve relevant face images from a database. The DNDCMH network consists of two separate components: an attribute-based deep cross-modal hashing (ADCMH) module, which uses a margin (m)-based loss function to efficiently learn compact binary codes to preserve similarity between modalities in the Hamming space, and a neural error correcting decoder (NECD), which is an error correcting decoder implemented with a neural network. The goal of NECD network in DNDCMH is to error correct the hash codes generated by ADCMH to improve the retrieval efficiency. The NECD network is trained such that it has an error correcting capability greater than or equal to the margin (m) of the margin-based loss function. This results in NECD can correct the corrupted hash codes generated by ADCMH up to the Hamming distance of m. We have evaluated and compared DNDCMH with state-of-the-art cross-modal hashing methods on standard datasets to demonstrate the superiority of our method.

A Method to Learn Embedding of a Probabilistic Medical Knowledge Graph: Algorithm Development.

BackgroundKnowledge graph embedding is an effective semantic representation method for entities and relations in knowledge graphs. Several translation-based algorithms, including TransE, TransH, TransR, TransD, and TranSparse, have been proposed to learn effective embedding vectors from typical knowledge graphs in which the relations between head and tail entities are deterministic. However, in medical knowledge graphs, the relations between head and tail entities are inherently probabilistic. This difference introduces a challenge in embedding medical knowledge graphs.ObjectiveWe aimed to address the challenge of how to learn the probability values of triplets into representation vectors by making enhancements to existing TransX (where X is E, H, R, D, or Sparse) algorithms, including the following: (1) constructing a mapping function between the score value and the probability, and (2) introducing probability-based loss of triplets into the original margin-based loss function.MethodsWe performed the proposed PrTransX algorithm on a medical knowledge graph that we built from large-scale real-world electronic medical records data. We evaluated the embeddings using link prediction task.ResultsCompared with the corresponding TransX algorithms, the proposed PrTransX performed better than the TransX model in all evaluation indicators, achieving a higher proportion of corrected entities ranked in the top 10 and normalized discounted cumulative gain of the top 10 predicted tail entities, and lower mean rank.ConclusionsThe proposed PrTransX successfully incorporated the uncertainty of the knowledge triplets into the embedding vectors.

Path-specific knowledge graph embedding

Knowledge graph embedding aims to represent entities, relations and multi-step relation paths of a knowledge graph as vectors in low-dimensional vector spaces, and supports many applications, such as entity prediction, relation prediction, etc. Existing embedding methods learn the representations of entities, relations, and multi-step relation paths by minimizing a general margin-based loss function shared by all relation paths. This setting fails to consider the differences among different relation paths.In this paper, we propose an embedding method by minimizing a path-specific margin-based loss function for knowledge graph embedding, called PaSKoGE. For each path, it adaptively determines its margin-based loss function by encoding the correlation between relations and multi-step relation paths for any given pair of entities. PaSKoGE outperforms the-state-of-the-art methods.

Deep Physiological Affect Network for the Recognition of Human Emotions

Here we present a robust physiological model for the recognition of human emotions, called Deep Physiological Affect Network. This model is based on a convolutional long short-term memory (ConvLSTM) network and a new temporal margin-based loss function. Formulating the emotion recognition problem as a spectral-temporal sequence classification problem of bipolar EEG signals underlying brain lateralization and photoplethysmogram signals, the proposed model improves the performance of emotion recognition. Specifically, the new loss function allows the model to be more confident as it observes more of specific feelings while training ConvLSTM models. The function is designed to result in penalties for the violation of such confidence. Our experiments on a public dataset show that our deep physiological learning technology significantly increases the recognition rate of state-of-the-art techniques by 15.96 percent increase in accuracy. An extensive analysis of the relationship between participants’ emotion ratings and physiological changes in brain lateralization function during the experiment is also presented.

Knowledge Graph Embedding

A knowledge graph is a graph with entities of different types as nodes and various relations among them as edges. The construction of knowledge graphs in the past decades facilitates many applications, such as link prediction, web search analysis, question answering, and so on. Knowledge graph embedding aims to represent entities and relations in a large-scale knowledge graph as elements in a continuous vector space. Existing methods, for example, TransE, TransH, and TransR, learn the embedding representation by defining a global margin-based loss function over the data. However, the loss function is determined during experiments whose parameters are examined among a closed set of candidates. Moreover, embeddings over two knowledge graphs with different entities and relations share the same set of candidates, ignoring the locality of both graphs. This leads to the limited performance of embedding related applications. In this article, a locally adaptive translation method for knowledge graph embedding, called TransA, is proposed to find the loss function by adaptively determining its margin over different knowledge graphs. Then the convergence of TransA is verified from the aspect of its uniform stability. To make the embedding methods up-to-date when new vertices and edges are added into the knowledge graph, the incremental algorithm for TransA, called iTransA, is proposed by adaptively adjusting the optimal margin over time. Experiments on four benchmark data sets demonstrate the superiority of the proposed method, as compared to the state-of-the-art ones.

Sub-Category Classifiers for Multiple-Instance Learning and its Application to Retinal Nerve Fiber Layer Visibility Classification.

We propose a novel multiple instance learning method to assess the visibility (visible/not visible) of the retinal nerve fiber layer (RNFL) in fundus camera images. Using only image-level labels, our approach learns to classify the images as well as to localize the RNFL visible regions. We transform the original feature space to a discriminative subspace, and learn a region-level classifier in that subspace. We propose a margin-based loss function to jointly learn this subspace and the region-level classifier. Experiments with a RNFL dataset containing 884 images annotated by two ophthalmologists give a system-annotator agreement (kappa values) of 0:73 and 0:72 respectively, with an inter-annotator agreement of 0:73. Our system agrees better with the more experienced annotator. Comparative tests with three public datasets (MESSIDOR and DR for diabetic retinopathy, UCSB for breast cancer) show that our novel MIL approach improves performance over the state-of-the-art. Our Matlab code is publicly available at https://github.com/ManiShiyam/Sub-category-classifiersfor- Multiple-Instance-Learning/wiki.

Locally Adaptive Translation for Knowledge Graph Embedding

Knowledge graph embedding aims to represent entities and relations in a large-scale knowledge graph as elements in a continuous vector space. Existing methods, e.g., TransE and TransH, learn embedding representation by defining a global margin-based loss function over the data. However, the optimal loss function is determined during experiments whose parameters are examined among a closed set of candidates. Moreover, embeddings over two knowledge graphs with different entities and relations share the same set of candidate loss functions, ignoring the locality of both graphs. This leads to the limited performance of embedding related applications. In this paper, we propose a locally adaptive translation method for knowledge graph embedding, called TransA, to find the optimal loss function by adaptively determining its margin over different knowledge graphs. Experiments on two benchmark data sets demonstrate the superiority of the proposed method, as compared to the-state-of-the-art ones.

Comparative analysis on margin based feature selection algorithms

Feature evaluation and selection is an important preprocessing step in classification and regression learning. As large quantity of irrelevant information is gathered, selecting the most informative features may help users to understand the task, and enhance the performance of the models. Margin has been widely accepted and used in evaluating feature quality these years. A collection of feature selection algorithms were developed using margin based loss functions and various search strategies. However, there is no comparative research conducted to study the effectiveness of these algorithms. In this work, we compare 14 margin based feature selections from the viewpoints of reduction capability, classification performance of reduced data and robustness, where four margin based loss functions and three search strategies are considered. Moreover, we also compare these techniques with two well-known margin based feature selection algorithms ReliefF and Simba. The derived conclusions give some guidelines for selecting features in practical applications.