Adaptive Margin Research Articles

Pure anomaly detection via self-supervised deep metric learning with adaptive margin

We address the problem of anomaly detection (AD) by a deep network pretrained using self-supervised learning for an auxiliary geometric transformation (GT) classification task. Our key contribution is a novel loss function that augments the standard cross-entropy by an additional term that plays a significant role in the later stages of self-supervised learning. The proposed enabling innovation is a triplet centre loss with an adaptive margin and a learnable metric, which relentlessly drives the GT classes to exhibit continuously improving compactness and inter-class separation. The pretrained network is finetuned for the downstream task using non-anomalous data only, and a GT model for the data is constructed. Anomalies are detected by fusing the output of several decision functions defined using the learnt GT class model. In contrast to the majority of existing methods, our approach strictly adheres to the pure AD design philosophy, which relies on the use of purely non-anomalous data for the design. Extensive experiments on four publicly available AD datasets demonstrate the effectiveness of the proposed contributions and lead to significant performance gains compared to the state-of-the-art (1.8% on F-MNIST, 1.0% on CIFAR-10, 1.2% on CIFAR-100, and 1.7% on CatVsDog). https://github.com/12sf12/Deep-Anomaly-Detection

Affinity Uncertainty-Based Hard Negative Mining in Graph Contrastive Learning.

Hard negative mining has shown effective in enhancing self-supervised contrastive learning (CL) on diverse data types, including graph CL (GCL). The existing hardness-aware CL methods typically treat negative instances that are most similar to the anchor instance as hard negatives, which helps improve the CL performance, especially on image data. However, this approach often fails to identify the hard negatives but leads to many false negatives on graph data. This is mainly due to that the learned graph representations are not sufficiently discriminative due to oversmooth representations and/or non-independent and identically distributed (non-i.i.d.) issues in graph data. To tackle this problem, this article proposes a novel approach that builds a discriminative model on collective affinity information (i.e., two sets of pairwise affinities between the negative instances and the anchor instance) to mine hard negatives in GCL. In particular, the proposed approach evaluates how confident/uncertain the discriminative model is about the affinity of each negative instance to an anchor instance to determine its hardness weight relative to the anchor instance. This uncertainty information is then incorporated into the existing GCL loss functions via a weighting term to enhance their performance. The enhanced GCL is theoretically grounded that the resulting GCL loss is equivalent to a triplet loss with an adaptive margin being exponentially proportional to the learned uncertainty of each negative instance. Extensive experiments on ten graph datasets show that our approach does the following: 1) consistently enhances different state-of-the-art (SOTA) GCL methods in both graph and node classification tasks and 2) significantly improves their robustness against adversarial attacks. Code is available at https://github.com/mala-lab/AUGCL.

Markerless liver online adaptive stereotactic radiotherapy: feasibility analysis

Objective. Radio-opaque markers are recommended for image-guided radiotherapy in liver stereotactic ablative radiotherapy (SABR), but their implantation is invasive. We evaluate in this in-silico study the feasibility of cone-beam computed tomography-guided stereotactic online-adaptive radiotherapy (CBCT-STAR) to propagate the target volumes without implanting radio-opaque markers and assess its consequence on the margin that should be used in that context. Approach. An emulator of a CBCT-STAR-dedicated treatment planning system was used to generate plans for 32 liver SABR patients. Three target volume propagation strategies were compared, analysing the volume difference between the GTVPropagated and the GTVConventional, the vector lengths between their centres of mass (l CoM), and the 95th percentile of the Hausdorff distance between these two volumes (HD95). These propagation strategies were: (1) structure-guided deformable registration with deformable GTV propagation; (2) rigid registration with rigid GTV propagation; and (3) image-guided deformable registration with rigid GTV propagation. Adaptive margin calculation integrated propagation errors, while interfraction position errors were removed. Scheduled plans (PlanNon-adaptive) and daily-adapted plans (PlanAdaptive) were compared for each treatment fraction. Main results. The image-guided deformable registration with rigid GTV propagation was the best propagation strategy regarding to l CoM (mean: 4.3 +/− 2.1 mm), HD95 (mean 4.8 +/− 3.2 mm) and volume preservation between GTVPropagated and GTVConventional. This resulted in a planning target volume (PTV) margin increase (+69.1% in volume on average). Online adaptation (PlanAdaptive) reduced the violation rate of the most important dose constraints (‘priority 1 constraints’, 4.2 versus 0.9%, respectively; p < 0.001) and even improved target volume coverage compared to non-adaptive plans (PlanNon-adaptive). Significance. Markerless CBCT-STAR for liver tumours is feasible using Image-guided deformable registration with rigid GTV propagation. Despite the cost in terms of PTV volumes, daily adaptation reduces constraints violation and restores target volumes coverage.

CoReFace: Sample-guided Contrastive Regularization for Deep Face Recognition

The discriminability of the feature representation is crucial for face recognition. However, previous methods rely solely on the learnable weights of the classification layer, which represent the identities. This reliance could be problematic as the evaluation process depends on the similarity between pairs of face images and requires minimal identity information learned during training. As a result, there is an inconsistency between the training and evaluation processes, which can confuse the feature encoder and hinder the effectiveness of identity-based methods. To address this problem, we propose a novel approach namely Contrastive Regularization for Face Recognition (CoReFace), which applies sample-level regularization in feature representation learning. Specifically, we employ sample-guided contrastive learning to directly regularize the training based on the sample-sample relationship and thus align it with the evaluation process. To avoid image quality degradation, we augment the embeddings instead of the images in order to integrate contrastive learning into face recognition. Additionally, we introduce a new contrastive loss function for the regularization of representation distribution. This function incorporates an adaptive margin and a supervised contrastive mask to ensure stable loss values and prevent interference with the identity supervision signals. Finally, we explore new pair-coupling protocols in order to overcome the problem of semantically repetitive signals in contrastive learning. Extensive experiments demonstrate the efficacy and efficiency of our CoReFace approach, which achieves competitive results compared to state-of-the-art methods. Code could be found https://github.com/IsidoreSong/CoreFace here.

Adapting to climate change accounting for individual beliefs

As the climate changes, efficient climate policy requires a better understanding of how individuals adapt. Despite extensive research on various climate adaptation frictions, including financial and technological constraints, models of adaptive decision-making assume that agents have perfect information and accurate beliefs about climate. Combining rural household data in Bangladesh with a meteorological measure of dryness, this paper studies the role of individual drought beliefs and their accuracy in irrigation decisions as a key adaptive margin. In a theoretical model, I introduce a behavioral friction to document how heterogeneous beliefs differentially influence responsiveness to the same meteorological signal in dryness. The empirical analysis reveals an asymmetric response to dry shocks in irrigation conditional on the accuracy of prior beliefs. A counterfactual analysis shows lower technology adoption levels and higher monetary losses when beliefs are inaccurate.

Zero-shot sketch-based image retrieval via adaptive relation-aware metric learning

Retrieving natural images with the query sketches under the zero-shot scenario is known as zero-shot sketch-based image retrieval (ZS-SBIR). Most of the best-performing methods adapt the triplet loss to learn projections that map natural images and sketches to a latent embedding space. They nevertheless neglect the modality gap between the hand-drawn sketches and the photos and consider no difference between any two incorrect classes, which limits their performance in real use cases. Towards this end, we put forward a simple and effective model, which adopts relation-aware metric learning to suppress the modality gap between the sketches and the photos. We also propose an adaptive margin that utilizes each anchor in embedding space to improve clustering ability in metric learning. Extensive experiments on the Sketchy and TU-Berlin datasets show the dominant position of our proposed model over SOTA competitors.

Spotting the Unseen: Reciprocal Consensus Network Guided by Visual Archetypes

Humans often require only a few visual archetypes to spot novel objects. Based on this observation, we present a strategy rooted in ``spotting the unseen" by establishing dense correspondences between potential query image regions and a visual archetype, and we propose the Consensus Network (CoNet). Our method leverages relational patterns intra and inter images via Auto-Correlation Representation (ACR) and Mutual-Correlation Representation (MCR). Within each image, the ACR module is capable of encoding both local self-similarity and global context simultaneously. Between the query and support images, the MCR module computes the cross-correlation across two image representations and introduces a reciprocal consistency constraint, which can incorporate to exclude outliers and enhance model robustness. To overcome the challenges of low-resource training data, particularly in one-shot learning scenarios, we incorporate an adaptive margin strategy to better handle diverse instances. The experimental results indicate the effectiveness of the proposed method across diverse domains such as object detection in natural scenes, and text spotting in both historical manuscripts and natural scenes, which demonstrates its sparkling generalization ability. Our code is available at: https://github.com/infinite-hwb/conet.

X2-Softmax: Margin adaptive loss function for face recognition

Learning the discriminative features of different faces is an important task in face recognition. By extracting face features with neural networks, it becomes easy to measure the similarity of different face images, which makes face recognition possible. To enhance a neural network’s face feature separability, incorporating an angular margin during training is common practice. The state-of-the-art loss functions CosFace and ArcFace apply fixed margins between the weights of classes to enhance the inter-class separation of face features. Since the distribution of samples in the training set is uneven, the similarities between different identities are unequal. Therefore, using an inappropriately fixed angular margin may lead to problems such as that the model has difficulty converging or that the face features are not sufficiently discriminative. It is more intuitive to use adaptive angular margins are angular adaptive, which can increase as the angles between classes increase. In this paper, we propose a new angular margin loss named X2-Softmax. X2-Softmax loss has adaptive angular margins, which increase as the angle between different classes increases. The angular adaptive margin ensures model flexibility and effectively improves the effect of face recognition. We trained a neural network with X2-Softmax loss on the MS1Mv3 dataset and tested it on several evaluation benchmarks to demonstrate the effectiveness and superiority of our loss function.

SWRM: Similarity Window Reweighting and Margin for Long-Tailed Recognition

Real-world data usually obeys a long-tailed distribution, where a few classes have higher number of samples compared to the other classes. Recent studies have been proposed to alleviate the extreme data imbalance from different perspectives. In this article, we experimentally find that due to the easily confusing visual features between some head- and tail classes, the cross-entropy model is prone to misclassify tail samples to similar head classes. Therefore, to alleviate the influence of the confusion on model performance and improve the classification of tail classes, we propose a Similarity Window Reweighting and Margin (SWRM) algorithm, where the SWRM consists of Similarity Window Reweighting (SWR) and Similarity Window Margin (SWM) algorithms. For the confusable head- and tail classes, SWR assigns larger weights to tail classes and smaller weights to head classes. Therefore, the model can enlarge the importance of tail classes and effectively improve their classification. Moreover, SWR considers the difference in label frequency and the impact of category similarity simultaneously, so that the weight coefficients are more reasonable and efficacious. SWM generates adaptive margins that are proportional to the ratio of the classifier’s weight norm, thus promoting the learning of tail classifier with small weight norm. Our SWRM effectively eliminates the confusion between head- and tail classes and alleviates the misclassification issues. Extensive experiments on three long-tailed datasets, i.e., CIFAR100-LT, ImageNet-LT, and Places-LT, verify our proposed method’s effectiveness and superiority over comparative methods.

Prediction of drug-disease associations based on reinforcement symmetric metric learning and graph convolution network.

Accurately identifying novel indications for drugs is crucial in drug research and discovery. Traditional drug discovery is costly and time-consuming. Computational drug repositioning can provide an effective strategy for discovering potential drug-disease associations. However, the known experimentally verified drug-disease associations is relatively sparse, which may affect the prediction performance of the computational drug repositioning methods. Moreover, while the existing drug-disease prediction method based on metric learning algorithm has achieved better performance, it simply learns features of drugs and diseases only from the drug-centered perspective, and cannot comprehensively model the latent features of drugs and diseases. In this study, we propose a novel drug repositioning method named RSML-GCN, which applies graph convolutional network and reinforcement symmetric metric learning to predict potential drug-disease associations. RSML-GCN first constructs a drug-disease heterogeneous network by integrating the association and feature information of drugs and diseases. Then, the graph convolutional network (GCN) is applied to complement the drug-disease association information. Finally, reinforcement symmetric metric learning with adaptive margin is designed to learn the latent vector representation of drugs and diseases. Based on the learned latent vector representation, the novel drug-disease associations can be identified by the metric function. Comprehensive experiments on benchmark datasets demonstrated the superior prediction performance of RSML-GCN for drug repositioning.

Tackling confusion among actions for action segmentation with adaptive margin and energy-driven refinement

Tackling confusion among actions for action segmentation with adaptive margin and energy-driven refinement

Evaluation of PTV margins with daily iterative online adaptive radiotherapy for postoperative treatment of endometrial and cervical cancer: a prospective single-arm phase 2 study

BackgroundTo determine the optimal planning target volume (PTV) margins for adequate coverage by daily iterative cone-beam computed tomography (iCBCT)-guided online adaptive radiotherapy (oART) in postoperative treatment of endometrial and cervical cancer and the benefit of reducing PTV margins.MethodsFifteen postoperative endometrial and cervical cancer patients treated with daily iCBCT-guided oART were enrolled in this prospective phase 2 study. Pre- and posttreatment iCBCT images of 125 fractions from 5 patients were obtained as a training cohort, and clinical target volumes (CTV) were contoured separately. Uniform three-dimensional expansions were applied to the PTVpre to assess the minimum margin required to encompass the CTVpost. The dosimetric advantages of the proposed online adaptive margins were compared with conventional margin plans (7–15 mm) using an oART emulator in another cohort of 125 iCBCT scans. A CTV-to-PTV expansion was verified on a validation cohort of 253 fractions from 10 patients, and further margin reduction and acute toxicity were studied.ResultsThe average time from pretreatment iCBCT to posttreatment iCBCT was 22 min. A uniform PTV margin of 5 mm could encompass nodal CTVpost in 100% of the fractions (175/175) and vaginal CTVpost in 98% of the fractions (172/175). The margin of 5 mm was verified in our validation cohort, and the nodal PTV margin could be further reduced to 4 mm if ≥ 95% CTV coverage was predicted to be achieved. The adapted plan with a 5 mm margin significantly improved pelvic organ-at-risk dosimetry compared with the conventional margin plan. Grade 3 toxicities were observed in only one patient with leukopenia, and no patients experienced acute urinary toxicity.ConclusionIn the postoperative treatment of endometrial and cervical cancer, oART could reduce PTV margins to 5 mm, which significantly decrease the dose to critical organs at risk and potentially lead to a lower incidence of acute toxicity.

Multi-Level Feature Representation Framework With Adaptive Margin Loss for Few-Shot Sonar Images Classification of AUVs.

Multi-Level Feature Representation Framework With Adaptive Margin Loss for Few-Shot Sonar Images Classification of AUVs.

Deep Boosting Learning: A Brand-new Cooperative Approach for Image-Text Matching.

Image-text matching remains a challenging task due to heterogeneous semantic diversity across modalities and insufficient distance separability within triplets. Different from previous approaches focusing on enhancing multi-modal representations or exploiting cross-modal correspondence for more accurate retrieval, in this paper we aim to leverage the knowledge transfer between peer branches in a boosting manner to seek a more powerful matching model. Specifically, we propose a brand-new Deep Boosting Learning (DBL) algorithm, where an anchor branch is first trained to provide insights into the data properties, with a target branch gaining more advanced knowledge to develop optimal features and distance metrics. Concretely, an anchor branch initially learns the absolute or relative distance between positive and negative pairs, providing a foundational understanding of the particular network and data distribution. Building upon this knowledge, a target branch is concurrently tasked with more adaptive margin constraints to further enlarge the relative distance between matched and unmatched samples. Extensive experiments validate that our DBL can achieve impressive and consistent improvements based on various recent state-of-the-art models in the image-text matching field, and outperform related popular cooperative strategies, e.g., Conventional Distillation, Mutual Learning, and Contrastive Learning. Beyond the above, we confirm that DBL can be seamlessly integrated into their training scenarios and achieve superior performance under the same computational costs, demonstrating the flexibility and broad applicability of our proposed method.

Deep Adaptive Quadruplet Hashing With Probability Sampling for Large-Scale Image Retrieval

With the preferable efficiency in storage and computation, hashing has shown potential application in large-scale multimedia retrieval. Compared with traditional hashing algorithms using hand-crafted characteristics, deep hashing inherits the representational capacity of deep neural networks to jointly learn semantic features and hash functions, encoding raw data into compact binary codes with significant discrimination. Generally, most of the current multi-wise hashing methods view the similarity margins between image pairs as constant values in training process. When the distance between sample pairs exceeds the fixed margin, the hashing network would not learn anything. Besides, available hashing methods commonly introduce the random sampling strategy to build training batches and ignore the sample distribution, which is harmful to parameter optimization. In this paper, we propose a novel Deep Adaptive Quadruplet Hashing with probability sampling (DAQH) for discriminative binary code learning. Specifically, with exploring the distribution relationship of raw samples, a non-uniform probability sampling strategy is proposed to build more informative and representative training batches, while maintaining the diversity of training samples. By introducing the prior similarity of sample pairs to calculate corresponding margins, an adaptive margin quadruplet loss is designed to dynamically preserve the underlying semantic relationships with its neighbors. To tune the attributes of binary codes, by combining quadruple regularization and orthogonality optimization, binary code constraint is developed to make the learned embedding with significant discrimination. Extensive experimental results on various benchmark datasets demonstrate our proposed DAQH framework achieves state-of-the-art visual similarity search performance.

Multi-level knowledge-driven feature representation and triplet loss optimization network for image–text retrieval

Image–text retrieval plays a considerable role in associating vision and language. Existing mainstream approaches focus on fine-grained alignment while ignoring the influence of prior knowledge on model performance and the limitation of using a fixed margin in the triplet loss. In this paper, we propose a Multi-level Knowledge-driven feature representation and Triplet Loss Optimization Network (MKTLON) that exploits prior knowledge to enhance visual and textual feature representations and utilizes an adaptive margin of the triplet loss to optimize network training. Specifically, we first present an Enhanced feature Representation scheme based on the Self-Attention (ERSA) module, which incorporates the prior knowledge randomly initialized by uniform distribution into the matrices K and V in the self-attention mechanism. Subsequently, we adopt cascaded ERSA modules to encode images and texts to obtain multi-level visual and textual features with prior knowledge. Furthermore, we develop an adaptive margin optimization strategy that models the relevance scores of positive and negative samples as two independent Gaussian distributions, and obtain the optimized margin by minimizing the intersection of these two distributions. Extensive experiments on two benchmarks, Flickr30K (155,000 image–text pairs) and MSCOCO (616,435 image–text pairs), show the proposed MKTLON achieves 5.7% and 4.3% improvements on rSum, respectively, compared to the state-of-the-art method. The source code will be released at https://github.com/FlyCuteBird/MKTLON.

HAMFace: Hardness adaptive margin loss for face recognition with various intra-class variations

The boost of convolutional neural networks (CNNs) has promoted the development of face recognition. Recently, the emergence of margin-based loss functions has further significantly improved the performance of face recognition. However, these methods sharply degrade in performance when dealing with large intra-class variations, including age, pose, illumination, resolution, and occlusion. Unlike most methods that target specific variations, our proposed approach, HAMFace, addresses the problems uniformly from the perspective of hard positive examples. To mitigate the intra-class variance, we argue that hard positive examples prefer larger margins, which can push them closer to their corresponding class centers. First, we design a hardness adaptive margin function to adjust the margin according to the hardness of the hard positive examples. Then, to enhance performance for unconstrained face recognition with various intra-class variations, we introduce a novel loss function named Hardness Adaptive Margin (HAM) Softmax Loss. This loss function allocates larger margins to hard positive examples during training based on their level of hardness. The proposed HAMFace is evaluated on nine challenging face recognition benchmarks and exhibits its superiority compared with other state-of-the-arts.

An adaptive self-correction joint training framework for person re-identification with noisy labels

Current person re-identification (ReID) methods heavily rely on well-annotated training data, and their performance suffers from significant degradation in the presence of noisy labels that are ubiquitous in real-life scenes. The reason is that noisy labels not only affect the prediction results of the classifier, but also impede feature refinement, making it difficult to distinguish between different person features. To address these issues, we propose an Adaptive Self-correction Classification (ASC) loss and an Adaptive Margin Self-correction Triplet (AMSTri) loss. Specifically, ASC loss helps the network to produce better predictions by balancing annotations and prediction labels, and pays more attention to the minority samples with the help of a focusing factor. On the other hand, the AMSTri loss introduces an adaptive margin that varies with sample features to accommodate complex data variations, and utilizes predicted labels to generate reliable triples for feature refinement. We then present an end-to-end adaptive self-correction joint training framework incorporating ASC loss and AMSTri loss to achieve a robust ReID model. Our comprehensive experiments demonstrate that the proposed framework outperforms most existing counterparts.

Improving adversarial robustness of deep neural networks via adaptive margin evolution

Adversarial training is the most popular and general strategy to improve Deep Neural Network (DNN) robustness against adversarial noises. Many adversarial training methods have been proposed in the past few years. However, most of these methods are highly susceptible to hyperparameters, especially the training noise upper bound. Tuning these hyperparameters is expensive and difficult for people not in the adversarial robustness research domain, which prevents adversarial training techniques from being used in many application fields. In this study, we propose a new adversarial training method, named Adaptive Margin Evolution (AME). Besides being hyperparameter-free for the user, our AME method places adversarial training samples into the optimal locations in the input space by gradually expanding the exploration range with self-adaptive and gradient-aware step sizes. We evaluate AME and the other seven well-known adversarial training methods on three common benchmark datasets (CIFAR10, SVHN, and Tiny ImageNet) under the most challenging adversarial attack: AutoAttack. The results show that: (1) On the three datasets, AME has the best overall performance; (2) On the Tiny ImageNet dataset, which is much more challenging, AME has the best performance at every noise level. Our work may pave the way for adopting adversarial training techniques in application domains where hyperparameter-free methods are preferred.

JAMsFace: joint adaptive margins loss for deep face recognition

Deep feature learning has become crucial in large-scale face recognition, and margin-based loss functions have demonstrated impressive success in this field. These methods aim to enhance the discriminative power of the softmax loss by increasing the feature margin between different classes. These methods assume class balance, where a fixed margin is sufficient to squeeze intra-class variation equally. However, real-face datasets often exhibit imbalanced classes, where the fixed margin is suboptimal, limiting the discriminative power and generalizability of the face recognition model. Furthermore, margin-based approaches typically focus on enhancing discrimination either in the angle or cosine space, emphasizing one boundary while disregarding the other. To overcome these limitations, we propose a joint adaptive margins loss function (JAMsFace) that learns class-related margins for both angular and cosine spaces. This approach allows adaptive margin penalties to adjust adaptively for different classes. We explain and analyze the proposed JAMsFace geometrically and present comprehensive experiments on multiple face recognition benchmarks. The results show that JAMsFace outperforms existing face recognition losses in mainstream face recognition tasks. Specifically, JAMsFace advances the state-of-the-art face recognition performance on LFW, CPLFW, and CFP-FP and achieves comparable results on CALFW and AgeDB-30. Furthermore, for the challenging IJB-B and IJB-C benchmarks, JAMsFace achieves impressive true acceptance rates (TARs) of 89.09% and 91.81% at a false acceptance rate (FAR) of 1e-4, respectively.