Negative Pairs Research Articles

Tractography-Based Automated Identification of Retinogeniculate Visual Pathway With Novel Microstructure-Informed Supervised Contrastive Learning.

The retinogeniculate visual pathway (RGVP) is responsible for carrying visual information from the retina to the lateral geniculate nucleus. Identification and visualization of the RGVP are important in studying the anatomy of the visual system and can inform the treatment of related brain diseases. Diffusion MRI (dMRI) tractography is an advanced imaging method that uniquely enables invivo mapping of the 3D trajectory of the RGVP. Currently, identification of the RGVP from tractography data relies on expert (manual) selection of tractography streamlines, which is time-consuming, has high clinical and expert labor costs, and is affected by inter-observer variability. In this paper, we present a novel deep learning framework, DeepRGVP, to enable fast and accurate identification of the RGVP from dMRI tractography data. We design a novel microstructure-informed supervised contrastive learning method that leverages both streamline label and tissue microstructure information to determine positive and negative pairs. We propose a new streamline-level data augmentation method to address highly imbalanced training data, where the number of RGVP streamlines is much lower than that of non-RGVP streamlines. In the experiments, we perform comparisons with several state-of-the-art deep learning methods that were designed for tractography parcellation. Furthermore, to assess the generalizability of the proposed RGVP method, we apply our method to dMRI tractography data from neurosurgical patients with pituitary tumors. In comparison with the state-of-the-art methods, we show superior RGVP identification results using DeepRGVP with significantly higher accuracy and F1 scores. In the patient data experiment, we show DeepRGVP can successfully identify RGVPs despite the effect of lesions affecting the RGVPs. Overall, our study shows the high potential of using deep learning to automatically identify the RGVP.

RC-DETR: Improving DETRs in crowded pedestrian detection via rank-based contrastive learning

The variants of DEtection TRansformer (DETRs) have achieved impressive performance in general object detection. However, they suffer notable performance degradation in scenarios involving crowded pedestrian detection. This decline primarily occurs during the training phase, where DETRs are constrained solely by pedestrian labels. This limitation leads to the production of indistinguishable image features between visually similar pedestrians and background elements, resulting in incorrect detections. To address this issue, this paper introduces a rank-based contrastive learning method, which constructs an additional and specific constraint for each indistinguishable training sample to produce distinguishable image features. Unlike previous methods that rely solely on pedestrian labels to achieve a consistent confidence score, our approach relies on multiple constraints and aims to ensure the correct rank of detection results, with confidence scores of pedestrians consistently surpassing those of background elements. Specifically, we first filter out some training samples that could interfere with our delineation of indistinguishable and distinguishable training samples. Then, based on the confidence score rank, we divide the rest of the training samples into distinguishable positive and negative training samples and indistinguishable positive and negative training samples. Finally, we combine these training samples into multiple positive and negative pairs and utilize these sample pairs to train DETRs via contrastive learning. Our method can be plugged into any DETRs and does not increase any overhead on inference. Extensive experiments on three DETRs show that our method achieves superior performance. Especially on the Crowdhuman dataset, our method achieved the state-of-the-art 38.9% MR.

Consistent positive correlation sample distribution: Alleviating the negative sample noise issue in contrastive adaptation

Contrastive learning has been widely applied in domain adaptation tasks due to its powerful representation learning capabilities in recent years. Constructing positive and negative sample pairs through different augmented views of instances promotes the effective application of contrastive learning in domain adaptation. However, most existing methods are based on Noise Contrastive Estimation (NCE), which primarily focuses on instance-level information and lacks attention to the relationships between different instances. These methods treat all negative samples as noise interference, leading to an increased distance between negative samples and anchor samples, disregarding instances in the negative samples that share the same category as the anchor sample. Consequently, these methods do not offer significant advantages in domain adaptation for image classification tasks. To address this issue, we propose a method called Consistent Positive Correlation Sample Distribution (CPCSD) to mitigate the problem of class collision among negative samples by leveraging the semantic similarity between instances. We introduce a positive correlation distribution loss within the contrastive adaptation framework to compute the similarity distribution between batches of augmented views and align them. Additionally, we sharpen the target similarity distribution to obtain a more emphasized relationship distribution, thereby alleviating the issue of negative sample noise in contrastive adaptation. Extensive experiments demonstrate that our proposed model has significant advantages in contrastive adaptation algorithms and improves the performance of downstream domain adaptation tasks.

Towards accurate anomaly detection for cloud system via graph-enhanced contrastive learning

As a critical technology, anomaly detection ensures the smooth operation of cloud systems while maintaining the market competitiveness of cloud service providers. However, the resource data in real-world cloud systems is predominantly unannotated, leading to insufficient supervised signals for anomaly detection. Moreover, complicated topological associations existed between cloud servers (e.g., computation, storage, and communication). While acquiring resource information, correlating the system topology is challenging. To this end, we propose the GCAD for cloud system anomaly detection, which integrates data augmentation, GraphGRU, contrastive learning, and reconstruction. First, GCAD constructs positive and negative sample pairs through the masking and Gaussian noise data augmentation. Then, the GraphGRU processes extended temporal graph data, extracting and fusing spatiotemporal features from resource status and system topology. In addition, GCAD introduces linear attention for encoding spatiotemporal representations to capture their global correlation information. The weight parameters of the encoder are optimized using a contrastive learning mechanism. Finally, GCAD utilizes a reconstruction technique to calculate anomaly scores, facilitating the evaluation of the state of the cloud system at each time point. Experimental results indicate that GCAD outperforms state-of-the-art compared methods on two real-world datasets that contain topology information.

Integration of mRNA and miRNA Analysis Sheds New Light on the Muscle Response to Heat Stress in Spotted Sea Bass (Lateolabrax maculatus).

Temperature is a crucial environmental factor for fish. Elevated temperatures trigger various physiological and molecular responses designed to maintain internal environmental homeostasis and ensure the proper functioning of the organism. In this study, we measured biochemical parameters and performed mRNA-miRNA integrated transcriptomic analysis to characterize changes in gene expression profiles in the muscle tissue of spotted sea bass (Lateolabrax maculatus) under heat stress. The measurement of biochemical parameters revealed that the activities of nine biochemical enzymes (ALP, γ-GT, AST, GLU, CK, ALT, TG, LDH and TC) were significantly affected to varying degrees by elevated temperatures. A total of 1940 overlapping differentially expressed genes (DEGs) were identified among the five comparisons in the muscle tissue after heat stress. Protein-protein interaction (PPI) analysis of DEGs indicated that heat shock protein genes (HSPs) were deeply involved in the response to heat stress. In addition, we detected 462 differential alternative splicing (DAS) events and 618 DAS genes, which are closely associated with sarcomere assembly in muscle, highlighting the role of alternative splicing in thermal response regulation. Moreover, 32 differentially expressed miRNAs (DEMs) were identified in response to heat stress, and 599 DEGs were predicted as potential target genes of those DEMs, generating 846 DEG-DEM negative regulatory pairs potentially associated with thermal response. Function enrichment analysis of the target genes suggested that lipid metabolism-related pathways and genes were regulated by miRNAs. By analyzing PPIs of target genes, we identified 28 key negative regulatory pairs, including 13 miRNAs (such as lma-miR-122, lma-miR-200b-5p and novel-miR-444) and 15 target genes (such as hspa13, dnaja1, and dnajb1a). This study elucidates the molecular mechanisms of response to high-temperature stress and offers valuable information for the selection and breeding of heat-tolerant strains of spotted sea bass.

Boosting Graph Contrastive Learning via Adaptive Sampling.

Contrastive learning (CL) is a prominent technique for self-supervised representation learning, which aims to contrast semantically similar (i.e., positive) and dissimilar (i.e., negative) pairs of examples under different augmented views. Recently, CL has provided unprecedented potential for learning expressive graph representations without external supervision. In graph CL, the negative nodes are typically uniformly sampled from augmented views to formulate the contrastive objective. However, this uniform negative sampling strategy limits the expressive power of contrastive models. To be specific, not all the negative nodes can provide sufficiently meaningful knowledge for effective contrastive representation learning. In addition, the negative nodes that are semantically similar to the anchor are undesirably repelled from it, leading to degraded model performance. To address these limitations, in this article, we devise an adaptive sampling strategy termed "AdaS." The proposed AdaS framework can be trained to adaptively encode the importance of different negative nodes, so as to encourage learning from the most informative graph nodes. Meanwhile, an auxiliary polarization regularizer is proposed to suppress the adverse impacts of the false negatives and enhance the discrimination ability of AdaS. The experimental results on a variety of real-world datasets firmly verify the effectiveness of our AdaS in improving the performance of graph CL.

Negative-Free Self-Supervised Gaussian Embedding of Graphs

Graph Contrastive Learning (GCL) has recently emerged as a promising graph self-supervised learning framework for learning discriminative node representations without labels. The widely adopted objective function of GCL benefits from two key properties: alignment and uniformity, which align representations of positive node pairs while uniformly distributing all representations on the hypersphere. The uniformity property plays a critical role in preventing representation collapse and is achieved by pushing apart augmented views of different nodes (negative pairs). As such, existing GCL methods inherently rely on increasing the quantity and quality of negative samples, resulting in heavy computational demands, memory overhead, and potential class collision issues. In this study, we propose a negative-free objective to achieve uniformity, inspired by the fact that points distributed according to a normalized isotropic Gaussian are uniformly spread across the unit hypersphere. Therefore, we can minimize the distance between the distribution of learned representations and the isotropic Gaussian distribution to promote the uniformity of node representations. Our method also distinguishes itself from other approaches by eliminating the need for a parameterized mutual information estimator, an additional projector, asymmetric structures, and, crucially, negative samples. Extensive experiments over seven graph benchmarks demonstrate that our proposal achieves competitive performance with fewer parameters, shorter training times, and lower memory consumption compared to existing GCL methods.

Classification of breast cancer histopathology images using a modified supervised contrastive learning method.

Deep neural networks have reached remarkable achievements in medical image processing tasks, specifically in classifying and detecting various diseases. However, when confronted with limited data, these networks face a critical vulnerability, often succumbing to overfitting by excessively memorizing the limited information available. This work addresses the challenge mentioned above by improving the supervised contrastive learning method leveraging both image-level labels and domain-specific augmentations to enhance model robustness. This approach integrates self-supervised pre-training with a two-stage supervised contrastive learning strategy. In the first stage, we employ a modified supervised contrastive loss that not only focuses on reducing false negatives but also introduces an elimination effect to address false positives. In the second stage, a relaxing mechanism is introduced that refines positive and negative pairs based on similarity, ensuring that only relevant image representations are aligned. We evaluate our method on the BreakHis dataset, which consists of breast cancer histopathology images, and demonstrate an increase in classification accuracy by 1.45% in the image level, compared to the state-of-the-art method. This improvement corresponds to 93.63% absolute accuracy, highlighting the effectiveness of our approach in leveraging properties of data to learn more appropriate representation space. The code implementation of this study is accessible on GitHub https://github.com/matinamehdizadeh/Breast-Cancer-Detection .

Self-supervised learning from images: No negative pairs, no cluster-balancing

Learning with self-derived targets provides a non-contrastive method for unsupervised image representation learning, where the variety in targets is crucial. Recent work has achieved good performance by learning with targets obtained via cluster-balancing. However, the equal-cluster-size constraint becomes too restrictive for handling data with imbalanced categories or coming in small batches. In this paper, we propose a new clustering-based approach for non-contrastive image representation learning with no need for a particular architecture design or extra memory bank and no explicit constraints on cluster size. A key formulation is to learn embedding consistency and variable decorrelation in the cluster space by tweaking the batch-wise cross-correlation matrix towards an identity one. With this identitization loss incorporated, predicted cluster assignments of two randomly augmented views of the same image serve as targets for each other. We carried out comprehensive experimental studies of linear classification with learned representations of benchmark image datasets. Our results show that the proposed approach significantly outperforms state-of-the-art approaches and is more robust to class imbalance than those with cluster balancing.

Enhancing aspect-based sentiment analysis with linking words-guided emotional augmentation and hybrid learning

Aspect-based sentiment analysis (ABSA) is a sophisticated task in the field of natural language processing that aims to identify emotional tendencies related to specific aspects of text. However, ABSA often faces significant data shortages, which limit the availability of annotated data for training and affects the robustness of models. Moreover, when a text contains multiple emotional dimensions, these dimensions can interact, complicating the judgments of emotional polarity. In response to these challenges, this study proposes an innovative training framework: Linking words-guided multidimensional emotional data augmentation and adversarial contrastive training (LWEDA-ACT). Specifically, this method alleviates the issue of data scarcity by synthesizing additional training samples using four different text generators. To obtain the most representative samples, we selected them by calculating sentence entropy. Meanwhile, to reduce potential noise, we introduced linking words to ensure text coherence. Additionally, by applying adversarial training, the model is able to learn generalized feature representations to handle minor input perturbations, thereby enhancing its robustness and accuracy in complex emotional dimension interactions. Through contrastive learning, we constructed positive and negative sample pairs, enabling the model to more accurately identify and distinguish the sentiment polarity of different aspect terms. We conducted comprehensive experiments on three popular ABSA datasets, namely Restaurant, Laptop, and Twitter, and compared our method against the current state-of-the-art techniques. The experimental results demonstrate that our approach achieved an accuracy improvement of +0.98% and a macro F1 score increase of +0.52% on the Restaurant dataset. Additionally, on the challenging Twitter dataset, our method improved accuracy by +0.77% and the macro F1 score by +1.14%.

Dual-level adaptive incongruity-enhanced model for multimodal sarcasm detection

Multimodal sarcasm detection leverages multimodal information, such as image, text, etc. to identify special instances whose superficial emotional expression is contrary to the actual emotion. Existing methods primarily focused on the incongruity between text and image information for sarcasm detection. Existing sarcasm methods in which the tendency of image encoders to encode similar images into similar vectors, and the introduction of noise in graph-level feature extraction due to negative correlations caused by the accumulation of GAT layers and the lack of representations for non-neighboring nodes. To address these limitations, we propose a Dual-Level Adaptive Incongruity-Enhanced Model (DAIE) to extract the incongruity between the text and image at both token and graph levels. At the token level, we bolster token-level contrastive learning with patch-based reconstructed image to capture common and specific features of images, thereby amplifying incongruities between text and images. At the graph level, we introduce adaptive graph contrast learning, coupled with negative pair similarity weights, to refine the feature representation of the model’s textual and visual graph nodes, while also enhancing the information exchange among neighboring nodes. We conduct experiments using a publicly available sarcasm detection dataset. The results demonstrate the effectiveness of our method, outperforming several state-of-the-art approaches by 3.33% and 4.34% on accuracy and F1 score, respectively.

Bootstrap each lead’s latent: A novel method for self-supervised learning of multilead electrocardiograms

Background and Objective:Electrocardiogram (ECG) is one of the most important diagnostic tools for cardiovascular diseases (CVDs). Recent studies show that deep learning models can be trained using labeled ECGs to achieve automatic detection of CVDs, assisting cardiologists in diagnosis. However, the deep learning models heavily rely on labels in training, while manual labeling is costly and time-consuming. This paper proposes a new self-supervised learning (SSL) method for multilead ECGs: bootstrap each lead’s latent (BELL) to reduce the reliance and boost model performance in various tasks, especially when training data are insufficient. Method:BELL is a variant of the well-known bootstrap your own latent (BYOL). The BELL aims to learn prior knowledge from unlabeled ECGs by pretraining, benefitting downstream tasks. It leverages the characteristics of multilead ECGs. First, BELL uses the multiple-branch skeleton, which is more effective in processing multilead ECGs. Moreover, it proposes intra-lead and inter-lead mean square error (MSE) to guide pretraining, and their fusion can result in better performances. Additionally, BELL inherits the main advantage of the BYOL: No negative pair is used in pretraining, making it more efficient. Results:In most cases, BELL surpasses previous works in the experiments. More importantly, the pretraining improves model performances by 0.69% ∼ 8.89% in downstream tasks when only 10% of training data are available. Furthermore, BELL shows excellent adaptability to uncurated ECG data from a real-world hospital. Only slight performance degradation occurs (<1% in most cases) when using these data. Conclusion:The results suggest that the BELL can alleviate the reliance on manual ECG labels from cardiologists, a critical bottleneck of the current deep learning-based models. In this way, the BELL can also help deep learning extend its application on automatic ECG analysis, reducing the cardiologists’ burden in real-world diagnosis.

Unsupervised multi-source domain adaptation via contrastive learning for EEG classification

Individual differences in electroencephalography (EEG) present significant challenges for subject-independent EEG classification in brain–computer interfaces (BCIs). Existing domain adaptation methods often address individual differences by merging all source domains indistinguishably into a single source and aligning features between this aggregate source and the target domain. Neglecting the relationships between different source domains would hinder the model’s adaptability and generalization. Therefore, we propose a method called Contrastive Learning-based Unsupervised multi-source Domain Adaptation (CLUDA) for learning subject-independent representations in motor imagery. Our method not only aligns the conditional distributions of each source domain with the target domain but also reduces discrepancies among the source domains using contrastive learning, thus learning more generalized domain-invariant representations. Specifically, CLUDA effectively eliminates semantic variances by maximizing the similarity between positive pairs (same class) and minimizing the similarity between negative pairs (different classes) across subjects. Finally, we validated CLUDA on four motor imagery datasets and consistently achieved state-of-the-art performance.

Drug-target interaction prediction with collaborative contrastive learning and adaptive self-paced sampling strategy

BackgroundDrug-target interaction (DTI) prediction plays a pivotal role in drug discovery and drug repositioning, enabling the identification of potential drug candidates. However, most previous approaches often do not fully utilize the complementary relationships among multiple biological networks, which limits their ability to learn more consistent representations. Additionally, the selection strategy of negative samples significantly affects the performance of contrastive learning methods.ResultsIn this study, we propose CCL-ASPS, a novel deep learning model that incorporates Collaborative Contrastive Learning (CCL) and Adaptive Self-Paced Sampling strategy (ASPS) for drug-target interaction prediction. CCL-ASPS leverages multiple networks to learn the fused embeddings of drugs and targets, ensuring their consistent representations from individual networks. Furthermore, ASPS dynamically selects more informative negative sample pairs for contrastive learning. Experiment results on the established dataset demonstrate that CCL-ASPS achieves significant improvements compared to current state-of-the-art methods. Moreover, ablation experiments confirm the contributions of the proposed CCL and ASPS strategies.ConclusionsBy integrating Collaborative Contrastive Learning and Adaptive Self-Paced Sampling, the proposed CCL-ASPS effectively addresses the limitations of previous methods. This study demonstrates that CCL-ASPS achieves notable improvements in DTI predictive performance compared to current state-of-the-art approaches. The case study and cold start experiments further illustrate the capability of CCL-ASPS to effectively predict previously unknown DTI, potentially facilitating the identification of new drug-target interactions.

AFSC: A self-supervised augmentation-free spatial clustering method based on contrastive learning for identifying spatial domains

Recent research in spatial transcriptomics allows researchers to analyze gene expression without losing spatial information. Spatial information can assist in cell communication, identification of new cell subtypes, which provides important research methods for multiple fields such as microenvironment interactions and pathological processes of diseases. Identifying spatial domains is an important step in spatial transcriptomics analysis, and improving spatial clustering methods can benefit for identifying spatial domains. In addition to eliminating noise in original gene expression, how to use spatial information to assist clustering has also become a new problem. A variety of calculating methods have been applied to spatial clustering, including contrastive learning methods. However, existing spatial clustering methods based on contrastive learning use data augmentation to generate positive and negative pairs, which will inevitably destroy the biological meaning of the data. We propose a new self-supervised spatial clustering method based on contrastive learning, Augmentation-Free Spatial Clustering (AFSC), which integrates spatial information and gene expression to learn latent representations. We construct a contrastive learning module without negative pairs or data augmentation by designing Teacher and Student Encoder. We also design an unsupervised clustering module to make clustering and contrastive learning be trained together. Experiments on multiple spatial transcriptomics datasets at different resolutions demonstrate that our method performs well in self-supervised spatial clustering tasks. Furthermore, the learned representations can be used for various downstream tasks including visualization and trajectory inference.

Cost-Sensitive Weighted Contrastive Learning Based on Graph Convolutional Networks for Imbalanced Alzheimer's Disease Staging.

Identifying the progression stages of Alzheimer's disease (AD) can be considered as an imbalanced multi-class classification problem in machine learning. It is challenging due to the class imbalance issue and the heterogeneity of the disease. Recently, graph convolutional networks (GCNs) have been successfully applied in AD classification. However, these works did not handle the class imbalance issue in classification. Besides, they ignore the heterogeneity of the disease. To this end, we propose a novel cost-sensitive weighted contrastive learning method based on graph convolutional networks (CSWCL-GCNs) for imbalanced AD staging using resting-state functional magnetic resonance imaging (rs-fMRI). The proposed method is developed on a multi-view graph constructed by the functional connectivity (FC) and high-order functional connectivity (HOFC) features of the subjects. A novel cost-sensitive weighted contrastive learning procedure is proposed to capture discriminative information from the minority classes, encouraging the samples in the minority class to provide adequate supervision. Considering the heterogeneity of the disease, the weights of the negative pairs are introduced into contrastive learning and they are computed based on the distance to class prototypes, which are automatically learned from the training data. Meanwhile, the cost-sensitive mechanism is further introduced into contrastive learning to handle the class imbalance issue. The proposed CSWCL-GCN is evaluated on 720 subjects (including 184 NCs, 40 SMC patients, 208 EMCI patients, 172 LMCI patients and 116 AD patients) from the ADNI (Alzheimer's Disease Neuroimaging Initiative). Experimental results show that the proposed CSWCL-GCN outperforms state-of-the-art methods on the ADNI database.

DCANet: Dense Convolutional Attention Network for infrared small target detection

Infrared small target detection and tracking technology plays a essential role in systems like infrared warning, guidance, and missile defense. Efficiently and reliably detecting fast-moving, small targets is a key technological challenge. This study introduces a dense convolutional attention network as a potential solution to address the identified challenge. This network incorporates a dense convolutional interaction module that effectively minimizes information loss across different levels. Additionally, a simple, parameter-free attention module is introduced to adaptively extract features and enhance the emphasis on important information. Furthermore, a dense pixel contrastive learning module is introduced to capture more features of targets with similar sizes by constructing positive and negative sample pairs and designing a pixel contrastive loss function. Experimental validation was conducted on four representative datasets, demonstrating the success of our approach in terms of performance superiority over other methods.

Adjacent Image Augmentation and Its Framework for Self-Supervised Learning in Anomaly Detection.

Anomaly detection has gained significant attention with the advancements in deep neural networks. Effective training requires both normal and anomalous data, but this often leads to a class imbalance, as anomalous data is scarce. Traditional augmentation methods struggle to maintain the correlation between anomalous patterns and their surroundings. To address this, we propose an adjacent augmentation technique that generates synthetic anomaly images, preserving object shapes while distorting contours to enhance correlation. Experimental results show that adjacent augmentation captures high-quality anomaly features, achieving superior AU-ROC and AU-PR scores compared to existing methods. Additionally, our technique produces synthetic normal images, aiding in learning detailed normal data features and reducing sensitivity to minor variations. Our framework considers all training images within a batch as positive pairs, pairing them with synthetic normal images as positive pairs and with synthetic anomaly images as negative pairs. This compensates for the lack of anomalous features and effectively distinguishes between normal and anomalous features, mitigating class imbalance. Using the ResNet50 network, our model achieved perfect AU-ROC and AU-PR scores of 100% in the bottle category of the MVTec-AD dataset. We are also investigating the relationship between anomalous pattern size and detection performance.

Domain-Adaptive Framework for ACL Injury Diagnosis Utilizing Contrastive Learning Techniques

In sports medicine, anterior cruciate ligament (ACL) injuries are common and have a major effect on knee joint stability. For the sake of prognosis evaluation and treatment planning, an accurate clinical auxiliary diagnosis of ACL injuries is essential. Although existing deep learning techniques for ACL diagnosis work well on single datasets, research on cross-domain data transfer is still lacking. Building strong domain-adaptive diagnostic models requires addressing domain disparities in ACL magnetic resonance imaging (MRI) from different hospitals and making efficient use of multiple ACL datasets. This work uses the publicly available KneeMRI dataset from Croatian hospitals coupled with the publicly available MRnet dataset from Stanford University to investigate domain adaptation and transfer learning models. First, an optimized model efficiently screens training data in the source domain to find unusually misclassified occurrences. Subsequently, before being integrated into the contrastive learning module, a target domain feature extraction module processes features of target domain samples to improve extraction efficiency. By using contrastive learning between positive and negative sample pairs from source and target domains, this method makes domain adaptation easier and improves the efficacy of ACL auxiliary diagnostic models. Utilizing a spatially augmented ResNet-18 backbone network, the suggested approach produces notable enhancements in experimentation. To be more precise, the AUC for transfer learning improved by 3.5% from MRnet to KneeMRI and by 2.5% from KneeMRI to MRnet (from 0.845 to 0.870). This method shows how domain transfer can be used to improve diagnostic accuracy on a variety of datasets and effectively progresses the training of a strong ACL auxiliary diagnostic model.

Enhancing clustering representations with positive proximity and cluster dispersion learning

Contemporary deep clustering approaches often rely on contrastive or non-contrastive techniques to acquire effective representations for clustering tasks. Contrastive methods apply negative pairs to achieve homogenous representations but can introduce class collision problems, potentially compromising clustering performance. In contrast, non-contrastive techniques prevent class collisions but may produce non-uniform representations that lead to clustering collapse. In this work, we propose a novel end-to-end deep clustering approach named PIPCDR, designed to harness the strengths of both approaches while mitigating their limitations. PIPCDR incorporates a positive instance proximity loss and a cluster dispersion regularizer. The positive instance proximity loss ensures alignment between augmented views of instances and their sampled neighbors, enhancing within-cluster compactness by selecting genuinely positive pairs within the embedding space. Meanwhile, the cluster dispersion regularizer maximizes inter-cluster distances while minimizing within-cluster compactness, promoting uniformity in the learned representations. PIPCDR excels in producing well-separated clusters, generating uniform representations, avoiding class collision issues, and enhancing within-cluster compactness. We extensively validate the effectiveness of PIPCDR within an end-to-end Majorize-Minimization framework, demonstrating its competitive performance on moderate-scale clustering benchmark datasets and establishing new state-of-the-art results on large-scale datasets.