Contrastive Learning Research Articles

Graph Contrastive Learning With Adaptive Proximity-Based Graph Augmentation.

Graph neural networks (GNNs) have been successful in a variety of graph-based applications. Recently, it is shown that capturing long-range relationships between nodes helps improve the performance of GNNs. The phenomenon is mostly confirmed in a supervised learning setting. In this article, inspired by contrastive learning (CL), we propose an unsupervised learning pipeline, in which different types of long-range similarity information are injected into the GNN model in an efficient way. We reconstruct the original graph in feature and topology spaces to generate three augmented views. During training, our model alternately picks an augmented view, and maximizes an agreement between the representations of the view and the original graph. Importantly, we identify the issue of diminishing utility of the augmented views as the model gradually learns useful information from the views. Hence, we propose a view update scheme that adaptively adjusts the augmented views, so that the views can continue to provide new information that helps with CL. The updated augmented views and the original graph are jointly used to train a shared GNN encoder by optimizing an efficient channel-level contrastive objective. We conduct extensive experiments on six assortative graphs and three disassortative graphs, which demonstrate the effectiveness of our method.

Description-Enhanced Label Embedding Contrastive Learning for Text Classification.

Text classification is one of the fundamental tasks in natural language processing, which requires an agent to determine the most appropriate category for input sentences. Recently, deep neural networks have achieved impressive performance in this area, especially pretrained language models (PLMs). Usually, these methods concentrate on input sentences and corresponding semantic embedding generation. However, for another essential component: labels, most existing works either treat them as meaningless one-hot vectors or use vanilla embedding methods to learn label representations along with model training, underestimating the semantic information and guidance that these labels reveal. To alleviate this problem and better exploit label information, in this article, we employ self-supervised learning (SSL) in model learning process and design a novel self-supervised relation of relation ( [Formula: see text]) classification task for label utilization from a one-hot manner perspective. Then, we propose a novel relation of relation learning network( [Formula: see text]-Net) for text classification, in which text classification and [Formula: see text] classification are treated as optimization targets. Meanwhile, triplet loss is employed to enhance the analysis of differences and connections among labels. Moreover, considering that one-hot usage is still short of exploiting label information, we incorporate external knowledge from WordNet to obtain multiaspect descriptions for label semantic learning and extend [Formula: see text]-Net to a novel description-enhanced label embedding network(DELE) from a label embedding perspective. One step further, since these fine-grained descriptions may introduce unexpected noise, we develop a mutual interaction module to select appropriate parts from input sentences and labels simultaneously based on contrastive learning (CL) for noise mitigation. Extensive experiments on different text classification tasks reveal that [Formula: see text]-Net can effectively improve the classification performance and DELE can make better use of label information and further improve the performance. As a byproduct, we have released the codes to facilitate other research.

Label-guided contrastive learning with weighted pseudo-labeling: A novel mechanical fault diagnosis method with insufficient annotated data

Exploring fault diagnosis methods for mechanical equipment with weak dependency on annotated data is essential for industrial production. Contrastive learning (CL), capable of learning representations without labeling information, has achieved satisfactory performance in mechanical fault diagnosis. However, current CL-based approaches mainly encounter two limitations. First, the pre-training stage uses either unannotated or annotated samples exclusively while the fine-tuning stage solely relies on annotated ones, leading to inefficient sample utilization. Second, the representation learned by contrastive loss alone in the pretext task is sub-optimal for downstream diagnostic tasks. To address these issues, this paper proposed a novel diagnostic framework based on label-guided contrastive learning (LgCL) and weighted pseudo-labeling (WPL) strategy to improve fault diagnosis accuracy. In the pre-training stage, the proposed LgCL integrates two types of contrastive loss together with classification loss, enabling the encoder to learn discriminative representations that directly benefit the downstream diagnostic task. The devised hybrid fine-tuning strategy allows both labeled and unlabeled data to participate in fine-tuning via pseudo-labeling, enhancing model generalization. The pertinently designed WPL strategy mitigates the defect of noisy pseudo labels. Comparison and ablation experiments on two public datasets and one self-designed dataset validate the superiority of the proposed method for fault diagnosis with limited annotated data, with diagnostic accuracies improved by 25.30%, 5.47% and 10.02% over supervised, semi-supervised and contrastive learning methods, respectively.

Deep contrastive learning for predicting cancer prognosis using gene expression values.

Recent advancements in image classification have demonstrated that contrastive learning (CL) can aid in further learning tasks by acquiring good feature representation from a limited number of data samples. In this paper, we applied CL to tumor transcriptomes and clinical data to learn feature representations in a low-dimensional space. We then utilized these learned features to train a classifier to categorize tumors into a high- or low-risk group of recurrence. Using data from The Cancer Genome Atlas (TCGA), we demonstrated that CL can significantly improve classification accuracy. Specifically, our CL-based classifiers achieved an area under the receiver operating characteristic curve (AUC) greater than 0.8 for 14 types of cancer, and an AUC greater than 0.9 for 3 types of cancer. We also developed CL-based Cox (CLCox) models for predicting cancer prognosis. Our CLCox models trained with the TCGA data outperformed existing methods significantly in predicting the prognosis of 19 types of cancer under consideration. The performance of CLCox models and CL-based classifiers trained with TCGA lung and prostate cancer data were validated using the data from two independent cohorts. We also show that the CLCox model trained with the whole transcriptome significantly outperforms the Cox model trained with the 16 genes of Oncotype DX that is in clinical use for breast cancer patients. The trained models and the Python codes are publicly accessible and provide a valuable resource that will potentially find clinical applications for many types of cancer.

DGCL: dual-graph neural networks contrastive learning for molecular property prediction.

In this paper, we propose DGCL, a dual-graph neural networks (GNNs)-based contrastive learning (CL) integrated with mixed molecular fingerprints (MFPs) for molecular property prediction. The DGCL-MFP method contains two stages. In the first pretraining stage, we utilize two different GNNs as encoders to construct CL, rather than using the method of generating enhanced graphs as before. Precisely, DGCL aggregates and enhances features of the same molecule by the Graph Isomorphism Network and the Graph Attention Network, with representations extracted from the same molecule serving as positive samples, and others marked as negative ones. In the downstream tasks training stage, features extracted from the two above pretrained graph networks and the meticulously selected MFPs are concated together to predict molecular properties. Our experiments show that DGCL enhances the performance of existing GNNs by achieving or surpassing the state-of-the-art self-supervised learning models on multiple benchmark datasets. Specifically, DGCL increases the average performance of classification tasks by 3.73$\%$ and improves the performance of regression task Lipo by 0.126. Through ablation studies, we validate the impact of network fusion strategies and MFPs on model performance. In addition, DGCL's predictive performance is further enhanced by weighting different molecular features based on the Extended Connectivity Fingerprint. The code and datasets of DGCL will be made publicly available.

Leukocyte classification using relative-relationship-guided contrastive learning

Hematologic diseases and blood disorders can be studied through microscopic examination of blood smear images or chemical assays. Many researchers are focused on utilizing deep learning (DL) to identify, quantify, and classify various types of blood cells, which is crucial for addressing theoretical and practical challenges in disease diagnosis and treatment planning. However, there is a significant deficiency in annotations for leukocyte classification, limiting the effectiveness of current DL methods. Contrastive learning (CL) facilitates the extraction of prior knowledge from unlabeled data, reducing the dependency on manual annotations. While CL demonstrates remarkable achievements in classifying natural images, the direct application to leukocyte classification presents certain defects. We observe that standard data augmentation techniques in CL exhibit varying sensitivity to different categories of white blood cell images. Employing a uniform augmentation strategy may lead the network into an optimization trap, acquiring inadequate representations from erroneous sample relationships. To address this issue, we propose a Relative-Relationship-Guided Contrastive Learning Representation (ReCLR) framework for the leukocyte classification. ReCLR mines positive and negative samples based on relative distance knowledge. Specifically, we provide adversarial guidance for the positive sample, restricting the distance between the positive sample and the original sample less than but as close as possible to the distance between the original sample and the farthest negative sample. Then, we utilize the entropy constraint to regulate the relative distance relationship between the negative and original samples. Finally, the guided positive and negative samples are employed for contrastive learning in leukocyte classification. The extensive experiments, including linear evaluation, domain transfer, and fine-tuning, demonstrate the effectiveness of the proposed method. Our ReCLR achieves accuracies of 92.07%, 65.36%, and 92.49% on three real-world leukocyte datasets, respectively, outperforming several state-of-the-art methods. The source code is released at https://github.com/AlchemyEmperor/ReCLR.

Graphical contrastive learning for multi-interest sequential recommendation

Multi-interest sequential recommendation aims to deliver accurate recommendations by modeling users’ sequential behaviors and multi-faceted interests. Contrastive learning (CL) possesses an inborn advantage in distinguishing users’ multiple interests. However, existing CL-based multi-interest sequential recommenders tend to overlook the issue of data sparsity which deteriorates model performance. Graph neural network (GNN), on the other hand, has been proven as an effective measure of tackling the data sparsity problem. However, incorporating GNN into CL requires a delicate design to simultaneously achieve distinctive interest representation learning and relieve data sparsity. To address the challenges, we propose a Graphical Contrastive Learning for Multi-Interest Sequential Recommendation (GCL4MI), which first extracts user interest and constructs interest-specific graph based on item clusters and then perform graphical contrastive learning to enhance the independence between multiple interests. Furthermore, we explore different extensible CL sampling strategies with the facilitation of GNN to learn distinct interests in a high-cohesion and low-coupling manner. Extensive experiments on three real-world datasets demonstrate that our proposed GCL4MI significantly outperforms state-of-the-arts, with average lifts of 7.0%, 7.1%, and 5.8% on recall, NDCG, and HR, respectively.

Affinity Uncertainty-Based Hard Negative Mining in Graph Contrastive Learning.

Hard negative mining has shown effective in enhancing self-supervised contrastive learning (CL) on diverse data types, including graph CL (GCL). The existing hardness-aware CL methods typically treat negative instances that are most similar to the anchor instance as hard negatives, which helps improve the CL performance, especially on image data. However, this approach often fails to identify the hard negatives but leads to many false negatives on graph data. This is mainly due to that the learned graph representations are not sufficiently discriminative due to oversmooth representations and/or non-independent and identically distributed (non-i.i.d.) issues in graph data. To tackle this problem, this article proposes a novel approach that builds a discriminative model on collective affinity information (i.e., two sets of pairwise affinities between the negative instances and the anchor instance) to mine hard negatives in GCL. In particular, the proposed approach evaluates how confident/uncertain the discriminative model is about the affinity of each negative instance to an anchor instance to determine its hardness weight relative to the anchor instance. This uncertainty information is then incorporated into the existing GCL loss functions via a weighting term to enhance their performance. The enhanced GCL is theoretically grounded that the resulting GCL loss is equivalent to a triplet loss with an adaptive margin being exponentially proportional to the learned uncertainty of each negative instance. Extensive experiments on ten graph datasets show that our approach does the following: 1) consistently enhances different state-of-the-art (SOTA) GCL methods in both graph and node classification tasks and 2) significantly improves their robustness against adversarial attacks. Code is available at https://github.com/mala-lab/AUGCL.

Contrastive Learning with Transformer Initialization and Clustering Prior for Text Representation

Acquiring labeled data for learning sentence embeddings in Natural Language Processing poses challenges due to limited availability and high costs. In order to tackle this issue, we introduce a novel method called Contrastive Learning with Transformer Initialization and Clustering Prior for Text Representation (CLTC). Our method utilizes Pre-Layernorm Transformers without warm-up, stabilizing the training process while also increasing the final performance. We employ Contrastive Learning (CL) with dropout-based augmentation to enhance sentence embeddings. Additionally, we integrate prior knowledge into the contrastive learning framework within an efficient clustering strategy. When evaluated on the SentEval task, our approach showcases a competitive performance when compared to state-of-the-art approaches in the contrastive learning domain. Our method offers stability, improved embeddings, and the utilization of prior knowledge for enhanced unsupervised representation learning in Natural Language Processing applications.

Advanced Global Prototypical Segmentation Framework for Few-Shot Hyperspectral Image Classification.

With the advancement of deep learning, related networks have shown strong performance for Hyperspectral Image (HSI) classification. However, these methods face two main challenges in HSI classification: (1) the inability to capture global information of HSI due to the restriction of patch input and (2) insufficient utilization of information from limited labeled samples. To overcome these challenges, we propose an Advanced Global Prototypical Segmentation (AGPS) framework. Within the AGPS framework, we design a patch-free feature extractor segmentation network (SegNet) based on a fully convolutional network (FCN), which processes the entire HSI to capture global information. To enrich the global information extracted by SegNet, we propose a Fusion of Lateral Connection (FLC) structure that fuses the low-level detailed features of the encoder output with the high-level features of the decoder output. Additionally, we propose an Atrous Spatial Pyramid Pooling-Position Attention (ASPP-PA) module to capture multi-scale spatial positional information. Finally, to explore more valuable information from limited labeled samples, we propose an advanced global prototypical representation learning strategy. Building upon the dual constraints of the global prototypical representation learning strategy, we introduce supervised contrastive learning (CL), which optimizes our network with three different constraints. The experimental results of three public datasets demonstrate that our method outperforms the existing state-of-the-art methods.

Contrastive Learning for Lane Detection via cross-similarity

Detecting lane markings in road scenes poses a significant challenge due to their intricate nature, which is susceptible to unfavorable conditions. While lane markings have strong shape priors, their visibility is easily compromised by varying lighting conditions, adverse weather, occlusions by other vehicles or pedestrians, road plane changes, and fading of colors over time. The detection process is further complicated by the presence of several lane shapes and natural variations, necessitating large amounts of high-quality and diverse data to train a robust lane detection model capable of handling various real-world scenarios.In this paper, we present a novel self-supervised learning method termed Contrastive Learning for Lane Detection via Cross-Similarity (CLLD) to enhance the resilience and effectiveness of lane detection models in real-world scenarios, particularly when the visibility of lane markings are compromised. CLLD introduces a novel contrastive learning (CL) method that assesses the similarity of local features within the global context of the input image. It uses the surrounding information to predict lane markings. This is achieved by integrating local feature contrastive learning with our newly proposed operation, dubbed cross-similarity.The local feature CL concentrates on extracting features from small patches, a necessity for accurately localizing lane segments. Meanwhile, cross-similarity captures global features, enabling the detection of obscured lane segments based on their surroundings. We enhance cross-similarity by randomly masking portions of input images in the process of augmentation. Extensive experiments on TuSimple and CuLane benchmark datasets demonstrate that CLLD consistently outperforms state-of-the-art contrastive learning methods, particularly in visibility-impairing conditions like shadows, while it also delivers comparable results under normal conditions. When compared to supervised learning, CLLD still excels in challenging scenarios such as shadows and crowded scenes, which are common in real-world driving.

PointUR-RL: Unified Self-Supervised Learning Method Based on Variable Masked Autoencoder for Point Cloud Reconstruction and Representation Learning

Self-supervised learning has made significant progress in point cloud processing. Currently, the primary tasks of self-supervised learning, which include point cloud reconstruction and representation learning, are trained separately due to their structural differences. This separation inevitably leads to increased training costs and neglects the potential for mutual assistance between tasks. In this paper, a self-supervised method named PointUR-RL is introduced, which integrates point cloud reconstruction and representation learning. The method features two key components: a variable masked autoencoder (VMAE) and contrastive learning (CL). The VMAE is capable of processing input point cloud blocks with varying masking ratios, ensuring seamless adaptation to both tasks. Furthermore, CL is utilized to enhance the representation learning capabilities and improve the separability of the learned representations. Experimental results confirm the effectiveness of the method in training and its strong generalization ability for downstream tasks. Notably, high-accuracy classification and high-quality reconstruction have been achieved with the public datasets ModelNet and ShapeNet, with competitive results also obtained with the ScanObjectNN real-world dataset.

Predicting Promoters in Multiple Prokaryotes with Prompt.

Promoters are important cis-regulatory elements for the regulation of gene expression, and their accurate predictions are crucial for elucidating the biological functions and potential mechanisms of genes. Many previous prokaryotic promoter prediction methods are encouraging in terms of the prediction performance, but most of them focus on the recognition of promoters in only one or a few bacterial species. Moreover, due to ignoring the promoter sequence motifs, the interpretability of predictions with existing methods is limited. In this work, we present a generalized method Prompt (Promoters in multiple prokaryotes) to predict promoters in 16 prokaryotes and improve the interpretability of prediction results. Prompt integrates three methods including RSK (Regression based on Selected k-mer), CL (Contrastive Learning) and MLP (Multilayer Perception), and employs a voting strategy to divide the datasets into high-confidence and low-confidence categories. Results on the promoter prediction tasks in 16 prokaryotes show that the accuracy (Accuracy, Matthews correlation coefficient) of Prompt is greater than 80% in highly credible datasets of 16 prokaryotes, and is greater than 90% in 12 prokaryotes, and Prompt performs the best compared with other existing methods. Moreover, by identifying promoter sequence motifs, Prompt can improve the interpretability of the predictions. Prompt is freely available at https://github.com/duqimeng/PromptPrompt , and will contribute to the research of promoters in prokaryote.

Exploring galaxy properties of eCALIFA with contrastive learning

Contrastive learning (CL) has emerged as a potent tool for building meaningful latent representations of galaxy properties across a broad spectrum of wavelengths, ranging from optical and infrared to radio frequencies. These latent representations facilitate a variety of downstream tasks, including galaxy classification, similarity searches in extensive datasets, and parameter estimation, which is why they are often referred to as foundation models for galaxies. In this study, we employ CL on the latest extended data release from the Calar Alto Legacy Integral Field Area (CALIFA) survey, which encompasses a total of 895 galaxies with enhanced spatial resolution that reaches the limits imposed by natural seeing (FWHMPSF ∼ 1.5). We demonstrate that CL can be effectively applied to Integral Field Unit (IFU) surveys, even with relatively small training sets, to construct meaningful embedding where galaxies are well separated based on their physical properties. We discover that the strongest correlations in the embedding space are observed with the equivalent width of Hα, galaxy morphology, stellar metallicity, luminosity-weighted age, stellar surface mass density, the [NII]/Hα ratio, and stellar mass, in descending order of correlation strength. Additionally, we illustrate the feasibility of unsupervised separation of galaxy populations along the star formation main sequence, successfully identifying the blue cloud and the red sequence in a two-cluster scenario, and the green valley population in a three-cluster scenario. Our findings indicate that galaxy luminosity profiles have minimal impact on the construction of the embedding space, suggesting that morphology and spectral features play a more significant role in distinguishing between galaxy populations. Moreover, we explore the use of CL for detecting variations in galaxy population distributions across different large-scale structures, including voids, clusters, and filaments and walls. Nonetheless, we acknowledge the limitations of the CL framework and our specific training set in detecting subtle differences in galaxy properties, such as the presence of an AGN or other minor scale variations that exceed the scope of primary parameters such as the stellar mass or morphology. Conclusively, we propose that CL can serve as an embedding function for the development of larger models capable of integrating data from multiple datasets, thereby advancing the construction of more comprehensive foundation models for galaxies.

IGU-Aug: Information-guided unsupervised augmentation and pixel-wise contrastive learning for medical image analysis.

Contrastive learning (CL) is a form of self-supervised learning and has been widely used for various tasks. Different from widely studied instance-level contrastive learning, pixel-wise contrastive learning mainly helps with pixel-wise dense prediction tasks. The counter-part to an instance in instance-level CL is a pixel, along with its neighboring context, in pixel-wise CL. Aiming to build better feature representation, there is a vast literature about designing instance augmentation strategies for instance-level CL; but there is little similar work on pixel augmentation for pixel-wise CL with a pixel granularity. In this paper, we attempt to bridge this gap. We first classify a pixel into three categories, namely low-, medium-, and high-informative, based on the information quantity the pixel contains. We then adaptively design separate augmentation strategies for each category in terms of augmentation intensity and sampling ratio. Extensive experiments validate that our information-guided pixel augmentation strategy succeeds in encoding more discriminative representations and surpassing other competitive approaches in unsupervised local feature matching. Furthermore, our pretrained model improves the performance of both one-shot and fully supervised models. To the best of our knowledge, we are the first to propose a pixel augmentation method with a pixel granularity for enhancing unsupervised pixel-wise contrastive learning. Code is available at https: //github.com/Curli-quan/IGU-Aug.

Towards Better Modeling With Missing Data: A Contrastive Learning-Based Visual Analytics Perspective.

Missing data can pose a challenge for machine learning (ML) modeling. To address this, current approaches are categorized into feature imputation and label prediction and are primarily focused on handling missing data to enhance ML performance. These approaches rely on the observed data to estimate the missing values and therefore encounter three main shortcomings in imputation, including the need for different imputation methods for various missing data mechanisms, heavy dependence on the assumption of data distribution, and potential introduction of bias. This study proposes a Contrastive Learning (CL) framework to model observed data with missing values, where the ML model learns the similarity between an incomplete sample and its complete counterpart and the dissimilarity between other samples. Our proposed approach demonstrates the advantages of CL without requiring any imputation. To enhance interpretability, we introduce CIVis, a visual analytics system that incorporates interpretable techniques to visualize the learning process and diagnose the model status. Users can leverage their domain knowledge through interactive sampling to identify negative and positive pairs in CL. The output of CIVis is an optimized model that takes specified features and predicts downstream tasks. We provide two usage scenarios in regression and classification tasks and conduct quantitative experiments, expert interviews, and a qualitative user study to demonstrate the effectiveness of our approach. In short, this study offers a valuable contribution to addressing the challenges associated with ML modeling in the presence of missing data by providing a practical solution that achieves high predictive accuracy and model interpretability.

Improving Graph Collaborative Filtering with Directional Behavior Enhanced Contrastive Learning

Graph Collaborative Filtering is a widely adopted approach for recommendation, which captures similar behavior features through Graph Neural Network (GNN). Recently, Contrastive Learning (CL) has been demonstrated as an effective method to enhance the performance of graph collaborative filtering. Typically, CL-based methods first perturb users’ history behavior data (e.g., drop clicked items), then construct a self-discriminating task for behavior representations under different random perturbations. However, for widely existing inactive users, random perturbation makes their sparse behavior information more incomplete, thereby harming the behavior feature extraction. To tackle the above issue, we design a novel directional perturbation-based CL method to improve the graph collaborative filtering performance. The idea is to perturb node representations through directionally enhancing behavior features. To do so, we propose a simple yet effective feedback mechanism, which fuses the representations of nodes based on behavior similarity. Then, to avoid irrelevant behavior preferences introduced by the feedback mechanism, we construct a behavior self-contrast task before and after feedback, to align the node representations between the final output and the first layer of GNN. Different from the widely adopted self-discriminating task, the behavior self-contrast task avoids complex message propagation on different perturbed graphs, which is more efficient than previous methods. Extensive experiments on three public datasets demonstrate that the proposed method has distinct advantages over other CL methods on recommendation accuracy.

Detection of the irrotational boundary using machine learning methods

Four machine learning methods, i.e., self-organizing map (SOM), Gaussian mixture model (GMM), eXtreme gradient boosting (XGBoost), and contrastive learning (CL), are used to detect the irrotational boundary (IB), which represents the outer edge of the turbulent and non-turbulent interface layer. To accurately evaluate the detection methods, high-resolution databases from direct numerical simulations of a temporally evolving turbulent plane jet are used. It is found that except for the SOM method, the general contour of the IB appears to be effectively captured using the GMM, XGBoost, and CL methods, which indicate the turbulent and non-turbulent regions can be roughly recognized. Furthermore, the intrinsic features of the detected IB using the GMM, XGBoost, and the CL methods are quantitatively evaluated. Unlike the conventional vorticity norm method, the three machine learning methods do not rely on a single threshold of vorticity magnitude to separate the turbulent and non-turbulent regions. A small part of the detected IB using the three machine learning methods is characterized by the rotational motions, which are expected to be only found inside the turbulent sublayer and turbulent core region. Compared to the vorticity norm and XGBoost methods, the fractal dimensions of the IB detected by the GMM and CL methods are relatively small, which are related to the missing detection of some highly contorted elements. With the three machine learning methods, a large part of the detected IB is characterized by a convex shape, similarly as with the vorticity norm. However, the probability density function profiles of the local curvature of the detected IB differ greatly between the three machine learning methods and the vorticity norm. A mild variation of the mean conditional distributions of the vorticity magnitude can be observed across the detected IB by the three machine learning methods. This study first implies that using the machine learning methods the turbulent and non-turbulent regions can be roughly distinguished, but it is still challenging to obtain the intrinsic features of the detected IB.

Inter-structure and intra-semantics graph contrastive learning for disease prediction

Ever-evolving healthcare applications have witnessed a surge in the utilization of electronic health records (EHR) for predicting future patient diagnoses. While Graph Neural Networks have demonstrated that promise in modeling disease-patient relationships, challenges arise from the sparsity and imbalance of patient and diagnostic data. Moreover, the existing models face difficulties in learning the unique disease combination features of patients. To address these challenges, we proposed a novel disease. prediction architecture based on Contrastive Learning (CL) from interstructural and intrasemantic perspectives, rather than traditional CL methods. We generated an initial global static disease graph to directly represent the relationships. among all diseases and a local dynamic disease graph to capture the indirect latent disease relationships among different patients. Multiple CL tasks were designed to learn sparse and imbalanced potentials. Relationships Between Diseases. Interstructure graph CL was first proposed to sample a graph enhancement, based on the distribution of nodes in the global disease graph. To further explore the deep embedding space of the disease, an intra-view graph CL was introduced by injecting noise at the semantic level for robust graph comparison. Experimental validation on two real EHR datasets demonstrates the superior performance of the approach by comparing it with state-of-the-art models.

LMACL: Improving Graph Collaborative Filtering with Learnable Model Augmentation Contrastive Learning

Graph collaborative filtering (GCF) has achieved exciting recommendation performance with its ability to aggregate high-order graph structure information. Recently, contrastive learning (CL) has been incorporated into GCF to alleviate data sparsity and noise issues. However, most of the existing methods employ random or manual augmentation to produce contrastive views that may destroy the original topology and amplify the noisy effects. We argue that such augmentation is insufficient to produce the optimal contrastive view, leading to suboptimal recommendation results. In this article, we proposed a L earnable M odel A ugmentation C ontrastive L earning (LMACL) framework for recommendation, which effectively combines graph-level and node-level collaborative relations to enhance the expressiveness of collaborative filtering (CF) paradigm. Specifically, we first use the graph convolution network (GCN) as a backbone encoder to incorporate multi-hop neighbors into graph-level original node representations by leveraging the high-order connectivity in user-item interaction graphs. At the same time, we treat the multi-head graph attention network (GAT) as an augmentation view generator to adaptively generate high-quality node-level augmented views. Finally, joint learning endows the end-to-end training fashion. In this case, the mutual supervision and collaborative cooperation of GCN and GAT achieves learnable model augmentation. Extensive experiments on several benchmark datasets demonstrate that LMACL provides a significant improvement over the strongest baseline in terms of Recall and NDCG by 2.5%–3.8% and 1.6%–4.0%, respectively. Our model implementation code is available at https://github.com/LiuHsinx/LMACL .