Representation Learning Module Research Articles

A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions

Clustering is a fundamental machine learning task which aims at assigning instances into groups so that similar samples belong to the same cluster while dissimilar samples belong to different clusters. Shallow clustering methods usually assume that data are collected and expressed as feature vectors within which clustering is performed. However, clustering high-dimensional data, such as images, texts, videos, and graphs, poses significant challenges for clustering tasks, such as indiscriminate representation and intricate relationships among instances. Over the past decades, deep learning has achieved remarkable success in effective representation learning and modeling complex relationships. Motivated by these advancements, Deep Clustering seeks to improve clustering outcomes through deep learning techniques, garnering considerable interest from both academia and industry. Despite many contributions to this vibrant area of research, the lack of systematic analysis and a comprehensive taxonomy has hindered progress in this field. In this survey, we first explore how deep learning can be integrated into deep clustering and identify two fundamental components: the representation learning module and the clustering module. Then we summarize and analyze the representative design of these two modules. Furthermore, we introduce a novel taxonomy of deep clustering based on how these two modules interact, specifically through multistage, generative, iterative, and simultaneous approaches. In addition, we present well-known benchmark datasets, evaluation metrics, and open-source tools to clearly demonstrate different experimental approaches. Finally, we examine the practical applications of deep clustering and propose challenging areas for future research.

A Novel Approach to Pedestrian Re-Identification in Low-Light and Zero-Shot Scenarios: Exploring Transposed Convolutional Reflectance Decoders

In recent years, pedestrian re-identification technology has made significant progress, with various neural network models performing well under normal conditions, such as good weather and adequate lighting. However, most research has overlooked extreme environments, such as rainy weather and nighttime. Additionally, the existing pedestrian re-identification datasets predominantly consist of well-lit images. Although some studies have started to address these issues by proposing methods for enhancing low-light images to restore their original features, the effectiveness of these approaches remains limited. We noted that a method based on Retinex theory designed a reflectance representation learning module aimed at restoring image features as much as possible. However, this method has so far only been applied in object detection networks. In response to this, we improved the method and applied it to pedestrian re-identification, proposing a transposed convolution reflectance decoder (TransConvRefDecoder) to better restore details in low-light images. Extensive experiments on the Market1501, CUHK03, and MSMT17 datasets demonstrated that our approach delivered superior performance.

CSMDC: Exploring consistently context semantics for multi-view document clustering

Multi-view document clustering, which aims to discover clustering partitions based on multiple document views, has attracted increasing research interest. However, the potential advantages of incorporating context semantics to enhance multi-view document clustering are yet to be fully explored. To address the above limitation, we propose a deep multi-view document clustering model that explores consistent context semantics called CSMDC which consists of three modules. Specifically, a novel view-translator is designed to convert non-contextual document views into contextual views. With its help, all document views can be processed to obtain their semantic representations within the view-translator representation learning module. Then the data-based view consistency self-supervising module is developed to fine-tune the semantic representations of document views by jointly incorporating view-wise representation relevance and consistent clustering assignments. Additionally, the task-based document clustering module is employed to simultaneously improve the view semantic representations and document clustering results. To the best of our knowledge, this is the first study to explicitly apply consistent context semantics under the guidance of data-based and task-based objectives in multi-view document clustering. Comprehensive experimental results demonstrated the effectiveness of the proposed model.

Effective type label-based synergistic representation learning for biomedical event trigger detection

BackgroundDetecting event triggers in biomedical texts, which contain domain knowledge and context-dependent terms, is more challenging than in general-domain texts. Most state-of-the-art models rely mainly on external resources such as linguistic tools and knowledge bases to improve system performance. However, they lack effective mechanisms to obtain semantic clues from label specification and sentence context. Given its success in image classification, label representation learning is a promising approach to enhancing biomedical event trigger detection models by leveraging the rich semantics of pre-defined event type labels.ResultsIn this paper, we propose the Biomedical Label-based Synergistic representation Learning (BioLSL) model, which effectively utilizes event type labels by learning their correlation with trigger words and enriches the representation contextually. The BioLSL model consists of three modules. Firstly, the Domain-specific Joint Encoding module employs a transformer-based, domain-specific pre-trained architecture to jointly encode input sentences and pre-defined event type labels. Secondly, the Label-based Synergistic Representation Learning module learns the semantic relationships between input texts and event type labels, and generates a Label-Trigger Aware Representation (LTAR) and a Label-Context Aware Representation (LCAR) for enhanced semantic representations. Finally, the Trigger Classification module makes structured predictions, where each label is predicted with respect to its neighbours. We conduct experiments on three benchmark BioNLP datasets, namely MLEE, GE09, and GE11, to evaluate our proposed BioLSL model. Results show that BioLSL has achieved state-of-the-art performance, outperforming the baseline models.ConclusionsThe proposed BioLSL model demonstrates good performance for biomedical event trigger detection without using any external resources. This suggests that label representation learning and context-aware enhancement are promising directions for improving the task. The key enhancement is that BioLSL effectively learns to construct semantic linkages between the event mentions and type labels, which provide the latent information of label-trigger and label-context relationships in biomedical texts. Moreover, additional experiments on BioLSL show that it performs exceptionally well with limited training data under the data-scarce scenarios.

DyHDGE: Dynamic heterogeneous transaction graph embedding for safety-centric fraud detection in financial scenarios

Dynamic graph fraud detection aims to distinguish fraudulent entities that deviate significantly from most benign entities within an ever-changing graph network. However, when dealing with different financial fraud scenarios, existing methods face challenges, resulting in difficulty in effectively ensuring financial security. In fraud scenarios, transaction data are generated in real time, in which a strong temporal relationship between multiple fraudulent transactions is observed. Traditional dynamic graph models struggle to effectively balance the temporal features of nodes and spatial structural features, failing to handle different types of nodes in the graph network. In this study, to extract the temporal and structural information, we proposed a dynamic heterogeneous transaction graph embedding (DyHDGE) network based on a dynamic heterogeneous transaction graph, considering both temporal and structural information while incorporating heterogeneous data. To separately extract temporal relationships between transactions and spatial structural relationships between nodes, we used a heterogeneous temporal graph representation learning module and a temporal graph structure information extraction module. Additionally, we designed two loss functions to optimize node feature representations. Extensive experiments demonstrated that the proposed DyHDGE significantly outperformed previous state-of-the-art methods on two simulated datasets of financial fraud scenarios. This capability contributes to enhancing security in financial consumption scenarios.

E-BDL: Enhanced Band-Dependent Learning Framework for Augmented Radar Sensing.

Radar sensors, leveraging the Doppler effect, enable the nonintrusive capture of kinetic and physiological motions while preserving privacy. Deep learning (DL) facilitates radar sensing for healthcare applications such as gait recognition and vital-sign measurement. However, band-dependent patterns, indicating variations in patterns and power scales associated with frequencies in time-frequency representation (TFR), challenge radar sensing applications using DL. Frequency-dependent characteristics and features with lower power scales may be overlooked during representation learning. This paper proposes an Enhanced Band-Dependent Learning framework (E-BDL) comprising an adaptive sub-band filtering module, a representation learning module, and a sub-view contrastive module to fully detect band-dependent features in sub-frequency bands and leverage them for classification. Experimental validation is conducted on two radar datasets, including gait abnormality recognition for Alzheimer's disease (AD) and AD-related dementia (ADRD) risk evaluation and vital-sign monitoring for hemodynamics scenario classification. For hemodynamics scenario classification, E-BDL-ResNet achieves competitive performance in overall accuracy and class-wise evaluations compared to recent methods. For ADRD risk evaluation, the results demonstrate E-BDL-ResNet's superior performance across all candidate models, highlighting its potential as a clinical tool. E-BDL effectively detects salient sub-bands in TFRs, enhancing representation learning and improving the performance and interpretability of DL-based models.

Dual-Pathway Deep Hashing-Based Adversarial Learning for Cross-Modal Retrieval

The application of deep hashing methods for cross-modal retrieval has seen growing interest due to their storage efficiency and fast query execution. However, the challenge posed by the “heterogeneity gap” in multi-modal datasets cannot be understated. To address this, we present a novel framework named Dual-Pathway Deep Hashing-Based Adversarial Learning (DP-DHAL), engineered to surmount this challenge. The architecture of DP-DHAL integrates three key components: (a) a dual-pathway representation learning module tasked with extracting modality-specific features; (b) an adversarial module working to align the distributions of cross-modal features; and (c) a deep hashing module responsible for generating hash codes that uphold the similarity relationships across different modalities. Additionally, we have developed a unique Hamming triplet-margin loss function to refine the assessment of content similarities. The DP-DHAL model is trained through an adversarial process where the adversarial module’s goal is to discern cross-modal features with the aim of reducing the heterogeneity gap. Simultaneously, the representation learning module is focused on producing representations that can both deceive the adversarial module and preserve cross-modal similarities to yield distinctive hash codes. Comprehensive experiments on varied datasets have shown that our proposed method outperforms other leading cross-modal hashing techniques.

Bootstrap Latent Prototypes for graph positive-unlabeled learning

Graph positive-unlabeled (GPU) learning aims to learn binary classifiers from only positive and unlabeled (PU) nodes. The state-of-the-art methods rely on provided class prior probabilistic and their performance lags far behind the fully labeled counterparts. To bridge the gap, we propose Bootstrap Latent Prototypes (BLP), a framework that consists of a graph representation learning module and a two-step strategy algorithm. The learning module bootstraps previous versions of node representations to serve as targets and learns enhanced representations by predicting the latent prototypes respectively for the P set and each individual in the U set. It eliminates the requirement for a class prior, while capturing positive similarity information, as well as the low-level semantic similarity and uniformity information, thereby producing closely aligned and discriminated representations for positive nodes. The algorithm module utilizes the obtained representations to select reliable negative nodes and train a binary classifier with both labeled positives and selected reliable negatives. Experimental results on diverse real-life datasets demonstrate that our proposed BLP method not only outperforms state-of-the-art approaches but also surpasses fully labeled classification models in most cases. The source code is available at https://github.com/lcq411/BLP.

SSM-Net: Semi-supervised multi-task network for joint lesion segmentation and classification from pancreatic EUS images

Pancreatic cancer does not show specific symptoms, which makes the diagnosis of early stages difficult with established image-based screening methods and therefore has the worst prognosis among all cancers. Although endoscopic ultrasonography (EUS) has a key role in diagnostic algorithms for pancreatic diseases, B-mode imaging of the pancreas can be affected by confounders such as chronic pancreatitis, which can make both pancreatic lesion segmentation and classification laborious and highly specialized. To address these challenges, this work proposes a semi-supervised multi-task network (SSM-Net) to leverage unlabeled and labeled EUS images for joint pancreatic lesion classification and segmentation. Specifically, we first devise a saliency-aware representation learning module (SRLM) on a large number of unlabeled images to train a feature extraction encoder network for labeled images by computing a contrastive loss with a semantic saliency map, which is obtained by our spectral residual module (SRM). Moreover, for labeled EUS images, we devise channel attention blocks (CABs) to refine the features extracted from the pre-trained encoder on unlabeled images for segmenting lesions, and then devise a merged global attention module (MGAM) and a feature similarity loss (FSL) for obtaining a lesion classification result. We collect a large-scale EUS-based pancreas image dataset (LS-EUSPI) consisting of 9,555 pathologically proven labeled EUS images (499 patients from four categories) and 15,500 unlabeled EUS images. Experimental results on the LS-EUSPI dataset and a public thyroid gland lesion dataset show that our SSM-Net clearly outperforms state-of-the-art methods.

Query-Oriented Micro-Video Summarization.

Query-oriented micro-video summarization task aims to generate a concise sentence with two properties: (a) summarizing the main semantic of the micro-video and (b) being expressed in the form of search queries to facilitate retrieval. Despite its enormous application value in the retrieval area, this direction has barely been explored. Previous studies of summarization mostly focus on the content summarization for traditional long videos. Directly applying these studies is prone to gain unsatisfactory results because of the unique features of micro-videos and queries: diverse entities and complex scenes within a short time, semantic gaps between modalities, and various queries in distinct expressions. To specifically adapt to these characteristics, we propose a query-oriented micro-video summarization model, dubbed QMS. It employs an encoder-decoder-based transformer architecture as the skeleton. The multi-modal (visual and textual) signals are passed through two modal-specific encoders to obtain their representations, followed by an entity-aware representation learning module to identify and highlight critical entity information. As to the optimization, regarding the large semantic gaps between modalities, we assign different confidence scores according to their semantic relevance in the optimization process. Additionally, we develop a novel strategy to sample the effective target query among the diverse query set with various expressions. Extensive experiments demonstrate the superiority of the QMS scheme, on both the summarization and retrieval tasks, over several state-of-the-art methods.

RamanCluster: A deep clustering-based framework for unsupervised Raman spectral identification of pathogenic bacteria

Raman spectroscopy serves as a powerful and reliable tool for the characterization of pathogenic bacteria. The integration of Raman spectroscopy with artificial intelligence techniques to rapidly identify pathogenic bacteria has become paramount for expediting disease diagnosis. However, the development of prevailing supervised artificial intelligence algorithms is still constrained by costly and limited well-annotated Raman spectroscopy datasets. Furthermore, tackling various high-dimensional and intricate Raman spectra of pathogenic bacteria in the absence of annotations remains a formidable challenge. In this paper, we propose a concise and efficient deep clustering-based framework (RamanCluster) to achieve accurate and robust unsupervised Raman spectral identification of pathogenic bacteria without the need for any annotated data. RamanCluster is composed of a novel representation learning module and a machine learning-based clustering module, systematically enabling the extraction of robust discriminative representations and unsupervised Raman spectral identification of pathogenic bacteria. The extensive experimental results show that RamanCluster has achieved high accuracy on both Bacteria-4 and Bacteria-6, with ACC values of 77 % and 74.1 %, NMI values of 75 % and 73 %, as well as AMI values of 74.6 % and 72.6 %, respectively. Furthermore, compared with other state-of-the-art methods, RamanCluster exhibits the superior accuracy on handling various complicated pathogenic bacterial Raman spectroscopy datasets, including situations with strong noise and a wide variety of pathogenic bacterial species. Additionally, RamanCluster also demonstrates commendable robustness in these challenging scenarios. In short, RamanCluster has a promising prospect in accelerating the development of low-cost and widely applicable disease diagnosis in clinical medicine.

Multi-grained fusion network with self-distillation for aspect-based multimodal sentiment analysis

Aspect-based multimodal sentiment analysis (ABMSA) is an important branch of multimodal sentiment analysis. The goal of ABMSA is to use multimodal information to infer users’ sentiment polarity toward the targeted aspect for supporting corresponding decision-making. The existing ABMSA methods usually focus on exploring aspect-aware fine-grained interactions and demonstrate the benefits of integrating multimodal information. However, such approaches still suffer from the following limitations: (1) coarse-grained semantic information extraction is ignored, (2) the relevance between image and aspect is ignored, and (3) model-agnostic techniques to improve multimodal representations are not employed. To address these limitations, we propose a novel multimodal approach named multi-grained fusion network with self-distillation (MGFN-SD). This approach includes unimodal feature extraction, multi-grained representation learning, and self-distillation-based sentiment prediction. The multi-grained representation learning module extracts fine- and coarse-grained interactions based on the aspect–image relevance computation and the similarity calculation for dynamically filtering potential noise brought by the original image. In the sentiment prediction module, self-distillation is employed to transfer knowledge from both hard and soft labels to supervise the training process of each student classifier for improving the quality of multimodal representations. The experimental results on three benchmark datasets indicate the superiority, rationality, and robustness of MGFN-SD.

AACFlow: an end-to-end model based on attention augmented convolutional neural network and flow-attention mechanism for identification of anticancer peptides.

Anticancer peptides (ACPs) have natural cationic properties and can act on the anionic cell membrane of cancer cells to kill cancer cells. Therefore, ACPs have become a potential anticancer drug with good research value and prospect. In this article, we propose AACFlow, an end-to-end model for identification of ACPs based on deep learning. End-to-end models have more room to automatically adjust according to the data, making the overall fit better and reducing error propagation. The combination of attention augmented convolutional neural network (AAConv) and multi-layer convolutional neural network (CNN) forms a deep representation learning module, which is used to obtain global and local information on the sequence. Based on the concept of flow network, multi-head flow-attention mechanism is introduced to mine the deep features of the sequence to improve the efficiency of the model. On the independent test dataset, the ACC, Sn, Sp, and AUC values of AACFlow are 83.9%, 83.0%, 84.8%, and 0.892, respectively, which are 4.9%, 1.5%, 8.0%, and 0.016 higher than those of the baseline model. The MCC value is 67.85%. In addition, we visualize the features extracted by each module to enhance the interpretability of the model. Various experiments show that our model is more competitive in predicting ACPs.

GCNGAT: Drug–disease association prediction based on graph convolution neural network and graph attention network

Predicting drug–disease associations can contribute to discovering new therapeutic potentials of drugs, and providing important association information for new drug research and development. Many existing drug–disease association prediction methods have not distinguished relevant background information for the same drug targeted to different diseases. Therefore, this paper proposes a drug–disease association prediction model based on graph convolutional network and graph attention network (GCNGAT) to reposition marketed drugs under the distinguishment of background information. Firstly, in order to obtain initial drug–disease information, a drug–disease heterogeneous graph structure is constructed based on all known drug–disease associations. Secondly, based on the heterogeneous graph structure, the corresponding subgraphs of each group of drug–disease association pairs are extracted to distinguish different background information for the same drug from different diseases. Finally, a model combining Graph neural network with global Average pooling (GnnAp) is designed to predict potential drug–disease associations by learning drug–disease interaction feature representations. The experimental results show that adding subgraph extraction can effectively improve the prediction performance of the model, and the graph representation learning module can fully extract the deep features of drug–disease. Using the 5-fold cross-validation, the proposed model (GCNGAT) achieves AUC (Area Under the receiver operating characteristic Curve) values of 0.9182 and 0.9417 on the PREDICT dataset and CDataset dataset, respectively. Compared with other predictors on the same dataset (PREDICT dataset), GCNGAT outperforms the existing best-performing model (PSGCN), with a 1.58% increase in the AUC value. It is anticipated that this model can provide experimental reference for drug repositioning and further promote the drug research and development process.

Improving academic performance predictions with dual graph neural networks

AbstractAcademic performance is a crucial issue in the field of Online learning analytics. While deep learning-based models have made significant progress in the era of big data, many of these methods need help to capture the complex relationships present in online learning activities and student attributes, which are essential for improving prediction accuracy. We present a novel model for predicting academic performance in this paper. This model harnesses the power of dual graph neural networks to effectively utilize both the structural information derived from interaction activities and the attribute feature spaces of students. The proposed model uses an interaction-based graph neural network module to learn local academic performance representations from online interaction activities and an attribute-based graph neural network to learn global academic performance representations from attribute features of all students using dynamic graph convolution operations. The learned representations from local and global levels are combined in a local-to-global representation learning module to generate predicted academic performances. The empirical study results demonstrate that the proposed model significantly outperforms existing methods. Notably, the proposed model achieves an accuracy of 83.96% for predicting students who pass or fail and an accuracy of 90.18% for predicting students who pass or withdraw on a widely recognized public dataset. The ablation studies confirm the effectiveness and superiority of the proposed techniques.

Iterative Deep Structural Graph Contrast Clustering for Multiview Raw Data.

Multiview clustering has attracted increasing attention to automatically divide instances into various groups without manual annotations. Traditional shadow methods discover the internal structure of data, while deep multiview clustering (DMVC) utilizes neural networks with clustering-friendly data embeddings. Although both of them achieve impressive performance in practical applications, we find that the former heavily relies on the quality of raw features, while the latter ignores the structure information of data. To address the above issue, we propose a novel method termed iterative deep structural graph contrast clustering (IDSGCC) for multiview raw data consisting of topology learning (TL), representation learning (RL), and graph structure contrastive learning to achieve better performance. The TL module aims to obtain a structured global graph with constraint structural information and then guides the RL to preserve the structural information. In the RL module, graph convolutional network (GCN) takes the global structural graph and raw features as inputs to aggregate the samples of the same cluster and keep the samples of different clusters away. Unlike previous methods performing contrastive learning at the representation level of the samples, in the graph contrastive learning module, we conduct contrastive learning at the graph structure level by imposing a regularization term on the similarity matrix. The credible neighbors of the samples are constructed as positive pairs through the credible graph, and other samples are constructed as negative pairs. The three modules promote each other and finally obtain clustering-friendly embedding. Also, we set up an iterative update mechanism to update the topology to obtain a more credible topology. Impressive clustering results are obtained through the iterative mechanism. Comparative experiments on eight multiview datasets show that our model outperforms the state-of-the-art traditional and deep clustering competitors.

Deep Representation Learning for Open Vocabulary Electroencephalography-to-Text Decoding.

Previous research has demonstrated the potential of using pre-trained language models for decoding open vocabulary Electroencephalography (EEG) signals captured through a non-invasive Brain-Computer Interface (BCI). However, the impact of embedding EEG signals in the context of language models and the effect of subjectivity, remain unexplored, leading to uncertainty about the best approach to enhance decoding performance. Additionally, current evaluation metrics used to assess decoding effectiveness are predominantly syntactic and do not provide insights into the comprehensibility of the decoded output for human understanding. We present an end-to-end architecture for non-invasive brain recordings that brings modern representational learning approaches to neuroscience. Our proposal introduces the following innovations: 1) an end-to-end deep learning architecture for open vocabulary EEG decoding, incorporating a subject-dependent representation learning module for raw EEG encoding, a BART language model, and a GPT-4 sentence refinement module; 2) a more comprehensive sentence-level evaluation metric based on the BERTScore; 3) an ablation study that analyses the contributions of each module within our proposal, providing valuable insights for future research. We evaluate our approach on two publicly available datasets, ZuCo v1.0 and v2.0, comprising EEG recordings of 30 subjects engaged in natural reading tasks. Our model achieves a BLEU-1 score of 42.75%, a ROUGE-1-F of 33.28%, and a BERTScore-F of 53.86%, achieving an increment over the previous state-of-the-art by 1.40%, 2.59%, and 3.20%, respectively.

Dual-Decoupling With Frequency-Spatial Domains for Image Manipulation Localization.

Leveraging trace-rich features within embedded spaces has been established as effective in image manipulation localization (IML). Nevertheless, the feature of manipulated traces frequently comprises substantial redundant information only loosely related to IML tasks. This complexity has hindered existing methods in fully comprehending the essence of trace features. In light of this challenge, we introduce a novel decoupling representation learning network (DRN) tailored for IML. The DRN excels at decoupling intricate multidomain information and transforming it into representations directly pertinent to IML objectives. This is achieved through a meticulously designed frequency decoupling representation learning module (FDM) and spatial decoupling representation learning module (SDM). Specifically, the FDM operates by acquiring distinct low and high-frequency components to effectively decouple redundant information. The decoupled high-frequency components are then harnessed as intricate trace complements, enhancing the overall aggregation process. In addition, the redundant information is expertly separated into authentic and manipulated representations through the use of channel activation maps in SDM. Through extensive experimentation on three public benchmarks including CASIA, NIST, and Coverage, our method consistently demonstrates superior performance and enhanced robustness compared with existing state-of-the-art methods.

WL-Align: Weisfeiler-Lehman Relabeling for Aligning Users Across Networks via Regularized Representation Learning

Aligning users across networks using graph representation learning has been found effective where the alignment is accomplished in a low-dimensional embedding space. Yet, highly precise alignment remains challenging, especially for nodes with long-range connectivity to labeled anchors. To alleviate this limitation, we propose WL-Align which employs a regularized representation learning framework to learn distinctive node representations. It extends the Weisfeiler-Lehman Isormorphism Test and learns the alignment in alternating phases of “across-network Weisfeiler-Lehman relabeling” and “proximity-preserving representation learning”. The across-network Weisfeiler-Lehman relabeling is achieved through iterating the anchor-based label propagation and a similarity-based hashing to exploit the known anchors' connectivity to different nodes in an efficient and robust manner. The representation learning module preserves the second-order proximity within individual networks and is regularized by the across-network Weisfeiler-Lehman hash labels. Extensive experiments on real-world and synthetic datasets have demonstrated that our proposed WL-Align outperforms the state-of-the-art methods, achieving significant performance improvements in the “exact matching” scenario. Data and code of WL-Align are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ChenPengGang/WLAlignCode</uri> .

ScGCC: Graph Contrastive Clustering With Neighborhood Augmentations for scRNA-Seq Data Analysis.

Single-cell RNA sequencing (scRNA-seq) has rapidly emerged as a powerful technique for analyzing cellular heterogeneity at the individual cell level. In the analysis of scRNA-seq data, cell clustering is a critical step in downstream analysis, as it enables the identification of cell types and the discovery of novel cell subtypes. However, the characteristics of scRNA-seq data, such as high dimensionality and sparsity, dropout events and batch effects, present significant computational challenges for clustering analysis. In this study, we propose scGCC, a novel graph self-supervised contrastive learning model, to address the challenges faced in scRNA-seq data analysis. scGCC comprises two main components: a representation learning module and a clustering module. The scRNA-seq data is first fed into a representation learning module for training, which is then used for data classification through a clustering module. scGCC can learn low-dimensional denoised embeddings, which is advantageous for our clustering task. We introduce Graph Attention Networks (GAT) for cell representation learning, which enables better feature extraction and improved clustering accuracy. Additionally, we propose five data augmentation methods to improve clustering performance by increasing data diversity and reducing overfitting. These methods enhance the robustness of clustering results. Our experimental study on 14 real-world datasets has demonstrated that our model achieves extraordinary accuracy and robustness. We also perform downstream tasks, including batch effect removal, trajectory inference, and marker genes analysis, to verify the biological effectiveness of our model.