Semi-supervised Classification Research Articles

Mixed Graph Contrastive Network for Semi-supervised Node Classification

Graph Neural Networks (GNNs) have achieved promising performance in semi-supervised node classification in recent years. However, the problem of insufficient supervision, together with representation collapse, largely limits the performance of the GNNs in this field. To alleviate the collapse of node representations in semi-supervised scenario, we propose a novel graph contrastive learning method, termed M ixed G raph C ontrastive N etwork (MGCN). In our method, we improve the discriminative capability of the latent embeddings by an interpolation-based augmentation strategy and a correlation reduction mechanism. Specifically, we first conduct the interpolation-based augmentation in the latent space and then force the prediction model to change linearly between samples. Second, we enable the learned network to tell apart samples across two interpolation-perturbed views through forcing the correlation matrix across views to approximate an identity matrix. By combining the two settings, we extract rich supervision information from both the abundant unlabeled nodes and the rare yet valuable labeled nodes for discriminative representation learning. Extensive experimental results on six datasets demonstrate the effectiveness and the generality of MGCN compared to the existing state-of-the-art methods. The code of MGCN is available at https://github.com/xihongyang1999/MGCN on Github.

A Fully Test-time Training Framework for Semi-supervised Node Classification on Out-of-Distribution Graphs

Graph neural networks (GNNs) have shown great potential in representation learning for various graph tasks. However, the distribution shift between the training and test sets poses a challenge to the efficiency of GNNs. To address this challenge, HomoTTT proposes a fully test-time training framework for GNNs to enhance the model’s generalization capabilities for node classification tasks. Specifically, our proposed HomoTTT designs a homophily-based and parameter-free graph contrastive learning task with adaptive augmentation to guide the model’s adaptation during the test-time training, allowing the model to adapt for specific target data. In the inference stage, HomoTTT proposes to integrate the original GNN model and the adapted model after TTT using a homophily-based model selection method, which prevents potential performance degradation caused by unconstrained model adaptation. Extensive experimental results on six benchmark datasets demonstrate the effectiveness of our proposed framework. Additionally, the exploratory study further validates the rationality of the homophily-based graph contrastive learning task with adaptive augmentation and the homophily-based model selection designed in HomoTTT .

Semi-Supervised SAR Image Classification via Adaptive Threshold Selection

Semi-supervised learning is a good way to train a classification model using a small number of labeled and large number of unlabeled data. We applied semi-supervised learning to a synthetic aperture radar(SAR) image classification model with a limited number of datasets that are difficult to create. To address the previous difficulties, semi-supervised learning uses a model trained with a small amount of labeled data to generate and learn pseudo labels. Besides, a lot of number of papers use a single fixed threshold to create pseudo labels. In this paper, we present a semi-supervised synthetic aperture radar(SAR) image classification method that applies different thresholds for each class instead of all classes sharing a fixed threshold to improve SAR classification performance with a small number of labeled datasets.

Neighborhood Pattern Is Crucial for Graph Convolutional Networks Performing Node Classification.

Graph convolutional networks (GCNs) are widely believed to perform well in the graph node classification task, and homophily assumption plays a core rule in the design of previous GCNs. However, some recent advances on this area have pointed out that homophily may not be a necessity for GCNs. For deeper analysis of the critical factor affecting the performance of GCNs, we first propose a metric, namely, neighborhood class consistency (NCC), to quantitatively characterize the neighborhood patterns of graph datasets. Experiments surprisingly illustrate that our NCC is a better indicator, in comparison to the widely used homophily metrics, to estimate GCN performance for node classification. Furthermore, we propose a topology augmentation graph convolutional network (TA-GCN) framework under the guidance of the NCC metric, which simultaneously learns an augmented graph topology with higher NCC score and a node classifier based on the augmented graph topology. Extensive experiments on six public benchmarks clearly show that the proposed TA-GCN derives ideal topology with higher NCC score given the original graph topology and raw features, and it achieves excellent performance for semi-supervised node classification in comparison to several state-of-the-art (SOTA) baseline algorithms.

Deep temporal semi-supervised one-class classification for GNSS radio frequency interference detection

Deep temporal semi-supervised one-class classification for GNSS radio frequency interference detection

Dimension reduction-based adaptive-to-model semi-supervised classification

Dimension reduction-based adaptive-to-model semi-supervised classification

Deep graph layer information mining convolutional network

Graph convolution network is a powerful method of deep learning of graph structure data. Existing methods usually adjust the neighborhood information aggregation mode or optimize the graph topology layer by layer for improving the graph convolution network. However, these methods seldom consider the discriminative information about hierarchical characteristics nodes (some special nodes only can be correctly classified in one layer and are the misclassification nodes in the other layers of deep graph convolutional networks) in the different layers for complementing the neighborhood topology information. To further find these information, a deep graph layer information mining convolutional network (GLIM) can alternately measure the neighborhood ranking information on topology structure and update the residual identity mapping node information on the different layers for enhancing the model classification performance. Moreover, GLIM can construct a unified framework with the various hyper-parameters for the different graph learning method based on graph convolution network. Experiments show GLIM outperforms the state-of-the-art methods for semi-supervised node classification in three cite datasets (Cora, CiteSeer,and PubMed) and three image datasets (MNIST, Cifar10 and Cifar100).

Semi-Supervised Medical Image Classification with Pseudo Labels Using Coalition Similarity Training

The development of medical image classification models necessitates a substantial number of labeled images for model training. In real-world scenarios, sample sizes are typically limited and labeled samples often constitute only a small portion of the dataset. This paper aims to investigate a collaborative similarity learning strategy that optimizes pseudo-labels to enhance model accuracy and expedite its convergence, known as the joint similarity learning framework. By integrating semantic similarity and instance similarity, the pseudo-labels are mutually refined to ensure their quality during initial training. Furthermore, the similarity score is utilized as a weight to guide samples away from misclassification predictions during the classification process. To enhance the model’s generalization ability, an adaptive consistency constraint is introduced into the loss function to improve performance on untrained datasets. The model achieved a satisfactory accuracy of 93.65% at 80% labeling ratio, comparable to supervised learning methods’ performance. Even with very low labeling ratio (e.g., 5%), the model still attained an accuracy of 74.28%. Comparison with other techniques such as Mean Teacher and FixMatch revealed that our approach significantly outperforms them in medical image classification tasks through improving accuracy by approximately 2%, demonstrating this framework’s leadership in medical image classification.

A sea–land clutter classification framework for over-the-horizon radar based on weighted loss semi-supervised generative adversarial network

Deep convolutional neural network has made great achievements in sea–land clutter classification for over-the-horizon radar (OTHR). The premise is that a large number of labeled training samples must be provided for a sea–land clutter classifier. In practical engineering applications, it is relatively easy to obtain label-free sea–land clutter samples. However, the labeling process is extremely cumbersome and requires expertise in the field of OTHR. To solve this problem, we propose an improved generative adversarial network, namely weighted loss semi-supervised generative adversarial network (WL-SSGAN). Specifically, we propose a joint feature matching loss by weighting the middle layer features of the discriminator of semi-supervised generative adversarial network (SSGAN). Furthermore, we propose the weighted loss of WL-SSGAN by linearly weighting the standard adversarial loss of SSGAN and the joint feature matching loss. The classification performance of WL-SSGAN is evaluated on sea–land clutter datasets. The experimental results show that WL-SSGAN can improve the performance of the fully supervised classifier with only a small number of labeled samples by utilizing a large number of unlabeled sea–land clutter samples. Further, the proposed weighted loss is superior to both the adversarial loss and the feature matching loss. Additionally, we compare WL-SSGAN with conventional semi-supervised classification methods and demonstrate that WL-SSGAN achieves the highest classification accuracy.

Geometric Multimodal Deep Learning With Multiscaled Graph Wavelet Convolutional Network.

Multimodal data provide complementary information of a natural phenomenon by integrating data from various domains with very different statistical properties. Capturing the intramodality and cross-modality information of multimodal data is the essential capability of multimodal learning methods. The geometry-aware data analysis approaches provide these capabilities by implicitly representing data in various modalities based on their geometric underlying structures. Also, in many applications, data are explicitly defined on an intrinsic geometric structure. Generalizing deep learning methods to the non-Euclidean domains is an emerging research field, which has recently been investigated in many studies. Most of those popular methods are developed for unimodal data. In this article, a multimodal graph wavelet convolutional network (M-GWCN) is proposed as an end-to-end network. M-GWCN simultaneously finds intramodality representation by applying the multiscale graph wavelet transform to provide helpful localization properties in the graph domain of each modality and cross-modality representation by learning permutations that encode correlations among various modalities. M-GWCN is not limited to either the homogeneous modalities with the same number of data or any prior knowledge indicating correspondences between modalities. Several semisupervised node classification experiments have been conducted on three popular unimodal explicit graph-based datasets and five multimodal implicit ones. The experimental results indicate the superiority and effectiveness of the proposed methods compared with both spectral graph domain convolutional neural networks and state-of-the-art multimodal methods.

Aspect-Aware Graph Attention Network for Heterogeneous Information Networks.

Graph Convolutional Networks (GCNs) derive inspiration from recent advances in computer vision, by stacking layers of first-order filters followed by a nonlinear activation function to learn entity or graph embeddings. Although GCNs have been shown to boost the performance of many network analysis tasks, they still face tremendous challenges in learning from Heterogeneous Information Networks (HINs), where relations play a decisive role in knowledge reasoning. What's more, there are multiaspect representations of entities in HINs, and a filter learned in one aspect do not necessarily apply to another. We address these challenges by proposing the Aspect-Aware Graph Attention Network (AGAT), a model that extends GCNs with alternative learnable filters to incorporate entity and relational information. Instead of focusing on learning the general entity embeddings, AGAT learns the adaptive entity embeddings based on prediction scenario. Experiments of link prediction and semi-supervised classification verify the effectiveness of our algorithm.

Towards Semi-Supervised Classification of Abnormal Spectrum Signals Based on Deep Learning

Towards Semi-Supervised Classification of Abnormal Spectrum Signals Based on Deep Learning

Modeling Tree-like Heterophily on Symmetric Matrix Manifolds.

Tree-like structures, characterized by hierarchical relationships and power-law distributions, are prevalent in a multitude of real-world networks, ranging from social networks to citation networks and protein-protein interaction networks. Recently, there has been significant interest in utilizing hyperbolic space to model these structures, owing to its capability to represent them with diminished distortions compared to flat Euclidean space. However, real-world networks often display a blend of flat, tree-like, and circular substructures, resulting in heterophily. To address this diversity of substructures, this study aims to investigate the reconstruction of graph neural networks on the symmetric manifold, which offers a comprehensive geometric space for more effective modeling of tree-like heterophily. To achieve this objective, we propose a graph convolutional neural network operating on the symmetric positive-definite matrix manifold, leveraging Riemannian metrics to facilitate the scheme of information propagation. Extensive experiments conducted on semi-supervised node classification tasks validate the superiority of the proposed approach, demonstrating that it outperforms comparative models based on Euclidean and hyperbolic geometries.

Improving Semi-Supervised Text Classification with Dual Meta-Learning

The goal of semi-supervised text classification (SSTC) is to train a model by exploring both a small number of labeled data and a large number of unlabeled data, such that the learned semi-supervised classifier performs better than the supervised classifier trained on solely the labeled samples. Pseudo-labeling is one of the most widely used SSTC techniques, which trains a teacher classifier with a small number of labeled examples to predict pseudo labels for the unlabeled data. The generated pseudo-labeled examples are then utilized to train a student classifier, such that the learned student classifier can outperform the teacher classifier. Nevertheless, the predicted pseudo labels may be inaccurate, making the performance of the student classifier degraded. The student classifier may perform even worse than the teacher classifier. To alleviate this issue, in this paper, we introduce a dual meta-learning ( DML ) technique for semi-supervised text classification, which improves the teacher and student classifiers simultaneously in an iterative manner. Specifically, we propose a meta-noise correction method to improve the student classifier by proposing a Noise Transition Matrix (NTM) with meta-learning to rectify the noisy pseudo labels. In addition, we devise a meta pseudo supervision method to improve the teacher classifier. Concretely, we exploit the feedback performance from the student classifier to further guide the teacher classifier to produce more accurate pseudo labels for the unlabeled data. In this way, both teacher and student classifiers can co-evolve in the iterative training process. Extensive experiments on four benchmark datasets highlight the effectiveness of our DML method against existing state-of-the-art methods for semi-supervised text classification. We release our code and data of this paper publicly at https://github.com/GRIT621/DML.

Building semi-supervised decision trees with semi-cart algorithm

AbstractDecision trees are a fundamental statistical learning tool for addressing classification and regression problems through a recursive partitioning approach that effectively accommodates numerical and categorical data [1, 2]. The Classification and regression tree (CART) algorithm underlies modern Boosting methodologies such as Gradient boosting machine (GBM), Extreme gradient boosting (XGBoost), and Light gradient boosting machine (LightGBM). However, the standard CART algorithm may require improvement due to its inability to learn from unlabeled data. This study proposes several modifications to incorporate test data into the training phase. Specifically, we introduce a method based on Graph-based semi-supervised learning called “Distance-based Weighting,” which calculates and removes irrelevant records from the training set to accelerate the training process and improve performance. We present Semi-supervised classification and regression tree (Semi-Cart), a new implementation of CART that constructs a decision tree using weighted training data. We evaluated its performance on thirteen datasets from various domains. Our results demonstrate that Semi-Cart outperforms standard CART methods and contributes to statistical learning.

Label reusing based graph neural network for unbalanced classification of personalized driver genes in cancer

It is a big challenge to identify personalized driver genes (PDGs) for understanding tumor heterogeneity of cancer individual patients. From the perspective of machine learning, identifying PDGs is an inherent class imbalance issue due to the fewer known driver genes than most passenger ones. However, existing machine learning based methods including unsupervised and supervised learning based methods ignore the importance of limited well-established cancer tissue specific driver genes(CSDGs) for this class imbalance issue. Here we converted the PDG prediction issue to a semi-supervised classification task and a novel method (namely PersonalizedGNN) was developed to identify PDGs by using graph attention neural network and label reuse strategy in personalized gene interaction network (PGIN). PersonalizedGNN effectively utilizes the structure information of PGIN and the limited well-established CSDG information for achieving promising performance. Using the breast cancer and lung cancer datasets from The Cancer Genome Atlas, we validated our method and compared it with other advanced methods. PersonalizedGNN showed outstanding potential in identifying cancer driver genes in terms of prediction precision. Furthermore, we could discover subtype-specific de novo cancer driver genes and in vitro cell-based assays for a novel driver gene FZD7 in lung squamous cell carcinoma cells further validated the PersonalizedGNN. In summary, PersonalizedGNN offers a new effective perspective for discovering PDGs by considering information of prior known CSDGs in PGIN which help researchers understand tumor heterogeneity of cancer individual patients.

Randomized feature selection based semi-supervised latent Dirichlet allocation for microbiome analysis

Health and disease are fundamentally influenced by microbial communities and their genes (the microbiome). An in-depth analysis of microbiome structure that enables the classification of individuals based on their health can be crucial in enhancing diagnostics and treatment strategies to improve the overall well-being of an individual. In this paper, we present a novel semi-supervised methodology known as Randomized Feature Selection based Latent Dirichlet Allocation (RFSLDA) to study the impact of the gut microbiome on a subject’s health status. Since the data in our study consists of fuzzy health labels, which are self-reported, traditional supervised learning approaches may not be suitable. As a first step, based on the similarity between documents in text analysis and gut-microbiome data, we employ Latent Dirichlet Allocation (LDA), a topic modeling approach which uses microbiome counts as features to group subjects into relatively homogeneous clusters, without invoking any knowledge of observed health status (labels) of subjects. We then leverage information from the observed health status of subjects to associate these clusters with the most similar health status making it a semi-supervised approach. Finally, a feature selection technique is incorporated into the model to improve the overall classification performance. The proposed method provides a semi-supervised topic modelling approach that can help handle the high dimensionality of the microbiome data in association studies. Our experiments reveal that our semi-supervised classification algorithm is effective and efficient in terms of high classification accuracy compared to popular supervised learning approaches like SVM and multinomial logistic model. The RFSLDA framework is attractive because it (i) enhances clustering accuracy by identifying key bacteria types as indicators of health status, (ii) identifies key bacteria types within each group based on estimates of the proportion of bacteria types within the groups, and (iii) computes a measure of within-group similarity to identify highly similar subjects in terms of their health status.

Semi-supervised multiview fuzzy broad learning

Semi-supervised learning models often rely on restricted assumptions, and can easily suffer from covariate shift or noise. Few studies have investigated the use of fuzzy rule-based methods in the semi-supervised discipline. To improve model accuracy against covariate shift and to introduce fuzzy methods for interpretability, we first build a semi-supervised fuzzy broad learning model named SSFBLS, which employs a Mean-Teacher framework. Then, a trusted multiview semi-supervised classification method, termed TMSSC, is proposed by integrating the SSFBLS with a multiview fusion network to enhance the robustness of the model. Under the Mean-Teacher framework, SSFBLS involves the Takagi-Sugeno-Kang fuzzy model which can effectively deal with imprecision and uncertainty, and broad learning system which has strong learning ability and high computational efficiency. TMSSC utilizes a trusted mechanism to blend multiple views, so as to enhance its learning ability in semi-supervised scenarios. Experiments on the benchmark datasets demonstrate that the proposed methods have better anti-noise ability, competitive classification accuracy, as well as fast running speed.

Manifold-based multi-graph embedding for semi-supervised classification

Graph semi-supervised learning (GSSL) plays an important role in semi-supervised classification by leveraging the similarity of graph topology and convex optimization with Laplacian-based regularization. Current approaches mainly focus on data representation in the Euclidean space and solely rely on a single graph structure, ignoring the underlined manifold-valued structure of the data. To this end, we propose a novel manifold-based multi-graph embedding (M2GE), which explicitly learns representations on both Riemannian manifold and Euclidean space in the embedding learning process. Incorporating multi-graph structure, the rich and varied features are used by M2GE to more fully capture categorical information, where the heterogeneous spatial representations from manifold are complementary. Extensive experimental results on three popular benchmarks verify demonstrate the superiority of our method over state-of-the art competitors.

Building Shortcuts between Distant Nodes with Biaffine Mapping for Graph Convolutional Networks

Multiple recent studies show a paradox in graph convolutional networks (GCNs)—that is, shallow architectures limit the capability of learning information from high-order neighbors, whereas deep architectures suffer from over-smoothing or over-squashing. To enjoy the simplicity of shallow architectures and overcome their limits of neighborhood extension, in this work we introduce a biaffine technique to improve the expressiveness of GCNs with a shallow architecture. The core design of our method is to learn direct dependency on long-distance neighbors for nodes, with which only 1-hop message passing is capable of capturing rich information for node representation. Besides, we propose a multi-view contrastive learning method to exploit the representations learned from long-distance dependencies. Extensive experiments on nine graph benchmark datasets suggest that the shallow biaffine graph convolutional networks (BAGCN) significantly outperform state-of-the-art GCNs (with deep or shallow architectures) on semi-supervised node classification. We further verify the effectiveness of biaffine design in node representation learning and the performance consistency on different sizes of training data.