Semi-supervised Node Classification Research Articles

Geometric Multimodal Deep Learning With Multiscaled Graph Wavelet Convolutional Network.

Multimodal data provide complementary information of a natural phenomenon by integrating data from various domains with very different statistical properties. Capturing the intramodality and cross-modality information of multimodal data is the essential capability of multimodal learning methods. The geometry-aware data analysis approaches provide these capabilities by implicitly representing data in various modalities based on their geometric underlying structures. Also, in many applications, data are explicitly defined on an intrinsic geometric structure. Generalizing deep learning methods to the non-Euclidean domains is an emerging research field, which has recently been investigated in many studies. Most of those popular methods are developed for unimodal data. In this article, a multimodal graph wavelet convolutional network (M-GWCN) is proposed as an end-to-end network. M-GWCN simultaneously finds intramodality representation by applying the multiscale graph wavelet transform to provide helpful localization properties in the graph domain of each modality and cross-modality representation by learning permutations that encode correlations among various modalities. M-GWCN is not limited to either the homogeneous modalities with the same number of data or any prior knowledge indicating correspondences between modalities. Several semisupervised node classification experiments have been conducted on three popular unimodal explicit graph-based datasets and five multimodal implicit ones. The experimental results indicate the superiority and effectiveness of the proposed methods compared with both spectral graph domain convolutional neural networks and state-of-the-art multimodal methods.

Building Shortcuts between Distant Nodes with Biaffine Mapping for Graph Convolutional Networks

Multiple recent studies show a paradox in graph convolutional networks (GCNs)—that is, shallow architectures limit the capability of learning information from high-order neighbors, whereas deep architectures suffer from over-smoothing or over-squashing. To enjoy the simplicity of shallow architectures and overcome their limits of neighborhood extension, in this work we introduce a biaffine technique to improve the expressiveness of GCNs with a shallow architecture. The core design of our method is to learn direct dependency on long-distance neighbors for nodes, with which only 1-hop message passing is capable of capturing rich information for node representation. Besides, we propose a multi-view contrastive learning method to exploit the representations learned from long-distance dependencies. Extensive experiments on nine graph benchmark datasets suggest that the shallow biaffine graph convolutional networks (BAGCN) significantly outperform state-of-the-art GCNs (with deep or shallow architectures) on semi-supervised node classification. We further verify the effectiveness of biaffine design in node representation learning and the performance consistency on different sizes of training data.

GANN: Graph Alignment Neural Network for semi-supervised learning

Graph neural networks (GNNs) have been widely investigated in the field of semi-supervised graph machine learning. Most methods fail to exploit adequate graph information when labeled data is limited, leading to the problem of oversmoothing. To overcome this issue, we propose the Graph Alignment Neural Network (GANN), a simple and effective graph neural architecture. A unique learning algorithm with three alignment rules is proposed to thoroughly explore hidden information for insufficient labels. Firstly, to better investigate attribute specifics, we suggest the feature alignment rule to align the inner product of both the attribute and embedding matrices. Secondly, to properly utilize the higher-order neighbor information, we propose the cluster center alignment rule, which involves aligning the inner product of the cluster center matrix with the unit matrix. Finally, to get reliable prediction results with few labels, we establish the minimum entropy alignment rule by lining up the prediction probability matrix with its sharpened result. Extensive studies on graph benchmark datasets demonstrate that GANN can achieve considerable benefits in semi-supervised node classification and outperform state-of-the-art competitors.

Reverse Graph Learning for Graph Neural Network.

Graph neural networks (GNNs) conduct feature learning by taking into account the local structure preservation of the data to produce discriminative features, but need to address the following issues, i.e., 1) the initial graph containing faulty and missing edges often affect feature learning and 2) most GNN methods suffer from the issue of out-of-example since their training processes do not directly generate a prediction model to predict unseen data points. In this work, we propose a reverse GNN model to learn the graph from the intrinsic space of the original data points as well as to investigate a new out-of-sample extension method. As a result, the proposed method can output a high-quality graph to improve the quality of feature learning, while the new method of out-of-sample extension makes our reverse GNN method available for conducting supervised learning and semi-supervised learning. Experimental results on real-world datasets show that our method outputs competitive classification performance, compared to state-of-the-art methods, in terms of semi-supervised node classification, out-of-sample extension, random edge attack, link prediction, and image retrieval.

Unsupervised Neighborhood Propagation Kernel Layers for Semi-supervised Node Classification

We present a deep Graph Convolutional Kernel Machine (GCKM) for semi-supervised node classification in graphs. The method is built of two main types of blocks: (i) We introduce unsupervised kernel machine layers propagating the node features in a one-hop neighborhood, using implicit node feature mappings. (ii) We specify a semi-supervised classification kernel machine through the lens of the Fenchel-Young inequality. We derive an effective initialization scheme and efficient end-to-end training algorithm in the dual variables for the full architecture. The main idea underlying GCKM is that, because of the unsupervised core, the final model can achieve higher performance in semi-supervised node classification when few labels are available for training. Experimental results demonstrate the effectiveness of the proposed framework.

NodeMixup: Tackling Under-Reaching for Graph Neural Networks

Graph Neural Networks (GNNs) have become mainstream methods for solving the semi-supervised node classification problem. However, due to the uneven location distribution of labeled nodes in the graph, labeled nodes are only accessible to a small portion of unlabeled nodes, leading to the under-reaching issue. In this study, we firstly reveal under-reaching by conducting an empirical investigation on various well-known graphs. Then, we demonstrate that under-reaching results in unsatisfactory distribution alignment between labeled and unlabeled nodes through systematic experimental analysis, significantly degrading GNNs' performance. To tackle under-reaching for GNNs, we propose an architecture-agnostic method dubbed NodeMixup. The fundamental idea is to (1) increase the reachability of labeled nodes by labeled-unlabeled pairs mixup, (2) leverage graph structures via fusing the neighbor connections of intra-class node pairs to improve performance gains of mixup, and (3) use neighbor label distribution similarity incorporating node degrees to determine sampling weights for node mixup. Extensive experiments demonstrate the efficacy of NodeMixup in assisting GNNs in handling under-reaching. The source code is available at https://github.com/WeigangLu/NodeMixup.

A Fully Test-Time Training Framework for Semi-Supervised Node Classification on Out-of-Distribution Graphs

Graph neural networks (GNNs) have shown great potential in representation learning for various graph tasks. However, the distribution shift between the training and test sets poses a challenge to the efficiency of GNNs. To address this challenge, HomoTTT propose a fully test-time training (FTTT) framework for GNNs to enhance the model’s generalization capabilities for node classification tasks. Specifically, our proposed HomoTTT designs a homophily-based and parameter-free graph contrastive learning task with adaptive augmentation to guide the model’s adaptation during the test time training, allowing the model to adapt for specific target data. In the inference stage, HomoTTT proposes to integrate the original GNN model and the adapted model after TTT using a homophily-based model selection method, which prevents potential performance degradation caused by unconstrained model adaptation. Extensive experimental results on six benchmark datasets demonstrate the effectiveness of our proposed framework. Additionally, the exploratory study further validates the rationality of the homophily-based graph contrastive learning task with adaptive augmentation and the homophily-based model selection designed in HomoTTT .

LGAT: a light graph attention network focusing on message passing for semi-supervised node classification

LGAT: a light graph attention network focusing on message passing for semi-supervised node classification

HITS-based Propagation Paradigm for Graph Neural Networks

In this article, we present a new propagation paradigm based on the principle of Hyperlink-Induced Topic Search (HITS) algorithm. The HITS algorithm utilizes the concept of a “self-reinforcing” relationship of authority-hub. Using HITS, the centrality of nodes is determined via repeated updates of authority-hub scores that converge to a stationary distribution. Unlike PageRank-based propagation methods, which rely solely on the idea of authorities (in-links), HITS considers the relevance of both authorities (in-links) and hubs (out-links), thereby allowing for a more informative graph learning process. To segregate node prediction and propagation, we use a Multilayer Perceptron in combination with a HITS-based propagation approach and propose two models: HITS-GNN and HITS-GNN+. We provided additional validation of our models’ efficacy by performing an ablation study to assess the performance of authority-hub in independent models. Moreover, the effect of the main hyper-parameters and normalization is also analyzed to uncover how these techniques influence the performance of our models. Extensive experimental results indicate that the proposed approach significantly improves baseline methods on the graph (citation network) benchmark datasets by a decent margin for semi-supervised node classification, which can aid in predicting the categories (labels) of scientific articles not exclusively based on their content but also based on the type of articles they cite.

Mixed Graph Contrastive Network for Semi-Supervised Node Classification

Graph Neural Networks (GNNs) have achieved promising performance in semi-supervised node classification in recent years. However, the problem of insufficient supervision, together with representation collapse, largely limits the performance of the GNNs in this field. To alleviate the collapse of node representations in semi-supervised scenario, we propose a novel graph contrastive learning method, termed M ixed G raph C ontrastive N etwork (MGCN). In our method, we improve the discriminative capability of the latent embeddings by an interpolation-based augmentation strategy and a correlation reduction mechanism. Specifically, we first conduct the interpolation-based augmentation in the latent space and then force the prediction model to change linearly between samples. Second, we enable the learned network to tell apart samples across two interpolation-perturbed views through forcing the correlation matrix across views to approximate an identity matrix. By combining the two settings, we extract rich supervision information from both the abundant unlabeled nodes and the rare yet valuable labeled nodes for discriminative representation learning. Extensive experimental results on six datasets demonstrate the effectiveness and the generality of MGCN compared to the existing state-of-the-art methods. The code of MGCN is available at https://github.com/xihongyang1999/MGCN on Github.

Contrastive learning of graphs under label noise

In the domain of graph-structured data learning, semi-supervised node classification serves as a critical task, relying mainly on the information from unlabeled nodes and a minor fraction of labeled nodes for training. However, real-world graph-structured data often suffer from label noise, which significantly undermines the performance of Graph Neural Networks (GNNs). This problem becomes increasingly severe in situations where labels are scarce. To tackle this issue of sparse and noisy labels, we propose a novel approach Contrastive Robust Graph Neural Network (CR-GNN), Firstly, considering label sparsity and noise, we employ unsupervised contrastive loss and further incorporate homophily in the graph structure, thus introducing neighbor contrastive loss. Moreover, data augmentation is typically used to construct positive and negative samples in contrastive learning, which may result in inconsistent prediction outcomes. Based on this, we propose a dynamic cross-entropy loss, which selects the nodes with consistent predictions as reliable nodes for cross-entropy loss and benefits to mitigate the overfitting to labeling noise. Finally, we propose cross-space consistency to narrow the semantic gap between the contrast and classification spaces. Extensive experiments on multiple publicly available datasets demonstrate that CR-GNN notably outperforms existing methods in resisting label noise.

Multimodal graph learning based on 3D Haar semi-tight framelet for student engagement prediction

With the increasing availability of multimodal educational data, there is a growing need to effectively integrate and exploit multiple data sources to enhance student engagement prediction accuracy. In this work, we propose a framework that combines multimodal data, including visual, textual and acoustic modalities that reflect the students’ personalities, their demographic information, their learning behavior and attention, with graph learning techniques. Specifically, 3D Haar semi-tight framelet transforms are developed to capture the inter-modal relationships and model the complex interactions within the multimodal data. Subsequently, we introduce a novel module for adaptive graph structure learning based on the spectrum of multimodal data, which takes into consideration the distinct contributions of low-pass and high-pass framelet coefficients by adaptively weighing their impact. By addressing a standard semi-supervised node classification problem, we successfully achieve the objective of student engagement prediction. The experiment evaluations on a real-world educational dataset demonstrate the effectiveness of the proposed approach, achieving superior performance compared to state-of-the-art methods. Our experimental studies demonstrate the importance of multimodal graph learning in accurately predicting student engagement and its potential to enhance educational outcomes.

DAHGN: Degree-Aware Heterogeneous Graph Neural Network

In recent years, Graph Neural Networks (GNNs), an emerging technology for learning from graph-structured data, have attracted much attention. Despite the widespread applications of GNNs across diverse domains, recent studies have shown that insufficient information aggregation for low-degree nodes (i.e., tail nodes) leads to inferior performance of GNNs. While numerous recent studies have aimed to address the issue of degree bias in homogeneous graphs, there are limited studies on analyzing such unfairness for heterogeneous information networks. In this article, we propose a novel semi-supervised contrastive learning framework named Degree-Aware Heterogeneous Graph Neural Network (DAHGN), which achieves promising performance by alleviating the degree bias. In brief, we first construct two contrastive graph views, one from a heterogeneous graph that focuses on capturing complementary information to enhance the node representations and one from an extracted homogeneous subgraph that focuses on applying different strategies to low- and high- degree nodes to eliminate the degree bias. We then leverage the label learned from the heterogeneous view to reconstruct the positive node pairs in the contrastive learning framework. Finally, we combine the task loss (semi-supervised) and the contrastive loss to optimize the model performance. We conduct extensive experiments on benchmark datasets to validate the effectiveness of our model, and the results demonstrate that the DAHGN outperforms a wide range of state-of-the-art methods in the semi-supervised node classification problem.

Toward Robust Graph Semi-Supervised Learning Against Extreme Data Scarcity.

The success of graph neural networks (GNNs) in graph-based web mining highly relies on abundant human-annotated data, which is laborious to obtain in practice. When only a few labeled nodes are available, how to improve their robustness is key to achieving replicable and sustainable graph semi-supervised learning. Though self-training is powerful for semi-supervised learning, its application on graph-structured data may fail because 1) larger receptive fields are not leveraged to capture long-range node interactions, which exacerbates the difficulty of propagating feature-label patterns from labeled nodes to unlabeled nodes and 2) limited labeled data makes it challenging to learn well-separated decision boundaries for different node classes without explicitly capturing the underlying semantic structure. To address the challenges of capturing informative structural and semantic knowledge, we propose a new graph data augmentation framework, augmented graph self-training (AGST), which is built with two new (i.e., structural and semantic) augmentation modules on top of a decoupled GST backbone. In this work, we investigate whether this novel framework can learn a robust graph predictive model under the low-data context. We conduct comprehensive evaluations on semi-supervised node classification under different scenarios of limited labeled-node data. The experimental results demonstrate the unique contributions of the novel data augmentation framework for node classification with few labeled data.

Pseudo Labeling Collaborative Embedding Representation: A Multi-Channel GCNs Framework for Semi-Supervised Node Classification

Pseudo Labeling Collaborative Embedding Representation: A Multi-Channel GCNs Framework for Semi-Supervised Node Classification

Neighborhood Pattern Is Crucial for Graph Convolutional Networks Performing Node Classification.

Graph convolutional networks (GCNs) are widely believed to perform well in the graph node classification task, and homophily assumption plays a core rule in the design of previous GCNs. However, some recent advances on this area have pointed out that homophily may not be a necessity for GCNs. For deeper analysis of the critical factor affecting the performance of GCNs, we first propose a metric, namely, neighborhood class consistency (NCC), to quantitatively characterize the neighborhood patterns of graph datasets. Experiments surprisingly illustrate that our NCC is a better indicator, in comparison to the widely used homophily metrics, to estimate GCN performance for node classification. Furthermore, we propose a topology augmentation graph convolutional network (TA-GCN) framework under the guidance of the NCC metric, which simultaneously learns an augmented graph topology with higher NCC score and a node classifier based on the augmented graph topology. Extensive experiments on six public benchmarks clearly show that the proposed TA-GCN derives ideal topology with higher NCC score given the original graph topology and raw features, and it achieves excellent performance for semi-supervised node classification in comparison to several state-of-the-art (SOTA) baseline algorithms.

Perturbation-augmented Graph Convolutional Networks: A Graph Contrastive Learning architecture for effective node classification tasks

In the context of recent advances in Graph Convolutional Networks (GCNs) for semi-supervised learning, a significant highlight is the potential of Graph Contrastive Learning (GCL). Many established GCL-based GCNs focus on developing various augmentation strategies and learning architectures to efficiently generate and learn self-supervised information. Unfortunately, the node representation fusion is often neglected in the process of learning representation. To address the issue, we propose a novel end-to-end GCL framework called Perturbation-augmented Graph Convolutional Networks (PA-GCN), by simultaneously considering supervised information and self-supervised information of graphs to fuse node representations from the input graph and the augmentation graphs. Essentially, PA-GCN develops a perturbation augmentation strategy based on original graph relations, serving as a basis to generate diverse node information and self-supervised information. Additionally, PA-GCN introduces three main components: A encoder for node representation learning; an attention mechanism for effective node representation fusion; and a discriminator capable of learning self-supervised information from unlabeled nodes, thereby enabling joint learning of supervised information, node representation fusion, and self-supervised information. We conduct experiments on five real world datasets to evaluate the performance of PA-GCN in semi-supervised node classification tasks. The experimental results demonstrate the strong adaptability of PA-GCN for base encoders and the superiority of PA-GCN over the most advanced methods.

Label Efficient Regularization and Propagation for Graph Node Classification.

An enhanced label propagation (LP) method called GraphHop was proposed recently. It outperforms graph convolutional networks (GCNs) in the semi-supervised node classification task on various networks. Although the performance of GraphHop was explained intuitively with joint node attribute and label signal smoothening, its rigorous mathematical treatment is lacking. In this paper, we propose a label efficient regularization and propagation (LERP) framework for graph node classification, and present an alternate optimization procedure for its solution. Furthermore, we show that GraphHop only offers an approximate solution to this framework and has two drawbacks. First, it includes all nodes in the classifier training without taking the reliability of pseudo-labeled nodes into account in the label update step. Second, it provides a rough approximation to the optimum of a subproblem in the label aggregation step. Based on the LERP framework, we propose a new method, named the LERP method, to solve these two shortcomings. LERP determines reliable pseudo-labels adaptively during the alternate optimization and provides a better approximation to the optimum with computational efficiency. Theoretical convergence of LERP is guaranteed. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of LERP. That is, LERP outperforms all benchmarking methods, including GraphHop, consistently on five common test datasets, two large-scale networks, and an object recognition task at extremely low label rates (i.e., 1, 2, 4, 8, 16, and 20 labeled samples per class).

Toward Effective Semi-supervised Node Classification with Hybrid Curriculum Pseudo-labeling

Semi-supervised node classification is a crucial challenge in relational data mining and has attracted increasing interest in research on graph neural networks (GNNs). However, previous approaches merely utilize labeled nodes to supervise the overall optimization, but fail to sufficiently explore the information of their underlying label distribution. Even worse, they often overlook the robustness of models, which may cause instability of network outputs to random perturbations. To address the aforementioned shortcomings, we develop a novel framework termed Hybrid Curriculum Pseudo-Labeling (HCPL) for efficient semi-supervised node classification. Technically, HCPL iteratively annotates unlabeled nodes by training a GNN model on the labeled samples and any previously pseudo-labeled samples, and repeatedly conducts this process. To improve the model robustness, we introduce a hybrid pseudo-labeling strategy that incorporates both prediction confidence and uncertainty under random perturbations, therefore mitigating the influence of erroneous pseudo-labels. Finally, we leverage the idea of curriculum learning to start from annotating easy samples, and gradually explore hard samples as the iteration grows. Extensive experiments on a number of benchmarks demonstrate that our HCPL beats various state-of-the-art baselines in diverse settings.

Label-guided graph contrastive learning for semi-supervised node classification

Semi-supervised node classification is a task of predicting the labels of unlabeled nodes using limited labeled nodes and numerous unlabeled nodes. Recently, Graph Neural Networks (GNNs) have achieved remarkable success in this task. However, GNNs typically have shallow architectures and only consider labeled nodes and their low-order neighbors during training. As a result, the supervision signals from the massive unlabeled nodes remain underutilized. To address this limitation, graph contrastive learning has been applied in semi-supervised node classification, which pulls together positive nodes and pushes away negative nodes in the embedding space. Nevertheless, existing node-level contrastive learning methods usually sample the same node from two augmented views as positive nodes and all different nodes as negative nodes. Consequently, many semantically similar nodes are not sampled as positive nodes but are mistakenly sampled as negative nodes. To tackle this issue, we propose a novel Label-guided Graph Contrastive Learning (LGGCL) training algorithm for semi-supervised node classification. Specifically, we first propose a Label-guided Graph Contrastive Learning framework as the basis of LGGCL training algorithm. Then we incorporate a self-checking mechanism based on deep clustering to ensure the authenticity of the sampled positive nodes. Moreover, we design a reweighting strategy based on the probability distribution of the anchor node to enhance the effect of hard negative nodes. Finally, experimental results on various graph benchmarks demonstrate the superiority of our LGGCL.