Semi-supervised Node Classification Research Articles

Self-Paced Co-Training of Graph Neural Networks for Semi-Supervised Node Classification.

Graph neural networks (GNNs) have demonstrated great success in many graph data-based applications. The impressive behavior of GNNs typically relies on the availability of a sufficient amount of labeled data for model training. However, in practice, obtaining a large number of annotations is prohibitively labor-intensive and even impossible. Co-training is a popular semi-supervised learning (SSL) paradigm, which trains multiple models based on a common training set while augmenting the limited amount of labeled data used for training each model via the pseudolabeled data generated from the prediction results of other models. Most of the existing co-training works do not control the quality of pseudolabeled data when using them. Therefore, the inaccurate pseudolabels generated by immature models in the early stage of the training process are likely to cause noticeable errors when they are used for augmenting the training data for other models. To address this issue, we propose a self-paced co-training for the GNN (SPC-GNN) framework for semi-supervised node classification. This framework trains multiple GNNs with the same or different structures on different representations of the same training data. Each GNN carries out SSL by using both the originally available labeled data and the augmented pseudolabeled data generated from other GNNs. To control the quality of pseudolabels, a self-paced label augmentation strategy is designed to make the pseudolabels generated at a higher confidence level to be utilized earlier during training such that the negative impact of inaccurate pseudolabels on training data augmentation, and accordingly, the subsequent training process can be mitigated. Finally, each of the trained GNN is evaluated on a validation set, and the best-performing one is chosen as the output. To improve the training effectiveness of the framework, we devise a pretraining followed by a two-step optimization scheme to train GNNs. Experimental results on the node classification task demonstrate that the proposed framework achieves significant improvement over the state-of-the-art SSL methods.

LaenNet: Learning robust GCNs by propagating labels

Graph Convolutional Networks (GCNs) can be acknowledged as one of the most significant methodologies for graph representation learning, and the family of GCNs has recently achieved great success in the community. However, in real-world scenarios, the graph data may be imperfect, e.g., with noisy and sparse features or labels, which poses a great challenge to the robustness of GCNs. To meet this challenge, we propose a simple-yet-effective LAbel-ENhanced Networks (LaenNet) architecture for GCNs, where the basic spirit is to propagate labels together with features. Specifically, we add an extra LaenNet module at one hidden layer of GCNs, which propagates labels along the graph and then integrates them with the hidden representations as the inputs to the deeper layer. The proposed LaenNet can be directly generalized to the variants of GCNs. We conduct extensive experiments to verify LaenNet on semi-supervised node classification tasks under four noisy and sparse graph data scenarios, including the graphs with noisy features, sparse features, noisy labels, and sparse labels. Empirical results indicate the superiority and robustness of LaenNet compared to the state-of-the-art baseline models. The implementation code is available to ease reproducibility11https://github.com/Zhangcx19/LAENNet..

Enhanced Graph Neural Network with Multi-Task Learning and Data Augmentation for Semi-Supervised Node Classification

Graph neural networks (GNNs) have achieved impressive success in various applications. However, training dedicated GNNs for small-scale graphs still faces many problems such as over-fitting and deficiencies in performance improvements. Traditional methods such as data augmentation are commonly used in computer vision (CV) but are barely applied to graph structure data to solve these problems. In this paper, we propose a training framework named MTDA (Multi-Task learning with Data Augmentation)-GNN, which combines data augmentation and multi-task learning to improve the node classification performance of GNN on small-scale graph data. First, we use Graph Auto-Encoders (GAE) as a link predictor, modifying the original graphs’ topological structure by promoting intra-class edges and demoting interclass edges, in this way to denoise the original graph and realize data augmentation. Then the modified graph is used as the input of the node classification model. Besides defining the node pair classification as an auxiliary task, we introduce multi-task learning during the training process, forcing the predicted labels to conform to the observed pairwise relationships and improving the model’s classification ability. In addition, we conduct an adaptive dynamic weighting strategy to distribute the weight of different tasks automatically. Experiments on benchmark data sets demonstrate that the proposed MTDA-GNN outperforms traditional GNNs in graph-based semi-supervised node classification.

CB-SAGE: A novel centrality based graph neural network for floor plan classification

Graph Neural Networks (GNNs) have emerged as one of the most prominent research areas in accomplishing machine learning tasks over graphical networks. GNNs are prominently used in performing tasks like semi-supervised node classification, link prediction, and community detection. GraphSAGE is one of the most recent GNN models which is being used to accomplish these tasks. A floor plan is an architectural design of a building that represents various floor compartments. In this paper, we represent floor plan(s) as graph(s). We characterize floor room (compartment) classification as a node classification problem. We propose a variation of the traditional GraphSAGE (sample and aggregation) algorithm: Centrality based GraphSAGE (CB-SAGE), which captures the structural properties of the network. We use the average clustering coefficient and average betweenness centrality to capture structure properties. We compute CB scores for all the floor plan graphs. Top 70% nodes (based on CB scores) are selected for the training. During the training, we append the betweenness centrality score of each node as an additional feature in the feature matrix for the embedding process. We conduct experiments on the House-GAN dataset, which contains 1,43,184 vectorized floor plan images. The proposed method outperforms the current state-of-art models in accomplishing the task of floor plan classification. We compare our results with the traditional machine learning approach (MLP) and other GNN-based methods. Our approach achieves an accuracy of 96.70%, which is significantly (approximately 16%) higher than other state-of-the-art methods.

Individuality-enhanced and multi-granularity consistency-preserving graph neural network for semi-supervised node classification

Semi-supervised node classification is an important task that aims at classifying nodes based on the graph structure, node features, and class labels for a subset of nodes. While most graph convolutional networks (GCNs) perform well when an ample number of labeled nodes are available, they often degenerate when the amount of labeled data is limited. To address this problem, we propose a scheme, namely, Individuality-enhanced and Multi-granularity Consistency-preserving graph neural Network (IMCN), which can alleviate the problem of losing individual information within the encoder while providing a reliable supervised signal for learning purposes. First, one simple encoder based on node features only is integrated to enhance node individuality and amend node commonality learned by the GCN-based encoder. Then, three constraints are defined at different levels of granularity, encompassing node embedding agreement, semantic class alignment, and node-to-class distribution identity. They can maintain the consistency between the individuality and commonality of nodes and be leveraged as latent supervised signals for learning representative embeddings. Finally, the trade-off between the individuality and commonality of nodes captured by two encoders is taken into consideration for node classification. Extensive experiments on six real-world datasets have been conducted to validate the superiority of IMCN against state-of-the-art baselines in handling node classification tasks with scarce labeled data.Graphical abstract

Improving Node Classification Accuracy of GNN through Input and Output Intervention

Graph Neural Networks (GNNs) are a popular machine learning framework for solving various graph processing applications. This framework exploits both the graph topology and the feature vectors of the nodes. One of the important applications of GNN is in the semi-supervised node classification task. The accuracy of the node classification using GNN depends on (i) the number and (ii) the choice of the training nodes. In this article, we demonstrate that increasing the training nodes by selecting nodes from the same class that are spread out across non-contiguous subgraphs, can significantly improve the accuracy. We accomplish this by presenting a novel input intervention technique that can be used in conjunction with different GNN classification methods to increase the non-contiguous training nodes and, thereby, improve the accuracy. We also present an output intervention technique to identify misclassified nodes and relabel them with their potentially correct labels. We demonstrate on real-world networks that our proposed methods, both individually and collectively, significantly improve the accuracy in comparison to the baseline GNN algorithms. Both our methods are agnostic. Apart from the initial set of training nodes generated by the baseline GNN methods, our techniques do not need any other extra knowledge about the classes of the nodes. Thus, our methods are modular and can be used as pre-and post-processing steps with many of the currently available GNN methods to improve their accuracy.

Dual Feature Interaction-Based Graph Convolutional Network

Graphs are widely used to model various practical applications. In recent years, graph convolution networks (GCNs) have attracted increasing attention due to the extension of convolution operation from traditional grid data to graph one. However, the representation ability of current GCNs is undoubtedly limited because existing work fails to consider feature interactions. Toward this end, we propose a Dual Feature Interaction-based GCN. Specifically, it models feature interaction in the aspects of 1) node features where we use Newton's identity to extract different-order cross features implicit in the original features and design an attention mechanism to fuse them; and 2) graph convolution where we capture the pairwise interactions among nodes in the neighborhood to expand a weighted sum operation. We evaluate the proposed model with graph data from different fields, and the experimental results on semi-supervised node classification and link prediction demonstrate the effectiveness of the proposed GCN. The data and source codes of this work are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ZZY-GraphMiningLab/DFI-GCN</uri> .

Ridge-Regression-Induced Robust Graph Relational Network.

Graph convolutional networks (GCNs) have attracted increasing research attention, which merits in its strong ability to handle graph data, such as the citation network or social network. Existing models typically use first-order neighborhood information to design specific convolution operations, which aggregate the features of all adjacent nodes. However, such models ignore the high-order spatial relationship among neighboring nodes in noisy data due to its modeling complexity. In this article, we propose a novel robust graph relational network to address this issue toward modeling high-order relationships in noisy data for graph convolution. Our key innovation lies in designing a generic relation network layer, which is used to infer the underlying relations among adjacent noisy nodes. Specifically, a fixed number of adjacent nodes for each node is chosen by solving the ridge regression problem, in which the regression coefficients are used to rank the adjacent nodes of each node in a graph. Furthermore, to mine the rich features, we extract high-order information from the nodes to significantly enhance the representation ability of the GCNs for extensive applications. We conduct extensive semisupervised node classification experiments on the noisy benchmark datasets, which clearly show that our model is superior to the existing methods and can achieve state-of-the-art performance.

Dual Mutual Robust Graph Convolutional Network for Weakly Supervised Node Classification in Social Networks of Internet of People

Social networks are a crucial component of the Internet of People (IoP) which is the frontier of the next generation of Internet of Things (IoT). Predicting a large number of unknown node labels with few known labels is one of the challenging problems in social networks analysis. Fortunately, Graph Convolution Network (GCN) and subsequent variants have achieved impressive performance on Semi-Supervised Node Classification (SSNC). However, existing works only focus on the case of clean labels and rarely study the problem of SSNC under Noisy Labels (SSNCNL), which is a more challenging and practical problem in the field of weakly supervised learning. To address this challenge, in this paper, we propose a novel Dual Mutual Robust Graph Convolutional Network named DMRGCN, which is inspired by deep mutual learning and robust learning in the domain of image recognition. Specifically, we first employ two GCNs with different learning abilities to construct network architecture. Then we define a new loss function which is a weighted sum of the supervised loss, mutual loss, and robust loss. Finally, we train the network in the paradigm of pseudo-siamese network. Experimental results on three social network benchmark datasets with different levels of noise on labels demonstrate that DMRGCN outperforms the vanilla GCN and several variants in terms of classification accuracy. In particular, under the two conditions of clean labels and noisy labels, the accuracy of node classification obtained by our proposed DMRGCN can be 3.05% and 6.44% higher than that of the vanilla GCN, respectively.

Self-Consistent Graph Neural Networks for Semi-Supervised Node Classification

Graph Neural Networks (GNNs), the powerful graph representation technique based on deep learning, have attracted great research interest in recent years. Although many GNNs have achieved the state-of-the-art accuracy on a set of standard benchmark datasets, they are still limited to traditional semi-supervised framework and lack of sufficient supervision information, especially for the large amount of unlabeled data. To overcome this issue, we propose a novel self-consistent graph neural networks (SCGNN) framework to enrich the supervision information from two aspects: the self-consistency of unlabeled data and the label information of labeled data. First, in order to extract the self-supervision information from the numerous unlabeled nodes, we perform graph data augmentation and leverage a self-consistent constraint to maximize the mutual information of the unlabeled nodes across different augmented graph views. The self-consistency can sufficiently utilize the intrinsic structural attributes of the graph to extract the self-supervision information from unlabeled data and improve the subsequent classification result. Second, to further extract supervision information from scarce labeled nodes, we introduce a fusion mechanism to obtain comprehensive node embeddings by fusing node representations of two positive graph views, and optimize the classification loss over labeled nodes to maximize the utilization of label information. We conduct comprehensive empirical studies on six public benchmark datasets in node classification task. In terms of accuracy, SCGNN improves by an average of 2.08% over the best baseline, and specifically by 5.8% on the Disease dataset.

MVMA-GCN: Multi-view multi-layer attention graph convolutional networks

The accuracy of graph representation learning is highly dependent on the precise characterization of node relationships. However, representing the complex and diverse networks in the real world using a single type of node or link is challenging, often resulting in incomplete information. Moreover, different types of nodes and links convey rich information, which makes it difficult to design a graph network that can integrate diverse links. This paper introduces a novel multi-view and multi-layer attention model designed to optimize node embeddings for semi-supervised node classification. The proposed model exploits various types of inter-node links and employs the Hilbert–Schmidt independence criterion to maximize the dissimilarity between distinct node relationships. Furthermore, the multi-layer attention mechanism is used to discern the impact of different neighboring nodes and relationships between various node relationships. The performance of the proposed model, MVMA-GCN, was assessed on numerous real-world multi-view datasets. It was observed that MVMA-GCN consistently outperformed existing models, demonstrating superior accuracy in semi-supervised classification tasks. We have made our code publicly available at here to ensure the reproducibility of our results.

Topological enhanced graph neural networks for semi-supervised node classification

Topological enhanced graph neural networks for semi-supervised node classification

Deep Graph-Convolutional Generative Adversarial Network for Semi-Supervised Learning on Graphs

Graph convolutional networks (GCNs) are neural network frameworks for machine learning on graphs. They can simultaneously perform end-to-end learning on the attribute information and the structure information of graph data. However, most existing GCNs inevitably encounter the limitations of non-robustness and low classification accuracy when labeled nodes are scarce. To address the two issues, the deep graph convolutional generative adversarial network (DGCGAN), a model combining GCN and deep convolutional generative adversarial networks (DCGAN), is proposed in this paper. First, the graph data is mapped to a highly nonlinear space by using the topology and attribute information of the graph for symmetric normalized Laplacian transform. Then, through the feature-structured enhanced module, the node features are expanded into regular structured data, such as images and sequences, which are input to DGCGAN as positive samples, thus expanding the sample capacity. In addition, the feature-enhanced (FE) module is adopted to enhance the typicality and discriminability of node features, and to obtain richer and more representative features, which is helpful for facilitating accurate classification. Finally, additional constraints are added to the network model by introducing DCGAN, thus enhancing the robustness of the model. Through extensive empirical studies on several standard benchmarks, we find that DGCGAN outperforms state-of-the-art baselines on semi-supervised node classification and remote sensing image classification.

LoyalDE: Improving the performance of Graph Neural Networks with loyal node discovery and emphasis

Recent years have witnessed an increasing focus on graph-based semi-supervised learning with Graph Neural Networks (GNNs). Despite existing GNNs having achieved remarkable accuracy, research on the quality of graph supervision information has inadvertently been ignored. In fact, there are significant differences in the quality of supervision information provided by different labeled nodes, and treating supervision information with different qualities equally may lead to sub-optimal performance of GNNs. We refer to this as the graph supervision loyalty problem, which is a new perspective for improving the performance of GNNs. In this paper, we devise FT-Score to quantify node loyalty by considering both the local feature similarity and the local topology similarity, and nodes with higher loyalty are more likely to provide higher-quality supervision. Based on this, we propose LoyalDE (Loyal Node Discovery and Emphasis), a model-agnostic hot-plugging training strategy, which can discover potential nodes with high loyalty to expand the training set, and then emphasize nodes with high loyalty during model training to improve performance. Experiments demonstrate that the graph supervision loyalty problem will fail most existing GNNs. In contrast, LoyalDE brings about at most 9.1% performance improvement to vanilla GNNs and consistently outperforms several state-of-the-art training strategies for semi-supervised node classification.

On the role of network topology in German-Jewish recommendation letter networks in the early twentieth century

Recommendation letters were an important instrument for orchestrating Jewish emigration from Germany in the late nineteenth and early twentieth century. Here, we present network-based analyses of manually collected meta-data from recommendation letters targeted at the Hebrew University (HU) in Jerusalem. Using standard semi-supervised node classification techniques and differential node centrality analyses, we show that the position of a recommendation letter in content-agnostic recommendation network models is predictive of its success, i.e., of whether or not the letter led to the recommendee obtaining a position at the HU. In particular, we show that authors of successful recommendation letters assume more central positions within the networks than authors of unsuccessful letters, while the opposite holds for the recommendation letters’ receivers. Beyond our application, these results showcase the potential of using network models for generating historical insights. Both the letter meta-data records and Python code to reproduce our analyses are available on GitHub: https://github.com/bionetslab/corrnet.

Complex exponential graph convolutional networks

Graph convolutional networks based on spectral methods have recently achieved considerable success in processing non-Euclidean structured data such as graphs. In this paper, we propose the complex exponential graph convolutional network (CEGCN), which is a novel spectral convolutional architecture for performing deep learning on graphs. Specifically, we design a complex exponential polynomial filter with powerful expressive ability and whose zeros can be changed to prevent the over-smoothing issue. Furthermore, a CEGCN variant, called CEGCN-hp, combines the graph Fourier transform-based and high-pass filters to capture the high-frequency components in the graph spectral domain. We perform a spectral analysis to illustrate the motivation and expressive power of the proposed model. Experimental results show that our model matches or outperforms various state-of-the-art baselines on three downstream tasks: semi-supervised node classification, community detection, and graph classification. Our implementation is available online.1

A modified GNN architecture with enhanced aggregator and Message Passing Functions

Graph neural networks (GNN) uphold the essence of irregularly structured information embedded in a graph via message passing among the nodes and aggregating the node features at various levels of the graph. In the past, researchers have extensively used the GNN models for several semi-supervised node classification tasks. Existing GNN models do not use nodes’ information sufficiently. The use of inter-node feature-level correlational information with the existing GNN models might lead to more powerful learning models. Here, a weighting scheme has been developed for message passing and aggregation functions. This model has been named “Vector GNN”, or in short, “VecGNN”, due to its relationship with vector space. VecGNN takes into consideration the relative position of a node with respect to its neighboring nodes in the feature space, which influences the weight of features passed to the information aggregation phase. These weights are assigned using two different statistical measures: Jaccard’s coefficient and Cosine similarity. The proposed weighting scheme uses a generalized approach that can be easily incorporated into several GNN frameworks. VecGNN is evaluated using three citation datasets: Citeseer, Pubmed, and Cora. On these datasets, three sets of experiments have been conducted with varying numbers of training and testing nodes. We have used training, validation, and test set nodes with ratios of 1:1:8, 2:1:7, and 3:1:6. Experimenting on these, we observe an improvement of 2%–4% over the baseline models: Graph Convolution Network (GCN), Graph Attention Network (GAT), and Jumping Knowledge Networks (JKNets). The source code is available at the link https://github.com/sourodeeproy/VecGNN.

Semi-Supervised Node Classification via Semi-Global Graph Transformer Based on Homogeneity Augmentation

As a kind of generalization of Transformers in the graph domain, Global Graph Transformers are good at learning distant knowledge by directly doing information interactions on complete graphs, which differs from Local Graph Transformers interacting on the original structures. However, we find that most prior works focus only on graph-level tasks (e.g., graph classification) and few Graph Transformer models can effectively solve node-level tasks, especially semi-supervised node classification, which obviously has important practical significance due to the limitation and expensiveness of these node labels. In order to fill this gap, this paper first summarizes the theoretical advantages of Graph Transformers. And based on some exploring experiments, we give some discussions on the main cause of their poor practical performance in semi-supervised node classifications. Secondly, based on this analysis, we design a three-stage homogeneity augmentation framework and propose a Semi-Global Graph Transformer. Considering both global and local perspectives, the proposed model combines various technologies including self-distillation, pseudo-label filtering, pre-training and fine-tuning, and metric learning. Furthermore, it simultaneously enhances the structure and the optimization, improving its effectiveness, scalability, and generalizability. Finally, extensive experiments on seven public homogeneous and heterophilous graph benchmarks show that the proposed method can achieve competitive or much better results compared to many baseline models including state-of-the-arts.

Are Graph Convolutional Networks With Random Weights Feasible?

Graph Convolutional Networks (GCNs), as a prominent example of graph neural networks, are receiving extensive attention for their powerful capability in learning node representations on graphs. There are various extensions, either in sampling and/or node feature aggregation, to further improve GCNs' performance, scalability and applicability in various domains. Still, there is room for further improvements on learning efficiency because performing batch gradient descent using the full dataset for every training iteration, as unavoidable for training (vanilla) GCNs, is not a viable option for large graphs. The good potential of random features in speeding up the training phase in large-scale problems motivates us to consider carefully whether GCNs with random weights are feasible. To investigate theoretically and empirically this issue, we propose a novel model termed Graph Convolutional Networks with Random Weights (GCN-RW) by revising the convolutional layer with random filters and simultaneously adjusting the learning objective with regularized least squares loss. Theoretical analyses on the model's approximation upper bound, structure complexity, stability and generalization, are provided with rigorous mathematical proofs. The effectiveness and efficiency of GCN-RW are verified on semi-supervised node classification task with several benchmark datasets. Experimental results demonstrate that, in comparison with some state-of-the-art approaches, GCN-RW can achieve better or matched accuracies with less training time cost.

Balanced neighbor exploration for semi-supervised node classification on imbalanced graph data

Graph neural networks (GNNs) are powerful models for node classification tasks. When the nodes exhibit a skew distribution, the imbalanced graph data and the message-passing way of GNNs may significantly influence the decision boundary. Existing methods, generating additional nodes and edges or assigning weights of nodes to balance the training scenario, generally have a complicated implementation and a high time complexity. To this end, we propose Balanced Neighbor Exploration (BNE), an algorithm to improve training efficiency and classification performance in a fundamentally different way without generating nodes and computing weights. Intuitively, nodes highly influenced by nodes in one class should have a higher probability of belonging to the class. Inspired by this intuition, BNE incorporates more nodes highly influenced by minority nodes into the minority set to provide a balanced training scenario. The neighbor exploration involves sum and sort operations whose time complexities are O(N) and O(Nlog⁡N), respectively, with a fast operation process. Moreover, BNE conducts the message-passing and neighbor exploration processes simultaneously, with a straightforward implementation. Experiments on real-world imbalanced graph data demonstrate that BNE vastly outperforms the state-of-the-art methods for semi-supervised node classification on imbalanced graph data.