Node Classification Tasks Research Articles

Multimodal weighted graph representation for information extraction from visually rich documents

This paper introduces a novel system for information extraction from visually rich documents (VRD) using a weighted graph representation. The proposed method aims to improve the performance of the information extraction task by capturing the relationships between various VRD components. The VRD is modeled as a weighted graph, in which visual, textual, and spatial features of text regions are encoded in nodes and edges representing the relationships between neighboring text regions. The information extraction task from VRD is performed as a node classification task through the use of a graph convolutional networks, where the VRD graphs are fed into the network. The approach is evaluated across diverse documents, encompassing invoices and receipts, revealing achievement levels equal to or surpassing robust baselines.

Temporally-aware node embeddings for evolving networks topologies

Static node embedding algorithms applied to snapshots of real-world applications graphs are unable to capture their evolving process. As a result, the absence of information about the dynamics in these node representations can harm the accuracy and increase processing time of machine learning tasks related to these applications. Aiming at fill the gap regarding the inability of static methods to capture evolving processes on dynamic networks, we propose a biased random walk method named Evolving Node Embedding (EVNE). EVNE leverages the sequential relationship of graph snapshots by incorporating historic information when generating embeddings for the next snapshot. It learns node representations through a neural network, but differs from existing methods as it: (i) incorporates previously run walks at each step; (ii) starts the optimization of the current embedding from the parameters obtained in the previous iteration; and (iii) uses two time-varying parameters to regulate the behavior of the biased random walks over the process of graph exploration. Through a wide set of experiments we show that our approach generates better embeddings, outperforming baselines by up to 20% in a downstream node classification task. EVNE’s embeddings achieve better performance than others, based on experiments with four classifiers and five datasets. In addition, we present seven variations of our model to show the impact of each of EVNE’s mechanisms.

AN-GCN: An Anonymous Graph Convolutional Network Against Edge-Perturbing Attacks.

Recent studies have revealed the vulnerability of graph convolutional networks (GCNs) to edge-perturbing attacks, such as maliciously inserting or deleting graph edges. However, theoretical proof of such vulnerability remains a big challenge, and effective defense schemes are still open issues. In this article, we first generalize the formulation of edge-perturbing attacks and strictly prove the vulnerability of GCNs to such attacks in node classification tasks. Following this, an anonymous GCN, named AN-GCN, is proposed to defend against edge-perturbing attacks. In particular, we present a node localization theorem to demonstrate how GCNs locate nodes during their training phase. In addition, we design a staggered Gaussian noise-based node position generator and a spectral graph convolution-based discriminator (in detecting the generated node positions). Furthermore, we provide an optimization method for the designed generator and discriminator. It is demonstrated that the AN-GCN is secure against edge-perturbing attacks in node classification tasks, as AN-GCN is developed to classify nodes without the edge information (making it impossible for attackers to perturb edges anymore). Extensive evaluations verify the effectiveness of the general edge-perturbing attack (G-EPA) model in manipulating the classification results of the target nodes. More importantly, the proposed AN-GCN can achieve 82.7% in node classification accuracy without the edge-reading permission, which outperforms the state-of-the-art GCN.

Graph Convolutional Neural Networks With Diverse Negative Samples via Decomposed Determinant Point Processes.

Graph convolutional neural networks (GCNs) have achieved great success in graph representation learning by extracting high-level features from nodes and their topology. Since GCNs generally follow a message-passing mechanism, each node aggregates information from its first-order neighbor to update its representation. As a result, the representations of nodes with edges between them should be positively correlated and thus can be considered positive samples. However, there are more non-neighbor nodes in the whole graph, which provide diverse and useful information for the representation update. Two non-adjacent nodes usually have different representations, which can be seen as negative samples. Besides the node representations, the structural information of the graph is also crucial for learning. In this article, we used quality-diversity decomposition in determinant point processes (DPPs) to obtain diverse negative samples. When defining a distribution on diverse subsets of all non-neighboring nodes, we incorporate both graph structure information and node representations. Since the DPP sampling process requires matrix eigenvalue decomposition, we propose a new shortest-path-base method to improve computational efficiency. Finally, we incorporate the obtained negative samples into the graph convolution operation. The ideas are evaluated empirically in experiments on node classification tasks. These experiments show that the newly proposed methods not only improve the overall performance of standard representation learning but also significantly alleviate over-smoothing problems.

HDGCN: Dual-Channel Graph Convolutional Network With Higher-Order Information for Robust Feature Learning

Graph convolutional network (GCN) algorithms have been employed to learn graph embedding due to its inductive inference property, which is extended to GCN with higher-order information. However, the application of these higher-order information is confusing and not effectively distinguished. In addition, single channel GCN using higher-order information is weak for robust feature learning, and existing dual-channel GCNs rarely take higher-order information into account. To alleviate the above problems, we propose a dual-channel GCN with higher-order information for robust feature learning, denoted as HDGCN. Firstly, features of positive and negative higher-order graphs are extracted that fully exploits the self-contained attributes and higher-order geometric information. Meantime, the features of original graph structure are extracted by a conventional GCN that utilizes the self-contained feature attributes. Then, node features are represented as edge features by a feature fusion function. For the selection of negative samples, a fractional staggered negative sampling method is applied, by which the trainable graph model gains better topological features. Finally, the performance on seven real-world datasets demonstrates that HDGCN obtains the state-of-the-art performance on pairwise link prediction, higher-order structure prediction, and node classification tasks. By changing the attributes of multiple tasks, it can be proved that HDGCN has good robustness.

A Dual-channel Progressive Graph Convolutional Network via subgraph sampling

<p>Graph Convolutional Networks (GCNs) demonstrate an excellent performance in node classification tasks by updating node representation via aggregating information from the neighbor nodes. Note that the complex interactions among all the nodes can produce challenges for GCNs. Independent subgraph sampling effectively limits the neighbor aggregation in convolutional computations, and it has become a popular method to improve the efficiency of training GCNs. However, there are still some improvements in the existing subgraph sampling strategies: 1) a loss of the model performance caused by ignoring the connection information among different subgraphs; and 2) a lack of representation power caused by an incomplete topology. Therefore, we propose a novel model called Dual-channel Progressive Graph Convolutional Network (DPGCN) via sub-graph sampling. We construct subgraphs via clustering and maintain the connection information among the different subgraphs. To enhance the representation power, we construct a dual channel fusion module by using both the geometric information of the node feature and the original topology. Specifically, we evaluate the complementary information of the dual channels based on the joint entropy between the feature information and the adjacency matrix, and effectively reduce the information redundancy by reasonably selecting the feature information. Then, the model convergence is accelerated through parameter sharing and weight updating in progressive training. We select 4 real datasets and 8 characteristic models for comparison on the semi-supervised node classification task. The results verify that the DPGCN possesses superior classification accuracy and robustness. In addition, the proposed architecture performs excellently in the low labeling rate, which is of practical value to label scarcity problems in real cases.</p>

Hierarchically Contrastive Hard Sample Mining for Graph Self-Supervised Pretraining.

Contrastive learning has recently emerged as a powerful technique for graph self-supervised pretraining (GSP). By maximizing the mutual information (MI) between a positive sample pair, the network is forced to extract discriminative information from graphs to generate high-quality sample representations. However, we observe that, in the process of MI maximization (Infomax), the existing contrastive GSP algorithms suffer from at least one of the following problems: 1) treat all samples equally during optimization and 2) fall into a single contrasting pattern within the graph. Consequently, the vast number of well-categorized samples overwhelms the representation learning process, and limited information is accumulated, thus deteriorating the learning capability of the network. To solve these issues, in this article, by fusing the information from different views and conducting hard sample mining in a hierarchically contrastive manner, we propose a novel GSP algorithm called hierarchically contrastive hard sample mining (HCHSM). The hierarchical property of this algorithm is manifested in two aspects. First, according to the results of multilevel MI estimation in different views, the MI-based hard sample selection (MHSS) module keeps filtering the easy nodes and drives the network to focus more on hard nodes. Second, to collect more comprehensive information for hard sample learning, we introduce a hierarchically contrastive scheme to sequentially force the learned node representations to involve multilevel intrinsic graph features. In this way, as the contrastive granularity goes finer, the complementary information from different levels can be uniformly encoded to boost the discrimination of hard samples and enhance the quality of the learned graph embedding. Extensive experiments on seven benchmark datasets indicate that the HCHSM performs better than other competitors on node classification and node clustering tasks. The source code of HCHSM is available at https://github.com/WxTu/HCHSM.

Bayesian graph convolutional network with partial observations.

As a widely studied model in the machine learning and data processing society, graph convolutional network reveals its advantage in non-grid data processing. However, existing graph convolutional networks generally assume that the node features can be fully observed. This may violate the fact that many real applications come with only the pairwise relationships and the corresponding node features are unavailable. In this paper, a novel graph convolutional network model based on Bayesian framework is proposed to handle the graph node classification task without relying on node features. First, we equip the graph node with the pseudo-features generated from the stochastic process. Then, a hidden space structure preservation term is proposed and embedded into the generation process to maintain the independent and identically distributed property between the training and testing dataset. Although the model inference is challenging, we derive an efficient training and predication algorithm using variational inference. Experiments on different datasets demonstrate the proposed graph convolutional networks can significantly outperform traditional methods, achieving an average performance improvement of 9%.

Local structure-aware graph contrastive representation learning

Traditional Graph Neural Network (GNN), as a graph representation learning method, is constrained by label information. However, Graph Contrastive Learning (GCL) methods, which tackles the label problem effectively, mainly focus on the feature information of the global graph or small subgraph structure (e.g., the first-order neighborhood). In this paper, we propose a Local Structure-aware Graph Contrastive representation Learning method (LS-GCL) to model the structural information of nodes from multiple views. Specifically, we construct the semantic subgraphs that are not limited to the first-order neighbors. For the local view, the semantic subgraph of each target node is input into a shared GNN encoder to obtain the target node embeddings at the subgraph-level. Then, we use a pooling function to generate the subgraph-level graph embeddings. For the global view, considering the original graph preserves indispensable semantic information of nodes, we leverage the shared GNN encoder to learn the target node embeddings at the global graph-level. The proposed LS-GCL model is optimized to maximize the common information among similar instances at three various perspectives through a multi-level contrastive loss function. Experimental results on six datasets illustrate that our method outperforms state-of-the-art graph representation learning approaches for both node classification and link prediction tasks.

Node and edge dual-masked self-supervised graph representation

Self-supervised graph representation learning has been widely used in many intelligent applications since labeled information can hardly be found in these data environments. Currently, masking and reconstruction-based (MR-based) methods lead the state-of-the-art records in the self-supervised graph representation field. However, existing MR-based methods did not fully consider both the deep-level node and structure information which might decrease the final performance of the graph representation. To this end, this paper proposes a node and edge dual-masked self-supervised graph representation model to consider both node and structure information. First, a dual masking model is proposed to perform node masking and edge masking on the original graph at the same time to generate two masking graphs. Second, a graph encoder is designed to encode the two generated masking graphs. Then, two reconstruction decoders are designed to reconstruct the nodes and edges according to the masking graphs. At last, the reconstructed nodes and edges are compared with the original nodes and edges to calculate the loss values without using the labeled information. The proposed method is validated on a total of 14 datasets for graph node classification tasks and graph classification tasks. The experimental results show that the method is effective in self-supervised graph representation. The code is available at: https://github.com/TangPeng0627/Node-and-Edge-Dual-Mask.

ESA-GCN: An Enhanced Graph-Based Node Classification Method for Class Imbalance Using ENN-SMOTE Sampling and an Attention Mechanism

In recent years, graph neural networks (GNNs) have achieved great success in handling node classification tasks. However, as data explosively grows in various industries, the problem of class imbalance becomes increasingly severe. Traditional GNNs tend to prioritize majority class nodes when dealing with imbalanced class distributions, which fail to adequately capture the features of minority class nodes, leading to significant difficulties and challenges in data classification. To address issues such as inaccurate edge generation during graph data oversampling, insufficient representation of minority classes, and the presence of noisy samples, this paper proposes the ESA-GCN model. The advantages of this model are as follows: (i) it employs the ENN-SMOTE comprehensive sampling method to balance the dataset by reducing majority class nodes and increasing minority class nodes; (ii) the ENN algorithm reduces the classifier’s error rate and improves performance stability by removing low-quality and noisy data; (iii) an attention mechanism is introduced during the edge generation phase between new nodes and original nodes, fully considering the mutual relationships between nodes and concentrating on a subset of key information with high weights, thereby significantly improving classification accuracy while reducing model parameters and computational complexity. Experiments conducted on three public datasets (Cora, Citeseer, and PubMed) demonstrate that the ESA-GCN has achieved significant results in class imbalance node classification tasks.

TAMNR: a network embedding learning algorithm using text attention mechanism.

Because many existing algorithms are mainly trained based on the structural features of the networks, the results are more inclined to the structural commonality of the networks. These algorithms ignore the rich external information and node attributes (such as node text content, community and labels, etc.) that have important implications for network data analysis tasks. Existing network embedding algorithms considering text features usually regard the co-occurrence words in the node's text, or use an induced matrix completion algorithm to factorize the text feature matrix or the network structure feature matrix. Although this kind of algorithm can greatly improve the network embedding performance, they ignore the contribution rate of different co-occurrence words in the node's text. This article proposes a network embedding learning algorithm combining network structure and co-occurrence word features, also incorporating an attention mechanism to model the weight information of the co-occurrence words in the model. This mechanism filters out unimportant words and focuses on important words for learning and training tasks, fully considering the impact of the different co-occurrence words to the model. The proposed network representation algorithm is tested on three open datasets, and the experimental results demonstrate its strong advantages in node classification, visualization analysis, and case analysis tasks.

Perturbation-augmented Graph Convolutional Networks: A Graph Contrastive Learning architecture for effective node classification tasks

In the context of recent advances in Graph Convolutional Networks (GCNs) for semi-supervised learning, a significant highlight is the potential of Graph Contrastive Learning (GCL). Many established GCL-based GCNs focus on developing various augmentation strategies and learning architectures to efficiently generate and learn self-supervised information. Unfortunately, the node representation fusion is often neglected in the process of learning representation. To address the issue, we propose a novel end-to-end GCL framework called Perturbation-augmented Graph Convolutional Networks (PA-GCN), by simultaneously considering supervised information and self-supervised information of graphs to fuse node representations from the input graph and the augmentation graphs. Essentially, PA-GCN develops a perturbation augmentation strategy based on original graph relations, serving as a basis to generate diverse node information and self-supervised information. Additionally, PA-GCN introduces three main components: A encoder for node representation learning; an attention mechanism for effective node representation fusion; and a discriminator capable of learning self-supervised information from unlabeled nodes, thereby enabling joint learning of supervised information, node representation fusion, and self-supervised information. We conduct experiments on five real world datasets to evaluate the performance of PA-GCN in semi-supervised node classification tasks. The experimental results demonstrate the strong adaptability of PA-GCN for base encoders and the superiority of PA-GCN over the most advanced methods.

GA-GWNN: Generalized Adaptive Graph Wavelet Neural Network

Wavelet-based graph neural networks have received increasing attention in the node classification task. Existing graph wavelet-based approaches, however, are not applicable to arbitrary graphs as they use predefined wavelet filters with built-in homophilic assumptions and disregard heterophily. Recent studies attempted to address this issue through a wavelet lifting transform, which requires a bipartite graph, therefore altering the graph topology and leading to undesirable wavelet filters. This paper proposes a generalized adaptive graph wavelet network that preserves the graph topology through computational trees while implementing the lifting scheme on arbitrary graphs. Moreover, this locally defined lifting scheme integrates both high-pass and low-pass frequency components to further enhance feature representation. Finally, we benchmark our model using nine homophilic and heterophilic datasets, and the results demonstrate the effectiveness of our method.

Progressive Hard Negative Masking: From Global Uniformity to Local Tolerance

Unsupervised contrastive learning has recently become increasingly popular due to its amazing performance without the need for costly annotations. However, indiscriminate sampling of negative pairs is accompanied by the uniformity-tolerance dilemma, which is especially serious in node-level graph contrastive learning due to the smoothing property of graph convolutional operators. Previous negative mining strategies that either overly emphasize hard negatives or rely on precise distribution estimation can make minor improvements or even degrade the performance in such a case. In this paper, we investigate the role of hard negatives in the uniformity-tolerance dilemma and propose a novel contrastive objective with a progressive hard negative masking scheme. The proposed objective, as an asymptotically-tightened lower bound of mutual information, is theoretically and empirically demonstrated to be capable of allowing higher local tolerance and stronger contrastive effects, thus leading to higher-quality embedding distributions and considerable performance improvement in downstream node classification tasks.

Label Efficient Regularization and Propagation for Graph Node Classification.

An enhanced label propagation (LP) method called GraphHop was proposed recently. It outperforms graph convolutional networks (GCNs) in the semi-supervised node classification task on various networks. Although the performance of GraphHop was explained intuitively with joint node attribute and label signal smoothening, its rigorous mathematical treatment is lacking. In this paper, we propose a label efficient regularization and propagation (LERP) framework for graph node classification, and present an alternate optimization procedure for its solution. Furthermore, we show that GraphHop only offers an approximate solution to this framework and has two drawbacks. First, it includes all nodes in the classifier training without taking the reliability of pseudo-labeled nodes into account in the label update step. Second, it provides a rough approximation to the optimum of a subproblem in the label aggregation step. Based on the LERP framework, we propose a new method, named the LERP method, to solve these two shortcomings. LERP determines reliable pseudo-labels adaptively during the alternate optimization and provides a better approximation to the optimum with computational efficiency. Theoretical convergence of LERP is guaranteed. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of LERP. That is, LERP outperforms all benchmarking methods, including GraphHop, consistently on five common test datasets, two large-scale networks, and an object recognition task at extremely low label rates (i.e., 1, 2, 4, 8, 16, and 20 labeled samples per class).

Collaborative Decision-Reinforced Self-Supervision for Attributed Graph Clustering.

Attributed graph clustering aims to partition nodes of a graph structure into different groups. Recent works usually use variational graph autoencoder (VGAE) to make the node representations obey a specific distribution. Although they have shown promising results, how to introduce supervised information to guide the representation learning of graph nodes and improve clustering performance is still an open problem. In this article, we propose a Collaborative Decision-Reinforced Self-Supervision (CDRS) method to solve the problem, in which a pseudo node classification task collaborates with the clustering task to enhance the representation learning of graph nodes. First, a transformation module is used to enable end-to-end training of existing methods based on VGAE. Second, the pseudo node classification task is introduced into the network through multitask learning to make classification decisions for graph nodes. The graph nodes that have consistent decisions on clustering and pseudo node classification are added to a pseudo-label set, which can provide fruitful self-supervision for subsequent training. This pseudo-label set is gradually augmented during training, thus reinforcing the generalization capability of the network. Finally, we investigate different sorting strategies to further improve the quality of the pseudo-label set. Extensive experiments on multiple datasets show that the proposed method achieves outstanding performance compared with state-of-the-art methods. Our code is available at https://github.com/Jillian555/TNNLS_CDRS.

Graph Decoupling Attention Markov Networks for Semisupervised Graph Node Classification.

Graph neural networks (GNNs) have been ubiquitous in graph node classification tasks. Most GNN methods update the node embedding iteratively by aggregating its neighbors' information. However, they often suffer from negative disturbances, due to edges connecting nodes with different labels. One approach to alleviate this negative disturbance is to use attention to learn the weights of aggregation, but current attention-based GNNs only consider feature similarity and suffer from the lack of supervision. In this article, we consider label dependency of graph nodes and propose a decoupling attention mechanism to learn both hard and soft attention. The hard attention is learned on labels for a refined graph structure with fewer interclass edges so that the aggregation's negative disturbance can be reduced. The soft attention aims to learn the aggregation weights based on features over the refined graph structure to enhance information gains during message passing. Particularly, we formulate our model under the expectation-maximization (EM) framework, and the learned attention is used to guide label propagation in the M-step and feature propagation in the E-step, respectively. Extensive experiments are performed on six well-known benchmark graph datasets to verify the effectiveness of the proposed method.

SARW: Similarity-Aware Random Walk for GCN

Graph Convolutional Network (GCN) is an important method for learning graph representations of nodes. For large-scale graphs, the GCN could meet with the neighborhood expansion phenomenon, which makes the model complexity high and the training time long. An efficient solution is to adopt graph sampling techniques, such as node sampling and random walk sampling. However, the existing sampling methods still suffer from aggregating too many neighbor nodes and ignoring node feature information. Therefore, in this paper, we propose a new subgraph sampling method, namely, Similarity-Aware Random Walk (SARW), for GCN with large-scale graphs. A novel similarity index between two adjacent nodes is proposed, describing the relationship of nodes with their neighbors. Then, we design a sampling probability expression between adjacent nodes using node feature information, degree information, neighbor set information, etc. Moreover, we prove the unbiasedness of the SARW-based GCN model for node representations. The simplified version of SARW (SSARW) has a much smaller variance, which indicates the effectiveness of our subgraph sampling method in large-scale graphs for GCN learning. Experiments on six datasets show our method achieves superior performance over the state-of-the-art graph sampling approaches for the large-scale graph node classification task.

Centrality-based Relation aware Heterogeneous Graph Neural Network

The representation of heterogeneous graph nodes has become a hot research topic due to its diverse applications. However, extant approaches can only give consideration partly to three aspects: node structure, semantics and features. To better integrate these three aspects, a new semi-supervised graph neural network is proposed in this paper, called the CRHGNN (Centrality-based Relation aware Heterogeneous Graph Neural Network). The CRHGNN consists of four components as follows. The first component performs the encoding work and aims to capture the structure and semantics of the nodes. The second and third components perform attention mechanisms and information aggregation, respectively. The CRHGNN learns the mutual attention between nodes and carries out feature learning with less overfitting and fewer oversmoothing problems. The last component performs relation fusion, aiming to obtain a compact representation of the nodes. Experiments are conducted to evaluate the representation learning of nodes on three real-world heterogeneous graph datasets and demonstrate that the proposed model is very competitive in terms of node classification and node clustering tasks.