Cosine similarity distance pruning algorithm Based on graph attention mechanism

We investigate works under the propagation-based fake news detection domain, which recently seeks to improve performance through the use of Graph Neural Networks (GNNs). Generally, existing works argue that using GNNs can give results superior to what was obtained using classic graph-based methods. We agree with this argument given that GNNs are capable of gaining superior performance by leveraging node features. But we argue that existing works haven’t identified the fact that the expressivity of GNNs is limited and bounded by node features. Existing works do not acknowledge that, by utilizing GNNs, they implicitly assume node features are strongly correlated to node labels. There are evidence that node features that have been employed do not necessarily correlate to node label. Instead of having a profound theoretical motivation, they have empirically observed that focusing on nodes features with strong feature-label correlation can increase predictive capability. This is a sub-optimal approach to view this problem, in fact, we argue that finding node features based on correlation is not practical or effective. Our first contribution is shifting readers from a node-level view i.e correlating node features with labels, to a graph-level view. In the graph-level view, we exploit the relationship between graph isomorphism and GNNs’ expressivity which can be utilized to well understand and interpret the relation between node features and GNNs’ expressivity. We conduct a wide range of experiments on basis of both node-level view and graph-level view and found graph-level view is more interpretable and strongly matches with results. Further, we gained insights on node features that wouldn’t be obtainable by a node-level view. In order to have a fair and comprehensive analysis of node features, we built a unified dataset that includes a wide range of node features. Our results indicate, as we improve model accuracy on basis of the graph level view, models’ generalizability decreases. We provide our hypothesis for this performance trade-off on the basis of the graph-level view. Our results and insights call for a much broader discussion on whether any sort of filtering method is effective. So, we conclude our work by providing readers with possible solutions that can be helpful to find harmony between node features and GNNs’ expressivity.

Graph neural networks (GNNs) have significant advantages in dealing with non-Euclidean data and have been widely used in various fields. However, most of the existing GNN models face two main challenges: (1) Most GNN models built upon the message-passing framework exhibit a shallow structure, which hampers their ability to efficiently transmit information between distant nodes. To address this, we aim to propose a novel message-passing framework, enabling the construction of GNN models with deep architectures akin to convolutional neural networks (CNNs), potentially comprising dozens or even hundreds of layers. (2) Existing models often approach the learning of edge and node features as separate tasks. To overcome this limitation, we aspire to develop a deep graph convolutional neural network learning framework capable of simultaneously acquiring edge embeddings and node embeddings. By utilizing the learned multi-dimensional edge feature matrix, we construct multi-channel filters to more effectively capture accurate node features. To address these challenges, we propose the Co-embedding of Edges and Nodes with Deep Graph Convolutional Neural Networks (CEN-DGCNN). In our approach, we propose a novel message-passing framework that can fully integrate and utilize both node features and multi-dimensional edge features. Based on this framework, we develop a deep graph convolutional neural network model that prevents over-smoothing and obtains node non-local structural features and refined high-order node features by extracting long-distance dependencies between nodes and utilizing multi-dimensional edge features. Moreover, we propose a novel graph convolutional layer that can learn node embeddings and multi-dimensional edge embeddings simultaneously. The layer updates multi-dimensional edge embeddings across layers based on node features and an attention mechanism, which enables efficient utilization and fusion of both node and edge features. Additionally, we propose a multi-dimensional edge feature encoding method based on directed edges, and use the resulting multi-dimensional edge feature matrix to construct a multi-channel filter to filter the node information. Lastly, extensive experiments show that CEN-DGCNN outperforms a large number of graph neural network baseline methods, demonstrating the effectiveness of our proposed method.

Graph Neural Networks (GNNs) have achieved tremendous success in various real-world applications due to their strong ability in graph representation learning. GNNs explore the graph structure and node features by aggregating and transforming information within node neighborhoods. However, through theoretical and empirical analysis, we reveal that the aggregation process of GNNs tends to destroy node similarity in the original feature space. There are many scenarios where node similarity plays a crucial role. Thus, it has motivated the proposed framework SimP-GCN that can effectively and efficiently preserve node similarity while exploiting graph structure. Specifically, to balance information from graph structure and node features, we propose a feature similarity preserving aggregation which adaptively integrates graph structure and node features. Furthermore, we employ self-supervised learning to explicitly capture the complex feature similarity and dissimilarity relations between nodes. We validate the effectiveness of SimP-GCN on seven benchmark datasets including three assortative and four disassorative graphs. The results demonstrate that SimP-GCN outperforms representative baselines. Further probe shows various advantages of the proposed framework. The implementation of SimP-GCN is available at \url{https://github.com/ChandlerBang/SimP-GCN}.

Graph Neural Networks (GNNs) have been a prevailing technique for tackling various analysis tasks on graph data. A key premise for the remarkable performance of GNNs relies on complete and trustworthy initial graph descriptions (i.e., node features and graph structure), which is often not satisfied since real-world graphs are often incomplete due to various unavoidable factors. In particular, GNNs face greater challenges when both node features and graph structure are incomplete at the same time. The existing methods either focus on feature completion or structure completion. They usually rely on the matching relationship between features and structure, or employ joint learning of node representation and feature (or structure) completion in the hope of achieving mutual benefit. However, recent studies confirm that the mutual interference between features and structure leads to the degradation of GNN performance. When both features and structure are incomplete, the mismatch between features and structure caused by the missing randomness exacerbates the interference between the two, which may trigger incorrect completions that negatively affect node representation. To this end, in this paper we propose a general GNN framework based on teacher-student distillation to improve the performance of GNNs on incomplete graphs, namely T2-GNN. To avoid the interference between features and structure, we separately design feature-level and structure-level teacher models to provide targeted guidance for student model (base GNNs, such as GCN) through distillation. Then we design two personalized methods to obtain well-trained feature and structure teachers. To ensure that the knowledge of the teacher model is comprehensively and effectively distilled to the student model, we further propose a dual distillation mode to enable the student to acquire as much expert knowledge as possible. Extensive experiments on eight benchmark datasets demonstrate the effectiveness and robustness of the new framework on graphs with incomplete features and structure.

Peptides have recently emerged as promising therapeutic agents against various diseases. For both research and safety regulation purposes, it is of high importance to develop computational methods to accurately predict the potential toxicity of peptides within the vast number of candidate peptides. In this study, we proposed ATSE, a peptide toxicity predictor by exploiting structural and evolutionary information based on graph neural networks and attention mechanism. More specifically, it consists of four modules: (i) a sequence processing module for converting peptide sequences to molecular graphs and evolutionary profiles, (ii) a feature extraction module designed to learn discriminative features from graph structural information and evolutionary information, (iii) an attention module employed to optimize the features and (iv) an output module determining a peptide as toxic or non-toxic, using optimized features from the attention module. Comparative studies demonstrate that the proposed ATSE significantly outperforms all other competing methods. We found that structural information is complementary to the evolutionary information, effectively improving the predictive performance. Importantly, the data-driven features learned by ATSE can be interpreted and visualized, providing additional information for further analysis. Moreover, we present a user-friendly online computational platform that implements the proposed ATSE, which is now available at http://server.malab.cn/ATSE. We expect that it can be a powerful and useful tool for researchers of interest.

Graph neural networks have been paid a lot attentions in recent years since many real-word data can naturally be represented by graph structures. Graph neural networks such as GCNs and GATs mainly focus on node features while ignoring edge features in graphs. However, in many graph structure data such as knowledge graphs, social networks, edge features are also important as they contain vital information about relations between nodes which are commonly ignored or simplified into binary or scalar values by existing methods. In this work we build a novel learning method on graphs called GNN-EE, i.e. GNN with Edge Enhanced, which takes both the node features and edge features into account while updating representations of graph components and can be applied to most of the common graph neural networks such as GCNs and GATs. Our GNN-EE method fits in the message-passing framework and thus is easy to generalize. In addition, we extend the random-walk-based algorithms on graphs so that they can consider both node and edge features on graphs. We use those random-walk-based algorithms as a pre-training method on graph with few initial features. We demonstrate the effectiveness and flexibility of our GNN-EE method through entity classification tasks and graph classification tasks.

At present, Graph Neural Network (GNN) methods usually follow the node centered message passing process and rely heavily on smooth node characteristics rather than graph structure. In view of this limitation, based on the heuristic method and graph attention mechanism, a feature fusion link prediction model (SAFL) combined with graph neural network is proposed. This model extracts the enclosing subgraphs around the target, combines the attention mechanism to assign neighbor weights to learn useful structural features, considers the impact of different nodes on the link, and fuses the graph neural network with the characteristics of input nodes to predict the link. The experiment on OGB dataset shows that the proposed link prediction model based on heuristic method enhances the graph structure characteristics, effectively represents the connectivity of enclosing subgraphs, and achieves better performance in link prediction.

Node classification using kernel propagation in graph neural networks

Graph Neural Networks (GNNs) have become increasingly popular for their ability to capture complex relationships within graphs by aggregating node neighbor information. However, in graphs exhibiting high levels of heterophily relevant distant nodes are missed during neighbor aggregation, thus limiting the GNN performance in tasks like node classification. To tackle the problem of incorporating long-range relevant neighbors into the GNN node aggregation mechanism, this paper introduces the Overlay Graph Neural Networks (OGN) model. OGN is inspired by P2P overlay networks, where the idea is to find neighbor peers (nodes) that, although not directly connected to a given node (a peer), are semantically similar and could favorably improve both query routing and query results. In our context, the network is the graph, and the routing is the message passing a GNN performs to aggregate node features. OGN networks are built by stacking one or more overlay layers, each taking as input the graph and a node feature matrix either available or derivable (e.g., by analyzing the graph’s structure). Each overlay layer combines base embeddings, learned by considering node features and short-range node neighbors, with overlay embeddings computed by projecting nodes with similar features close in an overlay space and then aggregating (overlay) neighbor nodes via a sliding window attention mechanism. Base and overlay embeddings are combined to capture nodes’ immediate and global context in a graph. We evaluate OGN in a node classification task using state-of-the-art benchmarks and show that OGN is competitive with the advantage of being easily portable to any existing GNN model.

Graph neural networks (GNNs) witness impressive performances on homophilic graphs characterized by a higher number of edges connecting nodes of similar class labels. A decline in the performance of GNNs can be experienced when applied to heterophilic graphs where most of the edges connect nodes with different class labels. This study presents a novel and versatile preprocessing framework comprising three fundamental stages. This framework can be seamlessly integrated with various GNN architectures to address heterophily within graphs effectively. In the initial stage, we predict class probabilities for nodes through a dense network. It is widely acknowledged that conventional feature-based similarity measures, such as cosine similarity, might not always accurately capture the correspondence between node pairs. Moving to the second stage, we introduce a reweighting strategy guided by class embeddings generated from autoencoders to counter this limitation. In the final stage, we utilize the reweighted similarity coefficients in a two-stage graph rewiring process. This process involves node deletion and subsequent insertion to generate a more homophily-oriented neighborhood. We reuse class embeddings by fusing them with the original node features to enrich the node features with class-level information. The updated node features and the rewired graph structure are ultimately fed into the GNN model. This facilitates effective message passing (MP) across neighborhoods. We extensively evaluate our approach on various standard graph datasets encompassing homophilic and heterophilic characteristics. Across these datasets, our framework consistently improves the performance of the established baseline methods.

Graph Neural Networks (GNNs) play a pivotal role in learning representations of brain networks for estimating brain age. However, the over-squashing impedes interactions between long-range nodes, hindering the ability of message-passing mechanism-based GNNs to learn the topological structure of brain networks. Graph rewiring methods and curvature GNNs have been proposed to alleviate over-squashing. However, most graph rewiring methods overlook node features and curvature GNNs neglect the geometric properties of signed curvature. In this study, a Signed Curvature GNN (SCGNN) was proposed to rewire the graph based on node features and curvature, and learn the representation of signed curvature. First, a Mutual Information Ollivier-Ricci Flow (MORF) was proposed to add connections in the neighborhood of edge with the minimal negative curvature based on the maximum mutual information between node features, improving the efficiency of information interaction between nodes. Then, a Signed Curvature Convolution (SCC) was proposed to aggregate node features based on positive and negative curvature, facilitating the model's ability to capture the complex topological structures of brain networks. Additionally, an Ollivier-Ricci Gradient Pooling (ORG-Pooling) was proposed to select the key nodes and topology structures by curvature gradient and attention mechanism, accurately obtaining the global representation for brain age estimation. Experiments conducted on six public datasets with structural magnetic resonance imaging (sMRI), spanning ages from 18 to 91 years, validate that our method achieves promising performance compared with existing methods. Furthermore, we employed the gaps between brain age and chronological age for identifying Alzheimer's Disease (AD), yielding the best classification performance.

Graph Neural Networks (GNNs), which generalize the deep neural networks to graph-structured data, have achieved great success in modeling graphs. However, as an extension of deep learning for graphs, GNNs lack explainability, which largely limits their adoption in scenarios that demand the transparency of models. Though many efforts are taken to improve the explainability of deep learning, they mainly focus on i.i.d data, which cannot be directly applied to explain the predictions of GNNs because GNNs utilize both node features and graph topology to make predictions. There are only very few work on the explainability of GNNs and they focus on post-hoc explanations. Since post-hoc explanations are not directly obtained from the GNNs, they can be biased and misrepresent the true explanations. Therefore, in this paper, we study a novel problem of self-explainable GNNs which can simultaneously give predictions and explanations. We propose a new framework which can find K-nearest labeled nodes for each unlabeled node to give explainable node classification, where nearest labeled nodes are found by interpretable similarity module in terms of both node similarity and local structure similarity. Extensive experiments on real-world and synthetic datasets demonstrate the effectiveness of the proposed framework for explainable node classification.

NE-GConv: A lightweight node edge graph convolutional network for intrusion detection

SGNN-T: Space graph neural network coupled transformer for molecular property prediction

Graph Neural Networks (GNNs) have been widely applied to link prediction tasks. GNN models generally follow a message passing scheme to recursively aggregate the attribute features of neighbor nodes. In this scheme, the GNN does not explicitly consider the structural information of the graph that is critical for link prediction. This inspires researchers to encode this information to consider the location of nodes and their roles. However, current studies mostly adopt a single encoding method, which does not sufficiently consider the structural information. In additional, the structural and attribute features are not effectively integrated. In this paper, we propose a novel framework named Hybrid Structure Encoding Graph neural networks with Attention mechanism (HSEGA) for link prediction. HSEGA uses PageRank, betweenness centrality, and node labeling for hybrid encoding of structural information to capture the importance, centrality, and location of graph nodes. Subsequently, the structural and attribute features are integrated as deep GNN inputs to learn from two domains. Finally, we use the attention mechanism to adaptively incorporate information. Extensive experiments on diverse benchmark datasets show that HSEGA consistently achieves state-of-the-art link prediction performance.