Optimization of Health Service Utilization Among Elderly People With Chronic Diseases in Rural Ethnic Minorities in Northwest Yunnan Using Graph Neural Networks.

Numerical reasoning machine reading comprehension (MRC) is a challenging natural language understanding task, which requires machines to have the numerical reasoning ability to add, subtract, sort and count over text to predict answers. Previous state-of-the-art models typically use graph neural networks (GNNs) to perform reasoning over heterogeneous graphs containing numbers and entities, to enhance their numerical reasoning ability. However, their constructed graphs contain only number nodes or only consider related entities in the same sentence, ignoring the relationship between entities. To alleviate the issue, in this paper, we propose a hierarchical graph attention network with heterogeneous tripartite graph (HGAT-HTG) for the numerical reasoning MRC task. It introduces additional sentence nodes as the intermediary between number nodes and entity nodes to construct the heterogeneous tripartite graph, then its hierarchical graph attention network (GAT) reasoning module first performs coarse-grained reasoning on the whole graph, then performs iterative fine-grained reasoning on the number-sentence subgraph and entity-sentence subgraph respectively, to enhance its numerical reasoning ability for the task. Experimental results indicate that HGAT-HTG model offers significant and consistent improvements, outperforming previous state-of-the-art QDGAT on DROP dataset.

The graph neural network (GNN) has demonstrated its superior power in various data mining tasks and has been widely applied in diversified fields. The core of GNN is the aggregation and combination functions, and mainstream GNN studies focus on the enhancement of these functions. However, GNNs face a common challenge, i.e., useless features contained in neighbor nodes may be integrated into the target node during the aggregation process. This leads to poor node embedding and undermines downstream tasks. To tackle this problem, this article proposes a novel GNN optimization framework GNN-MHSIC by introducing the nonparametric dependence method Hilbert-Schmidt independence criterion (HSIC) under the guidance of information bottleneck. HSIC is utilized to guide the information propagation among layers of a GNN from multiaspect views. GNN-MHSIC aims to achieve three main objectives: 1) minimizing the HSIC between the input features and the propagation layers; 2) maximizing the HSIC between the propagation layers and the ground truth; and 3) minimizing the HSIC between the propagation layers. With a multiaspect design, GNN-MHSIC can minimize the propagation of redundant information while preserving relevant information about the target node. We prove GNN-MHSIC's finite upper and lower bounds theoretically and evaluate it experimentally with four classic GNN models, including the graph convolutional network, the graph attention network (GAT), the heterogeneous GAT, and the heterogeneous graph (HG) propagation network on three widely used HGs. The results illustrate the usefulness and performance of GNN-MHSIC.

Graph neural networks have demonstrated its success in many applications on graph-structured data. Many efforts have been devoted to elaborating new network architectures and learning algorithms over the past decade. The exploration of applying ensemble learning techniques to enhance existing graph algorithms have been overlooked. In this work, we propose a simple generic bagging-based ensemble learning strategy which is applicable to any backbone graph models. We then propose two ensemble graph neural network models – Ensemble-GAT and Ensemble-HetGAT by applying the ensemble strategy to the graph attention network (GAT), and a heterogeneous graph attention network (HetGAT). We demonstrate the effectiveness of the proposed ensemble strategy on GAT and HetGAT through comprehensive experiments with four real-world homogeneous graph datasets and three real-world heterogeneous graph datasets on node classification tasks. The proposed Ensemble-GAT and Ensemble-HetGAT outperform the state-of-the-art graph neural network and heterogeneous graph neural network models on most of the benchmark datasets. The proposed ensemble strategy also alleviates the over-smoothing problem in GAT and HetGAT.

Graph neural networks are graph-based deep learning technologies that have attracted significant attention from researchers because of their powerful performance. Heterogeneous graph-based graph neural networks focus on the heterogeneity of the nodes and links in a graph. This is more effective at preserving semantic knowledge when representing data interactions in real-world graph structures. Unfortunately, most heterogeneous graph neural networks tend to transform heterogeneous graphs into homogeneous graphs when using meta-paths for representation learning. This paper therefore presents a novel motif-based hierarchical heterogeneous graph attention network algorithm, MBHAN, that addresses this problem by incorporating a hierarchical dual attention mechanism at the node-level and motif-level. Node-level attention aims to learn the importance between a node and its neighboring nodes within its corresponding motif. Motif-level attention is capable of learning the importance of different motifs in the heterogeneous graph. In view of the different vector space features of different types of nodes in heterogeneous graphs, MBHAN also aggregates the features of different types of nodes, so that they can jointly participate in downstream tasks after passing through segregated independent shallow neural networks. MBHAN’s superior network representation learning capability has been validated by extensive experiments on two real-world datasets.

In recommender systems, graph neural networks (GNN) can integrate interactions between users and items with their attributes, which makes GNN-based methods more powerful. However, directly stacking multiple layers in a graph neural network can easily lead to over-smoothing, hence recommendation systems based on graph neural networks typically underutilize higher-order neighborhoods in their learning. Although some heterogeneous graph random walk methods based on meta-paths can achieve higher-order aggregation, the focus is predominantly on the nodes at the ends of the paths. Moreover, these methods require manually defined meta-paths, which limits the model’s expressiveness and flexibility. Furthermore, path encoding in graph neural networks usually focuses only on the sequence leading to the target node. However, real-world interactions often do not follow this strict sequence, limiting the predictive performance of sequence-based network models. These problems prevent GNN-based methods from being fully effective. We propose a Graph Attention network with Information Propagation path aggregation for Social Recommendation (GAIPSRec). Firstly, we propose a universal heterogeneous graph sampling framework that does not require manually defining meta-paths for path sampling, thereby offering greater flexibility. Moreover, our method takes into account all nodes on the aggregation path and is capable of learning information from higher-order neighbors without succumbing to over-smoothing. Finally, our method utilizes a gate mechanism to fuse sequential and non-sequential dependence in encoding path instances, allowing a more holistic view of the data. Extensive experiments on real-world datasets show that our proposed GAIPSRec improves the performance significantly and outperforms state-of-the-art methods.

This article addresses the task of building personalized educational recommendations based on a heterogeneous knowledge graph that integrates data from university curricula, job vacancies, and online courses. To solve the problem of course recommendations by their relevance to a user’s competencies, a graph neural network (GNN)-based approach is proposed, specifically utilizing and comparing the Heterogeneous Graph Transformer (HGT) architecture, Graph Sample and Aggregate network (GraphSAGE), and Heterogeneous Graph Attention Network (HAN). Experiments were conducted on a heterogeneous graph comprising various node and relation types. The models were evaluated using regression and ranking metrics. The results demonstrated the superiority of the HGT-based recommendation model as a link regression task, especially in terms of ranking metrics, confirming its suitability for generating accurate and interpretable recommendations in educational systems. The proposed approach can be useful for developing adaptive learning recommendations aligned with users’ career goals.

The COVID-19 pandemic highlighted the urgent need for predictive models that can identify high-risk patients in hospitals to prevent severe complications. Traditional machine learning models rely on structured patient data, but recent advancements in Graph Neural Networks (GNNs) provide a novel approach to analyzing complex patient relationships. In this study, we propose a GNN-based framework to model patient interactions, symptoms, and underlying conditions using the COVID-19 Open Research Dataset (CORD-19). Our model constructs a heterogeneous patient-disease graph and applies Graph Convolutional Networks (GCN) and Graph Attention Networks (GAT) to predict the likelihood of severe complications. We perform extensive hyperparameter tuning to optimize performance, using evaluation metrics such as AUC-ROC, precision-recall, and SHAP for interpretability. Our results demonstrate that GNNs outperform traditional models in predicting high-risk patients, offering a scalable and effective solution for hospital decision-making. Key Words: COVID-19, Patient Risk Prediction, Graph Neural Networks, Healthcare Analytics, CORD-19, Machine Learning

Misinformation has become a frightening specter of society, especially fake news that concerning Covid-19. It massively spreads on the Internet, and then induces misunderstandings of information to the national and global communities during the pandemic. Detecting massive misinformation on the Internet is crucial and challenging because humans have struggled against this phenomenon for a long time. Our research concerns detecting fake news related to covid-19 using augmentation [random deletion (RD), random insertion (RI), random swap (RS), synonym replacement (SR)] and several graph neural network [graph convolutional network (GCN), graph attention network (GAT), and GraphSAGE (SAmple and aggreGatE)] model. We constructed nodes and edges in the graph, word-word node, and word-document node to graph neural network. Then, we tested those models in different amounts of sample training data to obtain accuracy for each model and compared them. For our fake news detection task, we found training accuracy steadily increasing for GCN, GAT, and SAGE models from the beginning to the end of the epochs. This result proved that the performance of GNN, whether GCN, GAT, or SAGE gained an entirely insignificant difference precision result.

Graph Neural Networks (GNNs) have shown superb performance in handling networked data, mainly attributed to their message passing and convolution process across neighbors. For most literature, the performance of GNNs is mainly reported based on noise-free data environments. No study has systematically evaluated GNNs’ performance under noise. In this article, we carry out an empirical study and theoretical analysis of four types of GNNs, including Graph Convolutional Networks (GCNs), Graph Attention Networks (GATs), Graph Contrastive Networks (GCL), and graph UniFilter under three types of noise, including attribute noise, structure noise, and label noise. Our study shows that GNNs behave tremendously differently in response to different types of noise. Overall, GAT is the most noise vulnerable and sensitive, whereas GCL is the most noise resilient. We further carry out theoretical analysis to explain the reason causing GAT to be sensitive to noise, and propose a solution to enhance its noise resilience. Our study brings in-depth firsthand knowledge of GNNs under noise for researchers and practitioners to better utilize GNNs in real-world applications.

Knowledge graph builds the bridge from massive data generated by the interaction and communication between various objects to intelligent applications and services in Internet of Things. The graph representation learning technology represented by graph neural networks plays an essential role in the understanding and reasoning of the knowledge graph with complicated internal structure. Although they are capable of assigning different attention weights to neighbors, the graph attention network (GAT) and its variants are inherently flawed and inadequate in modeling high-order knowledge graphs with high heterogeneity. Therefore, we propose a novel multirelational GAT framework in this article for knowledge reasoning over heterogeneous graphs by employing tensor and tensor operations. Specifically, we formulate the general high-order heterogeneous knowledge graph first. Then, the tensor GAT (TGAT), composed of three components: 1) heterogeneous information propagation; 2) multimodal semantic-aware attention; and 3) knowledge aggregation, is developed to simulate rich interactions between mixed triples, entities, and relationships when aggregating local information. What is more, we utilize the Tucker model to compress the parameters of TGAT and further reduce the storage and calculation consumption of the intermediate calculation process on the premise of maintaining the expressive power. We conduct extensive experiments to solve the link prediction task on four real-world heterogeneous graphs, and the results demonstrate that the TGAT model proposed in this article remarkably outperforms state-of-the-art competitors and improves the hits@1 accuracy by up to 7.6%.

Graphs can facilitate modeling of various complex systems and the analyses of the underlying relations within them, such as gene networks and power grids. Hence, learning over graphs has attracted increasing attention recently. Specifically, graph neural networks (GNNs) have been demonstrated to achieve state-of-the-art for various learning tasks over graphs. However, it is also shown that the use of graph structures in GNNs results in the amplification of algorithmic bias. Hence, fairness is an essential problem in GNNs. Motivated by this, this study proposes a novel fairness-aware graph attention network (GAT) design. Conventional GAT is one of the most popular and widely utilized GNN structure. Specifically, the present study first carries out a theoretical analysis in order to demonstrate the sources of algorithmic bias in GAT-based learning. Then, a novel fairness-aware GAT design is developed based the theoretical findings. Experimental results on real-world social networks show that the proposed fairness-aware GAT design can improve group fairness measures together with comparable classification accuracy to the conventional GAT for node classification.

Heterogeneous graphs are emerging as a prevalent form of data representation to capture complex structures and different relationships between a set of different types of objects in diverse disciplines. Node classification on heterogeneous graphs is a basic and critical task that remains unaddressed until the present day. Graph Neural Network is a powerful tool and has demonstrated remarkable performance in various tasks on graphs. However, most existing graph neural networks are based on the homophily assumption, which may be unsuitable for heterogeneous graphs. In this paper, we propose a graph neural network with type-feature attention mechanism to solve the problem of node classification on heterogeneous graphs. As a heterogeneous graph is composed of a group of edges between different types of nodes, it is reasonable to assume that each type of edge plays a different role in message propagation with different importance. An attention mechanism that considers the edge type and the features of the end nodes of the corresponding edge is built and incorporated into the process of message propagation of the graph neural network, by which the different heterogeneous information of nodes and edges is used jointly in solving the problem of node classification. We evaluate the proposed method on two public real-world heterogeneous graphs and the experimental results demonstrate the effectiveness of the proposed method.

Graph Neural Networks (GNNs) are deep learning methods which provide the current state of the art performance in node classification tasks. GNNs often assume homophily – neighboring nodes having similar features and labels–, and therefore may not be at their full potential when dealing with non-homophilic graphs. In this work, we focus on addressing this limitation and enable Graph Attention Networks (GAT), a commonly used variant of GNNs, to explore the structural information within each graph locality. Inspired by the positional encoding in the Transformers, we propose a framework, termed Graph Attentional Networks with Positional Embeddings (GAT-POS), to enhance GATs with positional embeddings which capture structural and positional information of the nodes in the graph. In this framework, the positional embeddings are learned by a model predictive of the graph context, plugged into an enhanced GAT architecture, which is able to leverage both the positional and content information of each node. The model is trained jointly to optimize for the task of node classification as well as the task of predicting graph context. Experimental results show that GAT-POS reaches remarkable improvement compared to strong GNN baselines and recent structural embedding enhanced GNNs on non-homophilic graphs.

Simple and deep graph attention networks

BackgroundThe recent development of high-throughput sequencing has created a large collection of multi-omics data, which enables researchers to better investigate cancer molecular profiles and cancer taxonomy based on molecular subtypes. Integrating multi-omics data has been proven to be effective for building more precise classification models. Most current multi-omics integrative models use either an early fusion in the form of concatenation or late fusion with a separate feature extractor for each omic, which are mainly based on deep neural networks. Due to the nature of biological systems, graphs are a better structural representation of bio-medical data. Although few graph neural network (GNN) based multi-omics integrative methods have been proposed, they suffer from three common disadvantages. One is most of them use only one type of connection, either inter-omics or intra-omic connection; second, they only consider one kind of GNN layer, either graph convolution network (GCN) or graph attention network (GAT); and third, most of these methods have not been tested on a more complex classification task, such as cancer molecular subtypes.ResultsIn this study, we propose a novel end-to-end multi-omics GNN framework for accurate and robust cancer subtype classification. The proposed model utilizes multi-omics data in the form of heterogeneous multi-layer graphs, which combine both inter-omics and intra-omic connections from established biological knowledge. The proposed model incorporates learned graph features and global genome features for accurate classification. We tested the proposed model on the Cancer Genome Atlas (TCGA) Pan-cancer dataset and TCGA breast invasive carcinoma (BRCA) dataset for molecular subtype and cancer subtype classification, respectively. The proposed model shows superior performance compared to four current state-of-the-art baseline models in terms of accuracy, F1 score, precision, and recall. The comparative analysis of GAT-based models and GCN-based models reveals that GAT-based models are preferred for smaller graphs with less information and GCN-based models are preferred for larger graphs with extra information.