Graph Neural Networks Research Articles

Advanced graph neural network-based surrogate model for granular flows in arbitrarily shaped domains

In next-generation manufacturing, the introduction of a surrogate model is crucial for constructing a digital twin. This trend is no exception in powder industry. Various mesh-free approaches such as the discrete element method (DEM) are versatile tools for facilitating simulation-based digital twins of powder industries. Unfortunately, the DEM-based high-fidelity simulation of actual industrial processes poses a significant challenge because of the prohibitive computational costs. In this context, the surrogate model has emerged as a promising technique to solve this problem. However, surrogate modeling of powder processes within arbitrarily shaped boundaries is substantially impossible. To resolve this essential problem in the existing surrogate models, we propose a novel surrogate model, namely, a signed distance function-based graph neural network (SGN), to accelerate the simulations of granular flows. In the SGN, the wall boundary is efficiently modeled by combining the graph structure with a signed distance function. Validation studies were conducted in typical powder systems, such as a sandglass, and a rotating-paddle mixer. A systematic comparison of the simulation results between the high-fidelity and SGN simulations shows that the proposed SGN accurately simulates granular dynamics, with a remarkable increase in computational efficiency. This study significantly contributes to the next-generation manufacturing in the powder industry.

Physics-Enhanced Graph Neural Networks for Soft Sensing in Industrial Internet of Things

Physics-Enhanced Graph Neural Networks for Soft Sensing in Industrial Internet of Things

MSSTGNN: Multi-Scaled Spatio-Temporal Graph Neural Networks for short- and long-term traffic prediction

Accurate traffic prediction plays a crucial role in ensuring traffic safety and minimizing property damage. The utilization of STGNNs in traffic prediction has gained significant attention from researchers aiming to capture the intricate time-varying relationships within traffic data. While existing STGNNs commonly rely on Euclidean distance to assess the similarity between nodes, which may fall short in reflecting POI or regional functions. The traffic network exhibits static from a macro perspective, whereas undergoes dynamic changes in the micro perspective. Previous researchers incorporating self-attention to capture time-varying features for constructing dynamic graphs have faced challenges in overlooking the connections between nodes due to the Softmax polarization effect, which tends to amplify extreme value differences, and fails to accurately represent the true relationships between nodes. To solve this problem, we introduce the Multi-Scaled Spatio-Temporal Graph Neural Networks (MSSTGNN), which aims to comprehensively capture characteristics within traffic from multiscale viewpoints to construct multi-perspective graphs. We employ a trainable matrix to enhance the predefined adjacency matrices and to construct an optimal dynamic graph based on both trend and period. Additionally, a graph aggregate technique is proposed to effectively merge trend and periodic dynamic graphs. TCN is developed to model nonstationary traffic data, and we leverage the skip and residual connections to increase the model depth. A two-stage learning approach and a novel MSELoss function is designed to enhance the model's performance. The experimental results demonstrate that the MSSTGNN model outperforms the existing methods, achieving state-of-the-art performances across multiple real-world datasets.

Wireless Network Security Prediction Using Machine Learning and Graph Neural Networks

Abstract: We study the application of graph neural networks and machine learning in the prediction of risks to wireless networks. Advanced predictive methods are necessary, since mere security measures cannot thwart the rising complexity of cyber-attacks. In the present work, we will use a hybrid model that combines several machine learning methods to reduce false negatives and positives and improve accuracy in the prediction of risks to wireless networks. Models are trained and validated with large amounts of data that include the performance indicators for accuracy, precision, recall, F1-score, and AUC-ROC. From this, it is also seen that the hybrid model performs much better than the standard models in real-time threat detection. The impact of the size of the dataset on the performance of the model was also studied, and it has come out that larger datasets improve predictive powers significantly. The result demonstrated that the latest advancements in machine learning techniques can lead to important improvements in the security of wireless networks.

Key Frame Selection for Temporal Graph Optimization of Skeleton-Based Action Recognition

Graph neural networks (GNNs) are extensively utilized to capture the spatial–temporal relationships among human body parts for skeleton-based action recognition. However, due to the inefficient information propagation caused by redundant sampling of video frames in the temporal domain, we focus on refining temporal graphs through key frame selection. To this end, we propose a multi-stage key frame selection (MSKFS) method, aiming to find the most representative frames as the graph nodes to learn compact temporal graph representations of human skeletons for action recognition. The MSKFS progressively selects key frames in two stages: (1) salient posture frame selection based on the global dynamics of body parts and (2) key frame refinement and alignment according to intra-frame correlations. The first stage captures the most salient information and aligns the corresponding information of skeleton sequences within the same category. The second stage enriches the subtle information for the integrity of the information derived by the salient frames. Moreover, variational inference is applied to differentiate the key frame refinement and alignment procedure, allowing the end-to-end optimization of arbitrary graph-based models to represent the obtained compact graph for skeleton-based action recognition. Our MSKFS method achieves state-of-the-art performances on two challenging action recognition datasets.

Anomaly Detection System Based on Graph Neural Network and Node Embedding for Cybersecurity

Anomaly Detection System Based on Graph Neural Network and Node Embedding for Cybersecurity

GVNNVAE: A Novel Microbe-Drug Association Prediction Model based on an Improved Graph Neural Network and the Variational Auto-Encoder

Abstract: Microorganisms play a crucial role in human health and disease. Identifying potential microbe- drug associations is essential for drug discovery and clinical treatment. In this manuscript, we proposed a novel prediction model named GVNNVAE by combining an Improved Graph Neural Network (GNN) and the Variational Auto-Encoder (VAE) to infer potential microbe-drug associations. In GVNNVAE, we first established a heterogeneous microbe-drug network N by integrating multiple similarity metrics of microbes, drugs, and diseases. Subsequently, we introduced an improved GNN and the VAE to extract topological and attribute representations for nodes in N respectively. Finally, through incorporating various original attributes of microbes and drugs with above two kinds of newly obtained topological and attribute representations, predicted scores of potential microbe-drug associations would be calculated. Furthermore, To evaluate the prediction performance of GVNNVAE, intensive experiments were done and comparative results showed that GVNNVAE could achieve a satisfactory AUC value of 0.9688, which outperformed existing competitive state-of-the-art methods. And moreover, case studies of known microbes and drugs confirmed the effectiveness of GVNNVAE as well, which highlighted its potential for predicting latent microbe-drug associations.

Multi-branch fusion graph neural network based on multi-head attention for childhood seizure detection

The most common manifestation of neurological disorders in children is the occurrence of epileptic seizures. In this study, we propose a multi-branch graph convolutional network (MGCNA) framework with a multi-head attention mechanism for detecting seizures in children. The MGCNA framework extracts effective and reliable features from high-dimensional data, particularly by exploring the relationships between EEG features and electrodes and considering the spatial and temporal dependencies in epileptic brains. This method incorporates three graph learning approaches to systematically assess the connectivity and synchronization of multi-channel EEG signals. The multi-branch graph convolutional network is employed to dynamically learn temporal correlations and spatial topological structures. Utilizing the multi-head attention mechanism to process multi-branch graph features further enhances the capability to handle local features. Experimental results demonstrate that the MGCNA exhibits superior performance on patient-specific and patient-independent experiments. Our end-to-end model for automatic detection of epileptic seizures could be employed to assist in clinical decision-making.

IGA-Graph-Net: Isogeometric analysis-reuse method based on graph neural networks for topology-consistent models

This paper introduces a novel isogeometric analysis-reuse framework called IGA-Graph-Net, which combines Graph Neural Networks with Isogeometric Analysis to overcome the limitations of Convolutional Neural Networks when dealing with B-spline data. Our network architecture incorporates ResNetV2 and PointTransformer for enhanced performance. We transformed the dataset creation process from using Convolutional Neural Networks to Graph Neural Networks. Additionally, we proposed a new loss function tailored for Dirichlet boundary conditions and enriched the input features. Several examples are presented to demonstrate the effectiveness of the proposed framework. In terms of accuracy when tested on the same set of partial differential equation data, our framework demonstrates significant improvements compared to the reuse method based on Convolutional Neural Networks for Isogeometric Analysis on topology-consistent geometries with complex boundaries.

Fault-Tolerant Scheduling Mechanism for Dynamic Edge Computing Scenarios Based on Graph Reinforcement Learning

With the proliferation of Internet of Things (IoT) devices and edge nodes, edge computing has taken on much of the real-time data processing and low-latency response tasks which were previously managed by cloud computing. However, edge computing often encounters challenges such as network instability and dynamic resource variations, which can lead to task interruptions or failures. To address these issues, developing a fault-tolerant scheduling mechanism is crucial to ensure that a system continues to operate efficiently even when some nodes experience failures. In this paper, we propose an innovative fault-tolerant scheduling model based on asynchronous graph reinforcement learning. This model incorporates a deep reinforcement learning framework built upon a graph neural network, allowing it to accurately capture the complex communication relationships between computing nodes. The model generates fault-tolerant scheduling actions as output, ensuring robust performance in dynamic environments. Additionally, we introduce an asynchronous model update strategy, which enhances the model’s capability of real-time dynamic scheduling through multi-threaded parallel interactions with the environment and frequent model updates via running threads. The experimental results demonstrate that the proposed method outperformed the baseline algorithms in terms of quality of service (QoS) assurance and fault-tolerant scheduling capabilities.

Urban Land Use Classification Model Fusing Multimodal Deep Features

Urban land use classification plays a significant role in urban studies and provides key guidance for urban development. However, existing methods predominantly rely on either raster structure deep features through convolutional neural networks (CNNs) or topological structure deep features through graph neural networks (GNNs), making it challenging to comprehensively capture the rich semantic information in remote sensing images. To address this limitation, we propose a novel urban land use classification model by integrating both raster and topological structure deep features to enhance the accuracy and robustness of the classification model. First, we divide the urban area into block units based on road network data and further subdivide these units using the fractal network evolution algorithm (FNEA). Next, the K-nearest neighbors (KNN) graph construction method with adaptive fusion coefficients is employed to generate both global and local graphs of the blocks and sub-units. The spectral features and subgraph features are then constructed, and a graph convolutional network (GCN) is utilized to extract the node relational features from both the global and local graphs, forming the topological structure deep features while aggregating local features into global ones. Subsequently, VGG-16 (Visual Geometry Group 16) is used to extract the image convolutional features of the block units, obtaining the raster structure deep features. Finally, the transformer is used to fuse both topological and raster structure deep features, and land use classification is completed using the softmax function. Experiments were conducted using high-resolution Google images and Open Street Map (OSM) data, with study areas on the third ring road of Shenyang and the fourth ring road of Chengdu. The results demonstrate that the proposed method improves the overall accuracy and Kappa coefficient by 9.32% and 0.17, respectively, compared to single deep learning models. Incorporating subgraph structure features further enhances the overall accuracy and Kappa by 1.13% and 0.1. The adaptive KNN graph construction method achieves accuracy comparable to that of the empirical threshold method. This study enables accurate large-scale urban land use classification with reduced manual intervention, improving urban planning efficiency. The experimental results verify the effectiveness of the proposed method, particularly in terms of classification accuracy and feature representation completeness.

A graph convolutional neural network model based on fused multi-subgraph as input and fused feature information as output

The graph convolution neural network (GCN)-based node classification model tackles the challenge of classifying nodes in graph data through learned feature representations. However, most existing graph neural networks primarily focus on the same type of edges, which might not accurately reflect the intricate real-world graph structure. This paper introduces a novel graph neural network model, MF-GCN, which integrates subgraphs with various edge types as input and combines feature information from each graph convolutional neural network layer to produce the final output. This model learns node feature representations by separately feeding subgraphs with different edge types into the graph convolutional layer. It then computes the weight vectors for fusing various edge type subgraphs based on the learned node features. Additionally, to efficiently extract feature information, the outputs of each graph convolution layer, without an activation function, are weighted and summed to obtain the final node features. This approach resolves the challenges of determining fusion weights and effectively extracting feature information during subgraph fusion. Experimental results show that the proposed model significantly improves performance on all three datasets, highlighting its effectiveness in node representation learning tasks.

An overview of graph neural networks for molecular biology: Challenges and solutions

This review explores the application of graph neural networks (GNNs) in molecular biology, with a particular focus on their role in drug discovery. Various use cases, including molecule classification for cardiotoxicity detection and the prediction of molecular properties have been examined. A comprehensive analysis compares methodologies, problem statements, datasets, and findings across different studies. A core themes of review is the intuition to regularize a graph neural network which was proven to be reliable in terms of robustness for drug discovery purposes using noise nodes techniques. In the conclusion section, key insights from the reviewed literature and promising future directions for research have been presented. This work aims to provide a foundational understanding and guide future innovations in leveraging GNNs for molecular applications.

Intelligent Nanomaterial Image Characterizations – A Comprehensive Review on AI Techniques that Power the Present and Drive the Future of Nanoscience

AbstractArtificial Intelligence (AI) is pivotal in advancing science, including nanomaterial studies. This review explores AI‐based image processing in nanoscience, focusing on algorithms to enhance characterization results from instruments like scanning electron microscopy, transmission electron microscopy, X‐ray diffraction, atomic force microscopy etc. It addresses the significance of AI in nanoscience, challenges in advancing AI‐based image processing for nano material characterization, and AI's role in structural analysis, property prediction, deriving structure‐property relations, dataset augmentation, and improving model robustness. Key AI techniques such as Graph Neural Networks, adversarial training, transfer learning, generative models, attention mechanisms, and federated learning are highlighted for their contributions to nano science studies. The review concludes by outlining persisting challenges and thrust areas for future research, aiming to propel nanoscience with AI. This comprehensive analysis underscores the importance of AI‐powered image processing in nanomaterial characterization, offering valuable insights for researchers.

Attribute-Aware Graph Convolutional Network Recommendation Method

In recent years, recommendation systems have made significant strides through the application of graph neural networks (GNNs). However, most of the existing methods primarily focus on modeling user–item interactions, often failing to account for the distinct roles of different types of neighboring nodes during the graph convolution process. Moreover, the intricate relationships between user and item attributes are frequently underexplored, which constrains further improvement in the performance of recommendation models. To overcome these challenges, this paper proposes an attribute-aware graph convolutional network recommendation model (AAGCNR). This model accounts for the complex interrelationships between user and item attributes while distinguishing between different types of neighboring nodes during graph convolution. First, a multi-head self-attention mechanism is applied to capture the semantic relationships among attributes across various semantic spaces. Additionally, a bilinear interaction module is employed to facilitate interactions between attributes. Since different neighboring nodes exert different influences on target nodes, the model performs convolutional aggregation by leveraging the interrelationships of these attributes throughout the graph convolution process. Experiments conducted on real-world datasets reveal that AAGCNR outperforms other benchmark algorithms, particularly in terms of RECALL and NDCG metrics.

GTransCYPs: an improved graph transformer neural network with attention pooling for reliably predicting CYP450 inhibitors

State‑of‑the‑art medical studies proved that predicting CYP450 enzyme inhibitors is beneficial in the early stage of drug discovery. However, accurate machine learning-based (ML) in silico methods for predicting CYP450 inhibitors remains challenging. Here, we introduce GTransCYPs, an improved graph neural network (GNN) with a transformer mechanism for predicting CYP450 inhibitors. This model significantly enhances the discrimination between inhibitors and non-inhibitors for five major CYP450 isozymes: 1A2, 2C9, 2C19, 2D6, and 3A4. GTransCYPs learns information patterns from molecular graphs by aggregating node and edge representations using a transformer. The GTransCYPs model utilizes transformer convolution layers to process features, followed by a global attention-pooling technique to synthesize the graph-level information. This information is then fed through successive linear layers for final output generation. Experimental results demonstrate that the GTransCYPs model achieved high performance, outperforming other state-of-the-art methods in CYP450 prediction.Scientific contributionThe prediction of CYP450 inhibition via computational techniques utilizing biological information has emerged as a cost-effective and highly efficient approach. Here, we presented a deep learning (DL) architecture based on GNN with transformer mechanism and attention pooling (GTransCYPs) to predict CYP450 inhibitors. Four GTransCYPs of different pooling technique were tested on an experimental tasks on the CYP450 prediction problem for the first time. Graph transformer with attention pooling algorithm achieved the best performances. Comparative and ablation experiments provide evidence of the efficacy of our proposed method in predicting CYP450 inhibitors. The source code is publicly available at https://github.com/zonwoo/GTransCYPs.

Efficient Session-based Recommendation with Contrastive Graph-based Shortest Path Search

Session-based recommendation aims to predict the next item based on a set of anonymous sessions. Capturing user intent from a short interaction sequence imposes a variety of challenges since no user profiles are available and interaction data is naturally sparse. Recent approaches relying on graph neural networks (GNNs) for session-based recommendation use global item relations to explore collaborative information from different sessions. These methods capture the topological structure of the graph and rely on multi-hop information aggregation in GNNs to exchange information along edges. Consequently, graph-based models suffer from noisy item relations in the training data and introduce high complexity for large item catalogs. We propose to explicitly model the multi-hop information aggregation mechanism over multiple layers via shortest-path edges based on knowledge from the sequential recommendation domain. Our approach does not require multiple layers to exchange information and ignores unreliable item-item relations. Furthermore, to address inherent data sparsity, we are the first to apply supervised contrastive learning by mining data-driven positive and hard negative item samples from the training data. Extensive experiments on four different datasets show that the proposed approach outperforms almost all of the state-of-the-art methods.

Two-level integrated verification evaluation method based on comprehensive weighted assessment in multiple scenarios

The complexity and diversity brought by the distributed architecture of the new generation electric information collection system have deepened the difficulty of constructing evaluation verification index systems and quality evaluation models. Moreover, the presence of differentiated components has made fair and scientific verification challenging. Therefore, leveraging graph neural networks and siamese networks, a integrated construction quality evaluation system based on comprehensive weighted assessment in multiple scenarios was developed. Firstly, a graph neural network was constructed based on terminal data of the branches electricity usage information collection system and the link topology structure. Subsequently, this network was deployed on the headquarters side to directly acquire terminal data and generate mirror network input from the branches data, enabling real-time acquisition of various system operational indicators. Finally, the similarity between the headquarters and branches data was calculated using siamese networks to compute accuracy compensation weights for checking and evaluating various indicators, thereby obtaining comprehensive weighted quality evaluation indicators of the branches new generation electric information collection system. We use three types of services including electricity data collection, load forecasting, and task scheduling as experimental scenarios. The results showed that the multidimensional comprehensive weighted quality assessment combined with accuracy compensation obtained from the siamese network resulted in business construction quality assessment values of 97.92%, 95.95%, and 99.96% in branch. This value is approximately equal to the quality evaluation value of manual work, so the method can effectively verify the construction quality of new systems in the branch.

Academic collaboration recommendation based on graph neural network and multi-attribute embedding

Academic collaboration recommendation (ACR) can help researchers find potential partners for research and thus promote academic innovation. Recent works mostly use graph learning-based methods to explore various ways of combining node information with topology, which consists of multiple steps, including network construction, node feature extraction, network representation learning and link prediction. One limitation is that they only conduct research on co-authorship networks and ignore citation relationship between publications. Besides, they tend to use attributes from a single dimension of researchers and do not take attributes of researchers from multiple dimensions into consideration at the same time. To address the above issues, we present the multi-dimensional attributes enhanced heterogeneous (MAH) network representation learning method, which constructs heterogeneous networks with both co-authorship and citation information and makes use of multi-dimensional attributes. Three research questions are addressed in this work: (RQ1) Is our proposed method effective on academic collaboration recommendation compared with existing state-of-the-art methods? (RQ2) Does incorporating citation information into co-authorship network help improve the performance of academic collaboration recommendation? (RQ3) How does fusing multi-dimensional attributes affect the performance of academic collaboration recommendation? A publicly available real-world data set is used in our experiments. The superior performance of MAH compared with baseline methods demonstrates that the proposed multi-dimensional feature-based researcher profile can enrich node information in academic network and effective researcher representations can be learned by applying graph representation learning methods on the network.

Disentangled Dynamic Graph Attention Network for Out-of-distribution Sequential Recommendation

Sequential recommendation, leveraging user-item interaction histories to provide personalized and timely suggestions, has drawn significant research interest recently. With the power of exploiting spatio-temporal dynamics, dynamic graph neural networks (DyGNNs) show great potential in sequential recommendation by modeling the dynamic relationship between users and items. However, spatio-temporal distribution shifts naturally exist in out-of-distribution sequential recommendation, where both user-item relationships and temporal sequences demonstrate pattern shifts. The out-of-distribution scenarios may lead to the failure of existing DyGNNs in handling spatio-temporal distribution shifts in sequential recommendation, given that the patterns they exploit tend to be variant w.r.t labels under distribution shifts. In this paper, we propose D isentangled I ntervention-based D ynamic graph A ttention networks with I nvariance Promotion ( I-DIDA ) to handle spatio-temporal distribution shifts in sequential recommendation by discovering and utilizing invariant patterns, ie , structures and features whose predictive abilities are stable across distribution shifts. Specifically, we first propose a disentangled spatio-temporal attention network to capture the variant and invariant patterns. By utilizing the disentangled patterns, we design a spatio-temporal intervention mechanism to create multiple interventional distributions and an environment inference module to infer the latent spatio-temporal environments, and minimize the invariance loss to leverage the invariant patterns with stable predictive abilities under distribution shifts. Extensive experiments demonstrate the superiority of our method over state-of-the-art sequential recommendation baselines under distribution shifts.