Graph Neural Network Architecture Research Articles

Predicting Stellar Masses of the First Galaxies Using Graph Neural Networks

Theoretical models of galaxy formation and evolution are primarily investigated through cosmological simulations and semi-analytical models. The former method consumes O(106) core-hours explicitly modeling the dynamics of the galaxies, whereas the latter method only requires O(103) core-hours foregoing directly simulating internal structure for computational efficiency. In this work, we present a proof-of-concept machine learning regression model, using a graph neural network architecture, to predict the stellar mass of high-redshift galaxies solely from their dark matter merger trees, trained from a radiation hydrodynamics cosmological simulation of the first galaxies.

Railway network delay evolution: A heterogeneous graph neural network approach

Accurate delay evolution prediction plays a pivotal role in train rescheduling decision-making for the railway network. Existing studies on delay prediction predominantly centered around predicting delays for each train in the subsequent stations (i.e., following a train-oriented perspective). Furthermore, train operations in the railway network involve different types of entities (stations, trains, etc.), making the current graph/network models with homogenous nodes (i.e., the same kind of nodes) incapable of effectively capturing the interactions between the entities. This paper develops a network-oriented model to investigate the train delay evolution on railway networks, by predicting the delays of running trains in the network after a given time interval. The proposed model combines the GraphSAGE graph neural network (GNN) and the heterogeneous graph neural network (HetGNN) architecture, thus called SAGE-Het, enabling it to capture interactions between heterogeneous nodes (i.e., different types of nodes) based on different edges (e.g., edges between trains, trains, and stations). Additionally, SAGE-Het allows for flexible inputs in contrast to conventional machine learning techniques, whose inputs must meet the consistent dimension requirement (e.g., in the form of rectangular or grid-like arrays). The performance and robustness of the suggested SAGE-Het model are assessed in experiments on the data from two sub-networks of the China railway network. The experimental results demonstrate that SAGE-Het outperforms the existing delay prediction methods and some advanced HetGNNs used for prediction tasks in other domains; SAGE-Het demonstrates excellent scalability, capable of handling various types of nodes; the predictive performances of SAGE-Het under different prediction time horizons (10/20/30 min ahead) all exhibit better performance over other baselines; the accuracies are over 90 % under the permissible 3-minute errors for the three prediction time horizons. Specifically, the impact of train interactions on delay evolution is investigated based on the flexible input characteristic of the proposed model. The results illustrate that train interactions become subtle with the increase of train headways. This finding directly contributes to decision-making in situations where conflict resolution or train-canceling actions are needed.

Mix-Key: graph mixup with key structures for molecular property prediction.

Molecular property prediction faces the challenge of limited labeled data as it necessitates a series of specialized experiments to annotate target molecules. Data augmentation techniques can effectively address the issue of data scarcity. In recent years, Mixup has achieved significant success in traditional domains such as image processing. However, its application in molecular property prediction is relatively limited due to the irregular, non-Euclidean nature of graphs and the fact that minor variations in molecular structures can lead to alterations in their properties. To address these challenges, we propose a novel data augmentation method called Mix-Key tailored for molecular property prediction. Mix-Key aims to capture crucial features of molecular graphs, focusing separately on the molecular scaffolds and functional groups. By generating isomers that are relatively invariant to the scaffolds or functional groups, we effectively preserve the core information of molecules. Additionally, to capture interactive information between the scaffolds and functional groups while ensuring correlation between the original and augmented graphs, we introduce molecular fingerprint similarity and node similarity. Through these steps, Mix-Key determines the mixup ratio between the original graph and two isomers, thus generating more informative augmented molecular graphs. We extensively validate our approach on molecular datasets of different scales with several Graph Neural Network architectures. The results demonstrate that Mix-Key consistently outperforms other data augmentation methods in enhancing molecular property prediction on several datasets.

Learning based multi-robot coverage algorithm

Multi-robot coverage algorithm is essential in exploration, search and rescue, tracking and other tasks. Nowadays, global planning-based approaches are difficult to solve the actual deployments of very large robot team coverage problems. In this article we use the heuristic algorithm based on graph neural networks to solve the multi robot coverage algorithm. Firstly, we discretize the coverage task and encode it into a graph. The location of graph and the robots are nodes. Then we design a graph neural network controller and use imitation methods to train the controller. The controller will generate the solution that is not inferior to the expert through imitating an open-loop expert solution based on VPR. Finally, we designed a graph neural network architecture to perform zero shot generalization on large maps and teams, enabling the system to be extended to larger map teams. It is difficult for the expert. And we successfully use this model to simulate 10 quadcopter and a number of buildings in a city. We also prove the GNN controller is better than the method based on the planning in the exploration task.

Identifying Guarantors of War Veterans Using Robust-SEAL: A Case of the Korean War

Most countries provide veterans with various benefits to reward their sacrifice. Unfortunately, many veterans have failed to prove their status due to loss of military records. Thus, some governments allow the verification of those veterans through "buddy statements" obtained from the people who can vouch for the buddy's participation in the war. However, it is still challenging for veterans to find guarantors directly. With this background, we suggest to utilizing historical war records of combined operations to increase the pool of potential guarantors for the buddy statements. However, a combined operation network among troops can have missing edges and perturbations on attributes of the troop due to inaccurate information. In this study, we learn from some recorded interactions which might be incomplete and noisy, and predict missing linkages among the troops that might have interacted together in the war, by proposing Robust-SEAL (learning from Subgraphs, Embeddings, and Attributes for Link prediction). It combines two Graph Neural Network (GNN) architectures: robust Graph Convolutional Network which considers the uncertainty of node attributes with a probabilistic approach, and SEAL which improves the expressive power of the GNN with a labeling trick. Our proposed approach was applied to Korean War data with perturbations. For experimentations, we hid some actual interactions and found that Robust-SEAL restores missing interactions better than other GNN-based baselines.

Multimodal Graph Neural Architecture Search under Distribution Shifts

Multimodal graph neural architecture search (MGNAS) has shown great success for automatically designing the optimal multimodal graph neural network (MGNN) architecture by leveraging multimodal representation, crossmodal information and graph structure in one unified framework. However, existing MGNAS fails to handle distribution shifts that naturally exist in multimodal graph data, since the searched architectures inevitably capture spurious statistical correlations under distribution shifts. To solve this problem, we propose a novel Out-of-distribution Generalized Multimodal Graph Neural Architecture Search (OMG-NAS) method which optimizes the MGNN architecture with respect to its performance on decorrelated OOD data. Specifically, we propose a multimodal graph representation decorrelation strategy, which encourages the searched MGNN model to output representations that eliminate spurious correlations through iteratively optimizing the feature weights and controller. In addition, we propose a global sample weight estimator that facilitates the sharing of optimal sample weights learned from existing architectures. This design promotes the effective estimation of the sample weights for candidate MGNN architectures to generate decorrelated multimodal graph representations, concentrating more on the truly predictive relations between invariant features and ground-truth labels. Extensive experiments on real-world multimodal graph datasets demonstrate the superiority of our proposed method over SOTA baselines.

Prediction of glycopeptide fragment mass spectra by deep learning

Deep learning has achieved a notable success in mass spectrometry-based proteomics and is now emerging in glycoproteomics. While various deep learning models can predict fragment mass spectra of peptides with good accuracy, they cannot cope with the non-linear glycan structure in an intact glycopeptide. Herein, we present DeepGlyco, a deep learning-based approach for the prediction of fragment spectra of intact glycopeptides. Our model adopts tree-structured long-short term memory networks to process the glycan moiety and a graph neural network architecture to incorporate potential fragmentation pathways of a specific glycan structure. This feature is beneficial to model explainability and differentiation ability of glycan structural isomers. We further demonstrate that predicted spectral libraries can be used for data-independent acquisition glycoproteomics as a supplement for library completeness. We expect that this work will provide a valuable deep learning resource for glycoproteomics.

Deep learning enhanced drug discovery for novel biomaterials in regenerative medicine utilizing graph neural network approach for predicting cellular responses

This study introduces a novel approach to drug discovery in regenerative medicine through the utilization of a graph neural network (GNN). The research methodology integrates the development and training of the GNN with a subsequent evaluation of its performance metrics. The first phase involves the generation of synthetic data simulating a biological network, employing networkX and NumPy libraries to construct a random graph with Erdos-Renyi topology. The data, representing cellular responses to biomaterials, is then converted into PyTorch tensors for compatibility with the GNN architecture. The GNN model, characterized by two fully connected layers with ReLU and log-softmax activations, captures intricate relationships within the graph-structured data. The second phase employs a stochastic gradient descent algorithm, specifically the Adam optimizer, to train the GNN over 100 epochs using the cross-entropy loss for multi-class classification. The research methodology extends to the evaluation phase, producing three distinct output graphs for analysis: Visualization of the graph structure, a comparison between predicted and true labels, and a plot illustrating training loss over epochs. Performance metrics, including accuracy, precision, recall, and F1-score, are computed to assess the model’s predictive capabilities quantitatively. The study concludes with a discussion on the nuances revealed by each graph and their implications for refining GNN models in the context of drug discovery for regenerative medicine.

AutoDDI: Drug-Drug Interaction Prediction With Automated Graph Neural Network.

Drug-drug interaction (DDI) has attracted widespread attention because when incompatible drugs are taken together, DDI will lead to adverse effects on the body, such as drug poisoning or reduced drug efficacy. The adverse effects of DDI are closely determined by the molecular structures of the drugs involved. To represent drug data effectively, researchers usually treat the molecular structure of drugs as a molecule graph. Then, previous studies can use the handcrafted graph neural network (GNN) model to learn the molecular graph representations of drugs for DDI prediction. However, in the field of bioinformatics, manually designing GNN architectures for specific molecular structure datasets is time-consuming and depends on expert experience. To address this problem, we propose an automatic drug-drug interaction prediction method named AutoDDI that can efficiently and automatically design the GNN architecture for drug-drug interaction prediction without manual intervention. To this end, we first design an effective search space for drug-drug interaction prediction by revisiting various handcrafted GNN architectures. Then, to efficiently and automatically design the optimal GNN architecture for each drug dataset from the search space, a reinforcement learning search algorithm is adopted. The experiment results show that AutoDDI can achieve the best performance on two real-world datasets. Moreover, the visual interpretation results of the case study show that AutoDDI can effectively capture drug substructure for drug-drug interaction prediction.

A Community Detection and Graph-Neural-Network-Based Link Prediction Approach for Scientific Literature

This study presents a novel approach that synergizes community detection algorithms with various Graph Neural Network (GNN) models to bolster link prediction in scientific literature networks. By integrating the Louvain community detection algorithm into our GNN frameworks, we consistently enhanced the performance across all models tested. For example, integrating the Louvain model with the GAT model resulted in an AUC score increase from 0.777 to 0.823, exemplifying the typical improvements observed. Similar gains were noted when the Louvain model was paired with other GNN architectures, confirming the robustness and effectiveness of incorporating community-level insights. This consistent increase in performance—reflected in our extensive experimentation on bipartite graphs of scientific collaborations and citations—highlights the synergistic potential of combining community detection with GNNs to overcome common link prediction challenges such as scalability and resolution limits. Our findings advocate for the integration of community structures as a significant step forward in the predictive accuracy of network science models, offering a comprehensive understanding of scientific collaboration patterns through the lens of advanced machine learning techniques.

Utilising energy function and variational inference training for learning a graph neural network architecture

Utilising energy function and variational inference training for learning a graph neural network architecture

Combining jumping knowledge into traffic forecasting: An attention-based spatial-temporal adaptive integration gated network

Traffic forecasting has become a core component of Intelligent Transportation Systems. However, accurate traffic forecasting is very challenging, caused by the complex traffic road networks. Most existing forecasting methods do not fully consider the topological structure information of road networks, making it difficult to extract accurate spatial features. In addition, spatial and temporal features have different impacts on traffic conditions, but the existing studies ignore the distribution of spatial-temporal features in traffic regions. To address these limitations, we propose a novel graph neural network architecture named Attention-based Spatial-Temporal Adaptive Integration Gated Network (AST-AIGN). The originality of AST-AIGN is to obtain a spatial feature that more accurately reflects the topological structure of the road networks by embedding Graph Attention Network (GAT) into Jumping Knowledge Net (JK-Net). We propose a data-dependent function called spatial-temporal adaptive integration gate to process the diversity of feature distribution and highlight features in road networks that significantly affects traffic conditions. We evaluate our model on two real-world traffic datasets from the Caltrans Performance Measurement System (PEMS04 and PEMS08), and the extensive experimental results demonstrate the proposed AST-AIGN architecture outperforms other baselines.

Physics-informed graphical neural network for power system state estimation

State estimation is highly critical for accurately observing the dynamic behavior of the power grids and minimizing risks from cyber threats. However, existing state estimation methods encounter challenges in accurately capturing power system dynamics, primarily because of limitations in encoding the grid topology and sparse measurements. This paper proposes a physics-informed graphical learning state estimation method to address these limitations by leveraging both domain physical knowledge and a graph neural network (GNN). We employ a GNN architecture that can handle the graph-structured data of power systems more effectively than traditional data-driven methods. The physics-based knowledge is constructed from the branch current formulation, making the approach adaptable to both transmission and distribution systems. The validation results of three IEEE test systems show that the proposed method can achieve lower mean square error more than 20% than the conventional methods.

Graph neural network and machine learning analysis of functional neuroimaging for understanding schizophrenia.

Graph representational learning can detect topological patterns by leveraging both the network structure as well as nodal features. The basis of our exploration involves the application of graph neural network architectures and machine learning to resting-state functional Magnetic Resonance Imaging (rs-fMRI) data for the purpose of detecting schizophrenia. Our study uses single-site data to avoid the shortcomings in generalizability of neuroimaging data obtained from multiple sites. The performance of our graph neural network models is on par with that of our machine learning models, each of which is trained using 69 graph-theoretical measures computed from functional correlations between various regions of interest (ROI) in a brain graph. Our deep graph convolutional neural network (DGCNN) demonstrates a promising average accuracy score of 0.82 and a sensitivity score of 0.84. This study provides insights into the role of advanced graph theoretical methods and machine learning on fMRI data to detect schizophrenia by harnessing changes in brain functional connectivity. The results of this study demonstrate the capabilities of using both traditional ML techniques as well as graph neural network-based methods to detect schizophrenia using features extracted from fMRI data. The study also proposes two methods to obtain potential biomarkers for the disease, many of which are corroborated by research in this area and can further help in the understanding of schizophrenia as a mental disorder.

CommGNAS: Unsupervised Graph Neural Architecture Search for Community Detection

Graph neural architecture search (GNAS) has been successful in many supervised learning tasks, such as node classification, graph classification, and link prediction. GNAS uses a search algorithm to sample graph neural network (GNN) architectures from the search space and evaluates sampled GNN architectures based on estimation strategies to generate feedback for the search algorithm. In traditional GNAS, the typical estimation strategy requires using labeled graph data to generate feedback, which plays a fundamental and vital role in the search algorithm to sample a better GNN architecture during the search process. However, a large portion of real-world graph data is unlabeled. The estimation strategy in traditional GNAS cannot use unlabeled graph data to generate feedback for the search algorithm, so the traditional supervised GNAS fails to solve unsupervised problems, such as community detection tasks. To solve this challenge, this paper proposed CommGNAS, an effective node representation learning method with unsupervised graph neural architecture search for community detection. In CommGNAS, we design an unsupervised evaluation strategy with self-supervised and self-representation learning. It represents the first research work in literature to solve the problems of unsupervised graph neural architecture search for community detection. The experimental results show that CommGNAS can obtain the best performance in community detection tasks on real-world graphs against the state-of-the-art baseline methods.

Molecular graph transformer: stepping beyond ALIGNN into long-range interactions

We present the Molecular Graph Representation (MGR) and the Molecular Graph Transformer (MGT): a novel graph neural network architecture that captures long-range interactions for predicting solid-state material properties.

Gegenbauer Graph Neural Networks for Time-Varying Signal Reconstruction.

Reconstructing time-varying graph signals (or graph time-series imputation) is a critical problem in machine learning and signal processing with broad applications, ranging from missing data imputation in sensor networks to time-series forecasting. Accurately capturing the spatio-temporal information inherent in these signals is crucial for effectively addressing these tasks. However, existing approaches relying on smoothness assumptions of temporal differences and simple convex optimization techniques that have inherent limitations. To address these challenges, we propose a novel approach that incorporates a learning module to enhance the accuracy of the downstream task. To this end, we introduce the Gegenbauer-based graph convolutional (GegenConv) operator, which is a generalization of the conventional Chebyshev graph convolution by leveraging the theory of Gegenbauer polynomials. By deviating from traditional convex problems, we expand the complexity of the model and offer a more accurate solution for recovering time-varying graph signals. Building upon GegenConv, we design the Gegenbauer-based time graph neural network (GegenGNN) architecture, which adopts an encoder-decoder structure. Likewise, our approach also uses a dedicated loss function that incorporates a mean squared error (MSE) component alongside Sobolev smoothness regularization. This combination enables GegenGNN to capture both the fidelity to ground truth and the underlying smoothness properties of the signals, enhancing the reconstruction performance. We conduct extensive experiments on real datasets to evaluate the effectiveness of our proposed approach. The experimental results demonstrate that GegenGNN outperforms state-of-the-art methods, showcasing its superior capability in recovering time-varying graph signals.

Using graph neural networks for wall modeling in compressible anisothermal flows

Abstract Compressible anisothermal flows, which are commonly found in industrial settings such as combustion chambers and heat exchangers, are characterized by significant variations in density, viscosity, and heat conductivity with temperature. These variations lead to a strong interaction between the temperature and velocity fields that impacts the near-wall profiles of both quantities. Wall-modeled large-eddy simulations (LESs) rely on a wall model to provide a boundary condition, for example, the shear stress and the heat flux that accurately represents this interaction despite the use of coarse cells near the wall, and thereby achieve a good balance between computational cost and accuracy. In this article, the use of graph neural networks for wall modeling in LES is assessed for compressible anisothermal flow. Graph neural networks are a type of machine learning model that can learn from data and operate directly on complex unstructured meshes. Previous work has shown the effectiveness of graph neural network wall modeling for isothermal incompressible flows. This article develops the graph neural network architecture and training to extend their applicability to compressible anisothermal flows. The model is trained and tested a priori using a database of both incompressible isothermal and compressible anisothermal flows. The model is finally tested a posteriori for the wall-modeled LES of a channel flow and a turbine blade, both of which were not seen during training.

PEB-DDI: A Task-Specific Dual-View Substructural Learning Framework for Drug-Drug Interaction Prediction.

Adverse drug-drug interactions (DDIs) pose potential risks in polypharmacy due to unknown physicochemical incompatibilities between co-administered drugs. Recent studies have utilized multi-layer graph neural network architectures to model hierarchical molecular substructures of drugs, achieving excellent DDI prediction performance. While extant substructural frameworks effectively encode interactions from atom-level features, they overlook valuable chemical bond representations within molecular graphs. More critically, given the multifaceted nature of DDI prediction tasks involving both known and novel drug combinations, previous methods lack tailored strategies to address these distinct scenarios. The resulting lack of adaptability impedes further improvements to model performance. To tackle these challenges, we propose PEB-DDI, a DDI prediction learning framework with enhanced substructure extraction. First, the information of chemical bonds is integrated and synchronously updated with the atomic nodes. Then, different dual-view strategies are selected based on whether novel drugs are present in the prediction task. Particularly, we constructed Molecular fingerprint-Molecular graph view for transduc- tive task, and Bipartite graph-Molecular graph view for inductive task. Rigorous evaluations on benchmark datasets underscore PEB-DDI's superior performance. Notably, on DrugBank, it achieves an outstanding accuracy rate of 98.18% when predicting previously unknown interactions among approved drugs. Even when faced with novel drugs, PEB-DDI consistently exhibits outstanding generalization capabilities with an accuracy rate of 88.06%, attributing to the proper migrating of molecular basic structure learning.

GNN-LoFI: A novel graph neural network through localized feature-based histogram intersection

Graph neural networks are increasingly becoming the framework of choice for graph-based machine learning. In this paper, we propose a new graph neural network architecture that substitutes classical message passing with an analysis of the local distribution of node features. To this end, we extract the distribution of features in the egonet for each local neighbourhood and compare them against a set of learned label distributions by taking the histogram intersection kernel. The similarity information is then propagated to other nodes in the network, effectively creating a message passing-like mechanism where the message is determined by the ensemble of the features. We perform an ablation study to evaluate the network’s performance under different choices of its hyper-parameters. Finally, we test our model on standard graph classification and regression benchmarks, and we find that it outperforms widely used alternative approaches, including both graph kernels and graph neural networks.