Deep learning simulation and decision support system for groundwater salinity risk assessment in the lower Chao Phraya River Basin, Thailand.

BackgroundThe proportion of Canadian youth seeking mental health support from an emergency department (ED) has risen in recent years. As EDs typically address urgent mental health crises, revisiting an ED may represent unmet mental health needs. Accurate ED revisit prediction could aid early intervention and ensure efficient healthcare resource allocation. We examine the potential increased accuracy and performance of graph neural network (GNN) machine learning models compared to recurrent neural network (RNN), and baseline conventional machine learning and regression models for predicting ED revisit in electronic health record (EHR) data.MethodsThis study used EHR data for children and youth aged 4–17 seeking services at McMaster Children’s Hospital’s Child and Youth Mental Health Program outpatient service to develop and evaluate GNN and RNN models to predict whether a child/youth with an ED visit had an ED revisit within 30 days. GNN and RNN models were developed and compared against conventional baseline models. Model performance for GNN, RNN, XGBoost, decision tree and logistic regression models was evaluated using F1 scores.ResultsThe GNN model outperformed the RNN model by an F1-score increase of 0.0511 and the best performing conventional machine learning model by an F1-score increase of 0.0470. Precision, recall, receiver operating characteristic (ROC) curves, and positive and negative predictive values showed that the GNN model performed the best, and the RNN model performed similarly to the XGBoost model. Performance increases were most noticeable for recall and negative predictive value than for precision and positive predictive value.ConclusionsThis study demonstrates the improved accuracy and potential utility of GNN models in predicting ED revisits among children and youth, although model performance may not be sufficient for clinical implementation. Given the improvements in recall and negative predictive value, GNN models should be further explored to develop algorithms that can inform clinical decision-making in ways that facilitate targeted interventions, optimize resource allocation, and improve outcomes for children and youth.

Deep learning has seen significant growth recently and is now applied to a wide range of conventional use cases, including graphs. Graph data provides relational information between elements and is a standard data format for various machine learning and deep learning tasks. Models that can learn from such inputs are essential for working with graph data effectively. This paper identifies nodes and edges within specific applications, such as text, entities, and relations, to create graph structures. Different applications may require various graph neural network (GNN) models. GNNs facilitate the exchange of information between nodes in a graph, enabling them to understand dependencies within the nodes and edges. The paper delves into specific GNN models like graph convolution networks (GCNs), GraphSAGE, and graph attention networks (GATs), which are widely used in various applications today. It also discusses the message-passing mechanism employed by GNN models and examines the strengths and limitations of these models in different domains. Furthermore, the paper explores the diverse applications of GNNs, the datasets commonly used with them, and the Python libraries that support GNN models. It offers an extensive overview of the landscape of GNN research and its practical implementations.

Background: Tertiary lymphoid structures (TLS) are vascularized lymphocyte aggregates in the tumor microenvironment (TME) that correlate with better patient outcomes. Previous studies identified a 12 chemokine gene expression signature associated with disease progression and the type and degree of TLS. These signatures could provide insight important for clinical decision making during pathologic evaluation, but predicting gene expression from whole slide images (WSI) may be impeded by low prediction accuracy and lack of interpretability. Here we report an artificial intelligence (AI)-based, state-of-the-art workflow to predict the 12-chemokine TLS gene signature from lung cancer WSI, and identify histological features relevant to model predictions. Methods: Models were trained using 538 cases of paired lung cancer WSI and mRNA-seq expression data (The Cancer Genome Atlas). Cell and tissue classifiers, based on convolutional neural networks (CNN) were trained on WSI, and a graph neural network (GNN) model that leverages the relative spatial arrangement of the CNN-identified cells and tissues was used to predict gene expression. GNN predictions of TLS signature genes were compared with the predictions of models trained using hand-crafted, task-specific features (TLS feature models) describing the number, size, and cellular composition of identified TLS. The Pearson correlation coefficient was used to assess the accuracy of GNN and TLS feature model predictions. GNNExplainer1, a tool that simultaneously identifies a subgraph and a subset of node features important for predictions, was applied to interpret the GNN model predictions. Results: GNN model predictions show reasonable accuracy: GNN models significantly predicted mRNA expression of all 12 genes (p<0.05), and the predicted expression of six genes was moderately correlated with ground-truth measurements (Pearson-r>0.5). The correlation of GNN predictions was higher than that of the TLS feature models for all 12 signature genes. The GNNExplainer identified relevant features including the mean and standard deviation of lymphocyte count, and fraction of lymphocytes in cancer stroma. Subgraphs selected by the GNNExplainer focus on, but extend beyond, regions of human-annotated TLS objects, indicating that TLS may influence gene expression and the TME in regions beyond their immediate vicinity. Conclusion: Here, we show a comparison of two interpretable AI methods for the prediction of TLS-induced gene expression from WSI. The outperforming GNN-based approach is highly reproducible and accurate, predicting histopathology features relevant to TLS that may be used to inform patient prognosis and treatment. These methods could be applied to predict additional clinically relevant transcriptomic signatures. 1. ​​Ying, R, et al. 2019. arXiv:1903.03894v4 Citation Format: Ciyue Shen, Collin Schlager, Deepta Rajan, Maryam Pouryahya, Mary Lin, Victoria Mountain, Ilan Wapinski, Amaro Taylor-Weiner, Benjamin Glass, Robert Egger, Andrew Beck. Application of an interpretable graph neural network to predict gene expression signatures associated with tertiary lymphoid structures in histopathological images [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 1922.

Many learning tasks require us to deal with graph data which contains rich relational information among elements, leading increasing graph neural network (GNN) models to be deployed in industrial products for improving the quality of service. However, they also raise challenges to model authentication. It is necessary to protect the ownership of the GNN models, which motivates us to present a watermarking method to GNN models in this paper. In the proposed method, an Erdos-Renyi (ER) random graph with random node feature vectors and labels is randomly generated as a trigger to train the GNN to be protected together with the normal samples. During model training, the secret watermark is embedded into the label predictions of the ER graph nodes. During model verification, by activating a marked GNN with the trigger ER graph, the watermark can be reconstructed from the output to verify the ownership. Since the ER graph was randomly generated, by feeding it to a non-marked GNN, the label predictions of the graph nodes are random, resulting in a low false alarm rate (of the proposed work). Experimental results have also shown that, the performance of a marked GNN on its original task will not be impaired. Moreover, it is robust against model compression and fine-tuning, which has shown the superiority and applicability.

Recent research on graph neural network (GNN) models successfully applied GNNs to classical graph algorithms and combinatorial optimisation problems. This has numerous benefits, such as allowing applications of algorithms when preconditions are not satisfied, or reusing learned models when sufficient training data is not available or can't be generated. Unfortunately, a key hindrance of these approaches is their lack of explainability, since GNNs are black-box models that cannot be interpreted directly. In this work, we address this limitation by applying existing work on concept-based explanations to GNN models. We introduce concept-bottleneck GNNs, which rely on a modification to the GNN readout mechanism. Using three case studies we demonstrate that: (i) our proposed model is capable of accurately learning concepts and extracting propositional formulas based on the learned concepts for each target class; (ii) our concept-based GNN models achieve comparative performance with state-of-the-art models; (iii) we can derive global graph concepts, without explicitly providing any supervision on graph-level concepts.

Field-programmable gate arrays (FPGAs) are an ideal candidate for accelerating graph neural networks (GNNs). However, the FPGA redeployment process is time-consuming when updating or switching between diverse GNN models across different applications. Existing GNN processors eliminate the need for FPGA redeployment when switching between different GNN models. However, adapting matrix multiplication types by switching processing units decreases hardware utilization. In addition, the bandwidth of DDR limits further improvements in hardware performance. This article proposes a highly flexible FPGA-based overlay processor for GNN accelerations. Graph-OPU provides excellent flexibility and programmability for users, as the executable code of GNN models is automatically compiled and reloaded without requiring FPGA redeployment. First, we customize the compiler and instruction sets for the inference process of different GNN models. Second, we customize the datapath and optimize the data format in the microarchitecture to fully leverage the advantages of high bandwidth memory (HBM). Third, we design a unified matrix multiplication to handle both sparse-dense matrix multiplication (SpMM) and general matrix multiplication (GEMM), enhancing Graph-OPU performance. During Graph-OPU execution, the computational units are shared between SpMM and GEMM instead of being switched, which improves the hardware utilization. Finally, we implement a hardware prototype on the Xilinx Alveo U50 and test the mainstream GNN models using various datasets. Experimental results show that Graph-OPU achieves up to 1,654 \(\times\) and 63 \(\times\) speedup, as well as up to 5,305 \(\times\) and 422 \(\times\) energy efficiency boosts, compared to implementations on CPU and GPU, respectively. Graph-OPU outperforms state-of-the-art (SOTA) end-to-end overlay accelerators for GNN, reducing latency by an average of 1.36 \(\times\) and improving energy efficiency by 1.41 \(\times\) on average. Moreover, Graph-OPU exhibits an average 1.45 \(\times\) speed improvement in end-to-end latency over the SOTA GNN processor. Graph-OPU represents an in-depth study of an FPGA-based overlay processor for GNNs, offering high flexibility, speedup, and energy efficiency.

This work presents a crystal structure prediction framework that employs a structural search using a derivative-free optimization method, with a supervised Graph Neural Network (GNN) model as the energy evaluator. We address the limitations of existing GNN-based crystal structure prediction (CSP) frameworks and propose methods for designing a robust and computationally efficient predictor. In particular, we first highlight the often-overlooked sensitivity of GNN models to weight initialization in crystal structure prediction, and to address this, we introduce a model selection framework that consistently identifies an appropriate GNN model for downstream crystal structure prediction tasks. Using this framework, we conduct a meaningful comparison of multiple GNN architectures for CSP involving a Bayesian optimization approach. Furthermore, we propose a data augmentation strategy that incorporates unrelaxed structures in the supervised training process, and additionally explore the impact of unsupervised GNN pretraining with and without augmentation on crystal structure prediction. Finally, we demonstrate that our proposed crystal structure prediction framework, in conjunction with the lightweight GNN architecture CGCNN, can achieve a level of performance comparable to that of more complex GNN architectures, which are typically computationally expensive to train and infer. The approaches introduced in this work are generic and can be extended to any GNN-based crystal structure prediction framework, paving the way for developing novel and high-throughput crystal structure predictors in the future.

This paper presents a framework for modeling and predicting impeded aircraft taxi-out times based on machine learning techniques. The presented framework can be integrated into departure management systems to support the pretactical/tactical planning of departure movements and the optimization of airport resources. The taxi-out time is modeled with two components: the time taken to travel from the gate to the departure queue and the time spent in the departure queue. The first component (termed the taxiing time) is mainly affected by surface traffic conditions, while the latter component (termed the queuing time) can be more accurately modeled using characteristics derived from the departure queue. To model the spatiotemporal dependencies on traffic flow, we represent the airport taxi system as a node-link model. Flow features are derived in the form of edge attributes based on route information and movement start times. Departure trajectories utilize the same node-link representation, in the form of a subgraph incorporating additional operationally available information. The taxi-out time of each trajectory is obtained by processing the subgraph using a graph neural network (GNN) with transformer layers. Predictions from the GNN model are compared against standard methodologies by the Federal Aviation Administration (FAA) and EUROCONTROL, as well as against predictions made by gradient boosting machines (GBM), a popular decision-tree-based machine learning technique. Results show that both GNN and GBM models outperform the standard FAA and EUROCONTROL methods (with the prediction errors of the former group lower by 40–60% relative to the latter), and the novel GNN model outperforms the GBM model by a considerable margin of approximately 8 s, translating to a 10% improvement in model performance of the GNN model relative to the GBM model.

In machine-learning-assisted high-throughput defect studies, a defect-aware latent representation of the supercell structure is crucial for the accurate prediction of defect properties. The performance of current graph neural network (GNN) models is limited due to the fact that defect properties depend strongly on the local atomic configurations near the defect sites and due to the oversmoothing problem of GNN. Herein, we demonstrate that persistent homology features, which encode the topological information on the local chemical environment around each atomic site, can characterize the structural information on defects. Using the dataset containing a wide spectrum of O-based perovskites with all available vacancies as an example, we show that incorporating the persistent homology features, along with proper choices of graph pooling operations, significantly increases the prediction accuracy, with the MAE reduced by 55%. Those features can be easily integrated into the state-of-the-art GNN models, including the graph Transformer network and the equivariant neural network, and universally improve their performance. Besides, our model also overcomes the convergence issue with respect to the supercell size that was present in previous GNN models. Furthermore, using the datasets of defective BaTiO3 with multiple substitutions and multiple vacancies as examples, our GNN model can also predict the defect-defect interactions accurately. These results suggest that persistent homology features can effectively improve the performance of machine learning models and assist the accelerated discovery of functional defects for technological applications.

Sex classification is a major benchmark of previous work in learning on the structural connectome, a naturally occurring brain graph that has proven useful for studying cognitive function and impairment. While graph neural networks (GNNs), specifically graph convolutional networks (GCNs), have gained popularity lately for their effectiveness in learning on graph data, achieving strong performance in adult sex classification tasks, their application to pediatric populations remains unexplored. We seek to characterize the capacity for GNN models to learn connectomic patterns on pediatric data through an exploration of training techniques and architectural design choices. Two datasets comprising an adult BRIGHT dataset (N = 147 Hodgkin's lymphoma survivors and N = 162 age similar controls) and a pediatric Human Connectome Project in Development (HCP-D) dataset (N = 135 healthy subjects) were utilized. Two GNN models (GCN simple and GCN residual), a deep neural network (multi-layer perceptron), and two standard machine learning models (random forest and support vector machine) were trained. Architecture exploration experiments were conducted to evaluate the impact of network depth, pooling techniques, and skip connections on the ability of GNN models to capture connectomic patterns. Models were assessed across a range of metrics including accuracy, AUC score, and adversarial robustness. GNNs outperformed other models across both populations. Notably, adult GNN models achieved 85.1% accuracy in sex classification on unseen adult participants, consistent with prior studies. The extension of the adult models to the pediatric dataset and training on the smaller pediatric dataset were sub-optimal in their performance. Using adult data to augment pediatric models, the best GNN achieved comparable accuracy across unseen pediatric (83.0%) and adult (81.3%) participants. Adversarial sensitivity experiments showed that the simple GCN remained the most robust to perturbations, followed by the multi-layer perceptron and the residual GCN. These findings underscore the potential of GNNs in advancing our understanding of sex-specific neurological development and disorders and highlight the importance of data augmentation in overcoming challenges associated with small pediatric datasets. Further, they highlight relevant tradeoffs in the design landscape of connectomic GNNs. For example, while the simpler GNN model tested exhibits marginally worse accuracy and AUC scores in comparison to the more complex residual GNN, it demonstrates a higher degree of adversarial robustness.

Salinization of groundwater by seawater intrusion is a major concern for the coastal aquifers worldwide. Seawater intrusion occurs mainly due to overpumping of freshwater and sea-level rise which causes lateral and vertical movements of seawater into the coastal aquifers. There are several methods to identify and assess the extent of seawater ingress towards land. In the present study, a hydrochemical approach is adopted to understand the status of intrusion. The ions such as Na+, K+, Ca2+, Mg2+, Cl−, SO4 2−, HCO 3 2− , and in-situ parameters namely TDS, pH, and EC were determined for 174 water samples collected both in pre- and post-monsoon in Puducherry region, India. Ionic ratios such as HCO3 2−/Cl−, Na+/Cl−, Ca2+/Cl−, Ca2+/Na+, Mg2+/Cl−, K+/Cl−, SO4 2−/Cl−, and Cl−/Br− were calculated and correlated with total dissolved solids (TDS) to evaluate seawater intrusion. The ionic ratios such as HCO3 2−/Cl−, Ca2+/Cl−, Mg2+/Cl−, K+/Cl−, and SO4 2−/Cl− vs. TDS shows negative correlation indicating salinization of groundwater. Cl−/Br− ratio is used to distinguish the causes of salinity in groundwater. The Hydrochemical Facies Evolution diagram (HFE-diagram) and heat maps generated out of it have been very well used to understand evolution of seawater intrusion and freshening process in the coastal aquifer to time. The majority of samples in pre-monsoon fall under the facies Na–HCO3/SO4, followed by Na–mixHCO3/mix SO4, Mix Na–HCO3/Mix SO4, and MixNa–MixHCO3/mix SO4 facies indicating direct cations exchange process, whereas, in post-monsoon, Na–Cl, Mix Na–Cl, and Mix Ca–Cl facies are dominant indicating reverse ion exchange process. In the study area, five locations, viz. Ariyankuppam, Kariambattur, Kalapet, Mutialpet, and Parikalpet, fall under Na–Cl and Ca–Cl facies in pre- and post-monsoon which indicates consistent seawater intrusion. The hydrochemical changes that take place during seawater freshwater interaction along coastal aquifer are determined by ionic exchange. About 24.2% of samples in pre-monsoon and 13.5% of samples in post-monsoon show mixing of seawater. The highly negative ionic exchange values of sodium during pre-monsoon indicate increased amount of seawater fraction in groundwater.

Alzheimer's disease (AD) is a neurodegenerative disease that significantly impacts the quality of life of patients and their families. Neuroimaging-driven brain age prediction has been proposed as a potential biomarker to detect mental disorders, such as AD, aiding in studying its effects on functional brain networks. Previous studies have shown that individuals with AD display impaired resting-state functional connections. However, most studies on brain age prediction have used structural magnetic resonance imaging (MRI), with limited studies based on resting-state functional MRI (rs-fMRI). In this study, we applied a graph neural network (GNN) model on controls to predict brain ages using rs-fMRI in patients with AD. We compared the performance of the GNN model with traditional machine learning models. Finally, the post hoc model was also used to identify the critical brain regions in AD. The experimental results demonstrate that our GNN model can predict brain ages of normal controls using rs-fMRI data from the ADNI database. Moreover the differences between brain ages and chronological ages were more significant in AD patients than in normal controls. Our results also suggest that AD is associated with accelerated brain aging and that the GNN model based on resting-state functional connectivity is an effective tool for predicting brain age. Our study provides evidence that rs-fMRI is a promising modality for brain age prediction in AD research, and the GNN model proves to be effective in predicting brain age. Furthermore, the effects of the hippocampus, parahippocampal gyrus, and amygdala on brain age prediction are verified.

The cold-start issue is a fundamental challenge in Recommender Systems. The recent self-supervised learning (SSL) on Graph Neural Networks (GNNs) model, PT-GNN, pre-trains the GNN model to reconstruct the cold-start embeddings and has shown great potential for cold-start recommendation. However, due to the over-smoothing problem, PT-GNN can only capture up to 3-order relation, which cannot provide much useful auxiliary information to depict the target cold-start user or item. Besides, the embedding reconstruction task only considers the intra-correlations within the subgraph of users and items, while ignoring the inter-correlations across different subgraphs. To solve the above challenges, we propose a multi-strategy-based pre-training method for cold-start recommendation (MPT), which extends PT-GNN from the perspective of model architecture and pretext tasks to improve the cold-start recommendation performance. 1 Specifically, in terms of the model architecture, in addition to the short-range dependencies of users and items captured by the GNN encoder, we introduce a Transformer encoder to capture long-range dependencies. In terms of the pretext task, in addition to considering the intra-correlations of users and items by the embedding reconstruction task, we add an embedding contrastive learning task to capture inter-correlations of users and items. We train the GNN and Transformer encoders on these pretext tasks under the meta-learning setting to simulate the real cold-start scenario, making the model able to be easily and rapidly adapted to new cold-start users and items. Experiments on three public recommendation datasets show the superiority of the proposed MPT model against the vanilla GNN models, the pre-training GNN model on user/item embedding inference, and the recommendation task.

A Laplacian regularized graph neural network for predictive modeling of multiple chronic conditions

Developing fast and accurate computational models to simulate intricate physical phenomena has been a persistent research challenge. Recent studies have demonstrated remarkable capabilities in predicting various physical outcomes through machine learning-assisted approaches. However, it remains challenging to generalize current methods, usually crafted for a specific problem, to other more complex or broader scenarios. To address this challenge, we developed graph neural network (GNN) models with enhanced generalizability derived from the distinct GNN architecture and neural operator techniques. As a proof of concept, we employ our GNN models to predict finite element (FE) simulation results for three-dimensional solid mechanics problems with varying boundary conditions. Results show that our GNN model achieves accurate and robust performance in predicting the stress and deformation profiles of structures compared with FE simulations. Furthermore, the neural operator embedded GNN approach enables learning and predicting various solid mechanics problems in a generalizable fashion, making it a promising approach for surrogate modeling.