Graph Machine Learning Research Articles

Use of Patterns of Service Utilization and Hierarchical Survival Analysis in Planning and Providing Care for Overdose Patients and Predicting the Time-to-Second Overdose

Individuals from a variety of backgrounds are affected by the opioid crisis. To provide optimal care for individuals at risk of opioid overdose and prevent subsequent overdoses, a more targeted response that goes beyond the traditional taxonomical diagnosis approach to care management needs to be adopted. In previous works, Graph Machine Learning and Natural Language Processing methods were used to model the products for planning and evaluating the treatment of patients with complex issues. This study proposes a methodology of partitioning patients in the opioid overdose cohort into various communities based on their patterns of service utilization (PSUs) across the continuum of care using graph community detection and applying survival analysis to predict time-to-second overdose for each of the communities. The results demonstrated that the overdose cohort is not homogeneous with respect to the determinants of risk. Moreover, the risk for subsequent overdose was quantified: there is a 51% higher chance of experiencing a second overdose for a high-risk community compared to a low-risk community. The proposed method can inform a more efficient treatment heterogeneity approach for a cohort made of diverse individuals, such as the opioid overdose cohort. It can also guide targeted support for patients at risk of subsequent overdoses.

Analyzing Patterns of Service Utilization Using Graph Topology to Understand the Dynamic of the Engagement of Patients with Complex Problems with Health Services

Background: Providing care to persons with complex problems is inherently difficult due to several factors, including the impacts of proximal determinants of health, treatment response, the natural emergence of comorbidities, and service system capacity to provide timely required services. Providing visibility into the dynamics of patients’ engagement can help to optimize care for patients with complex problems. Method: In a previous work, graph machine learning and NLP methods were used to model the products of service system dynamics as atemporal entities, using a data model that collapsed patient encounter events across time. In this paper, the order of events is put back into the data model to provide topological depictions of the dynamics that are embodied in patients’ movement across a complex healthcare system. Result: The results show that directed graphs are well suited to the task of depicting the way that the diverse components of the system are functionally coupled—or remain disconnected—by patient journeys. Conclusion: By setting the resolution on the graph topology visualization, important characteristics can be highlighted, including highly prevalent repeating sequences of service events readily interpretable by clinical subject matter experts. Moreover, this methodology provides a first step in addressing the challenge of locating potential operational problems for patients with complex issues engaging with a complex healthcare service system.

Exploring the Potential of Large Language Models (LLMs)in Learning on Graphs

Learning on Graphs has attracted immense attention due to its wide real-world applications. The most popular pipeline for learning on graphs with textual node attributes primarily relies on Graph Neural Networks (GNNs), and utilizes shallow text embedding as initial node representations, which has limitations in general knowledge and profound semantic understanding. In recent years, Large Language Models (LLMs) have been proven to possess extensive common knowledge and powerful semantic comprehension abilities that have revolutionized existing workflows to handle text data. In this paper, we aim to explore the potential of LLMs in graph machine learning, especially the node classification task, and investigate two possible pipelines: LLMs-as-Enhancers and LLMs-as-Predictors. The former leverages LLMs to enhance nodes' text attributes with their massive knowledge and then generate predictions through GNNs. The latter attempts to directly employ LLMs as standalone predictors. We conduct comprehensive and systematical studies on these two pipelines under various settings. From comprehensive empirical results, we make original observations and find new insights that open new possibilities and suggest promising directions to leverage LLMs for learning on graphs. Our codes and datasets are available at: https://github.com/CurryTang/Graph-LLM .

Robust Graph Meta-Learning for Weakly Supervised Few-Shot Node Classification

Graph machine learning (Graph ML) models typically require abundant labeled instances to provide sufficient supervision signals, which is commonly infeasible in real-world scenarios since labeled data for newly emerged concepts (e.g., new categorizations of nodes) on graphs is rather limited. To efficiently learn with a small amount of data on graphs, meta-learning has been investigated in Graph ML. By transferring the knowledge learned from previous experiences to new tasks, graph meta-learning approaches have demonstrated promising performance on few-shot graph learning problems. However, most existing efforts predominately assume that all the data from the seen classes is gold labeled, yet those methods may lose their efficacy when the seen data is weakly labeled with severe label noise. As such, we aim to investigate a novel problem of weakly supervised graph meta-learning for improving the model robustness in terms of knowledge transfer. To achieve this goal, we propose Meta-GIN (Meta Graph Interpolation Network), a new graph meta-learning framework. Based on a new robustness-enhanced episodic training paradigm, Meta-GIN is meta-learned to interpolate node representations from weakly labeled data and extracts highly transferable meta-knowledge, which enables the model to quickly adapt to unseen tasks with few labeled instances. Extensive experiments demonstrate the superiority of Meta-GIN over existing graph meta-learning studies on the task of weakly supervised few-shot node classification.

Applying precision medicine principles to the management of multimorbidity: the utility of comorbidity networks, graph machine learning, and knowledge graphs.

The current management of patients with multimorbidity is suboptimal, with either a single-disease approach to care or treatment guideline adaptations that result in poor adherence due to their complexity. Although this has resulted in calls for more holistic and personalized approaches to prescribing, progress toward these goals has remained slow. With the rapid advancement of machine learning (ML) methods, promising approaches now also exist to accelerate the advance of precision medicine in multimorbidity. These include analyzing disease comorbidity networks, using knowledge graphs that integrate knowledge from different medical domains, and applying network analysis and graph ML. Multimorbidity disease networks have been used to improve disease diagnosis, treatment recommendations, and patient prognosis. Knowledge graphs that combine different medical entities connected by multiple relationship types integrate data from different sources, allowing for complex interactions and creating a continuous flow of information. Network analysis and graph ML can then extract the topology and structure of networks and reveal hidden properties, including disease phenotypes, network hubs, and pathways; predict drugs for repurposing; and determine safe and more holistic treatments. In this article, we describe the basic concepts of creating bipartite and unipartite disease and patient networks and review the use of knowledge graphs, graph algorithms, graph embedding methods, and graph ML within the context of multimorbidity. Specifically, we provide an overview of the application of graph theory for studying multimorbidity, the methods employed to extract knowledge from graphs, and examples of the application of disease networks for determining the structure and pathways of multimorbidity, identifying disease phenotypes, predicting health outcomes, and selecting safe and effective treatments. In today's modern data-hungry, ML-focused world, such network-based techniques are likely to be at the forefront of developing robust clinical decision support tools for safer and more holistic approaches to treating older patients with multimorbidity.

Machine learning assisted crystal structure prediction made simple

Crystal structure prediction (CSP) plays a crucial role in condensed matter physics and materials science, with its importance evident not only in theoretical research but also in the discovery of new materials and the advancement of novel technologies. However, due to the diversity and complexity of crystal structures, trial-and-error experimental synthesis is time-consuming, labor-intensive, and insufficient to meet the increasing demand for new materials. In recent years, machine learning (ML) methods have significantly boosted CSP. In this review, we present a comprehensive review of the ML models applied in CSP. We first introduce the general steps for CSP and highlight the bottlenecks in conventional CSP methods. We further discuss the representation of crystal structures and illustrate how ML-assisted CSP works. In particular, we review the applications of graph neural networks (GNNs) and ML force fields in CSP, which have been demonstrated to significantly speed up structure search and optimization. In addition, we provide an overview of advanced generative models in CSP, including variational autoencoders (VAEs), generative adversarial networks (GANs), and diffusion models. Finally, we discuss the remaining challenges in ML-assisted CSP.

Credit Risk Modeling with Graph Machine Learning

Accurate credit ratings are an essential ingredient in the decision-making process for investors, rating agencies, bond portfolio managers, bankers, and policy makers, as well as an important input for risk management and regulation. Credit ratings are traditionally generated from models that use financial statement data and market data, which are tabular (numeric and categorical). Using machine learning methods, we construct a network of firms using U.S. Securities and Exchange Commission (SEC) filings (denoted CorpNet) to enhance the traditional tabular data set with a corporate graph. We show that this generates accurate rating predictions with comparable and better performance to tabular models. We ensemble graph convolutional networks with highly-performant ensembled machine learning models using AutoGluon. This paper demonstrates both transductive and inductive methodologies to extend credit scoring models based on tabular data, which have been used by the ratings industry for decades, to the class of machine learning models on networks. The methodology is extensible to other financial machine learning models that may be enhanced using a corporate graph. History: David Martens served as the senior editor for this article. Data Ethics & Reproducibility Note: No data ethics considerations are foreseen related to this article. The paper deals with corporate credit risk and not consumer credit, which usually entails issues around privacy and bias. The code capsule is available on Code Ocean at https://codeocean.com/capsule/5230264/tree/v2 and in the e-Companion to this article (available at https://doi.org/10.1287/ijds.2022.00018 ).

Visibility graph analysis for brain: scoping review.

In the past two decades, network-based analysis has garnered considerable attention for analyzing time series data across various fields. Time series data can be transformed into graphs or networks using different methods, with the visibility graph (VG) being a widely utilized approach. The VG holds extensive applications in comprehending, identifying, and predicting specific characteristics of time series data. Its practicality extends to domains such as medicine, economics, meteorology, tourism, and others. This research presents a scoping review of scholarly articles published in reputable English-language journals and conferences, focusing on VG-based analysis methods related to brain disorders. The aim is to provide a foundation for further and future research endeavors, beginning with an introduction to the VG and its various types. To achieve this, a systematic search and refinement of relevant articles were conducted in two prominent scientific databases: Google Scholar and Scopus. A total of 51 eligible articles were selected for a comprehensive analysis of the topic. These articles categorized based on publication year, type of VG used, rationale for utilization, machine learning algorithms employed, frequently occurring keywords, top authors and universities, evaluation metrics, applied network properties, and brain disorders examined, such as Epilepsy, Alzheimer's disease, Autism, Alcoholism, Sleep disorders, Fatigue, Depression, and other related conditions. Moreover, there are recommendations for future advancements in research, which involve utilizing cutting-edge techniques like graph machine learning and deep learning. Additionally, the exploration of understudied medical conditions such as attention deficit hyperactivity disorder and Parkinson's disease is also suggested.

Machine learning estimated probability of relapse in early-stage non-small-cell lung cancer patients with aneuploidy imputation scores and knowledge graph embeddings

Motivation:Low-stage lung cancer is known to recur unpredictably, and patients receiving various treatment methods like radiation, chemotherapy, and immunotherapies have been seen to respond very differently. Identifying a priori if a patient is going to relapse or not could make a difference in terms of saving lives and personalized care offered. In this work, we provide an answer to the following research question: Is it possible to enhance the machine learning (ML) of the estimated probability of relapse in early-stage non-small-cell lung cancer (NSCLC) patients with aneuploidy imputation scores? Results:To predict recurrence in 1,348 early-stage (I–II) NSCLC patients, we train graph ML models utilizing the Spanish pulmonary cancer group knowledge graph enriched with triples from pathway imputation. ML models trained on Knowledge graph data enriched with triples from pathway score imputation present an 82% Precision and 91% Specificity in predicting relapse over 200 patients from a held-out test set. ML models trained using graphs data could prove useful supplemental tool in the TNM classification systems and improve a lung cancer patient’s prognosis.

Demonstration of SPARQL ML : An Interfacing Language for Supporting Graph Machine Learning for RDF Graphs

This demo paper presents KGNet, a graph machine learning-enabled RDF engine. KGNet integrates graph machine learning (GML) models with existing RDF engines as query operators to support node classification and link prediction tasks. For easy integration, KGNet extends the SPARQL language with user-defined predicates to support the GML operators. We refer to this extension as SPARQL ML query. Our SPARQL ML query optimizer is in charge of optimizing the selection of the near-optimal GML models. The development of KGNet poses research opportunities in various areas spanning KG management. In the paper, we demonstrate the ease of integration between the RDF engines and GML models through the SPARQL ML inference query language. We present several real use cases of different GML tasks on real KGs. Using KGNet, users do not need to learn a new scripting language or have a deep understanding of GML methods. The audience will experience KGNet with different KGs and GML models, as shown in our demo video and Colab notebook.

KG-Hub-building and exchanging biological knowledge graphs.

Knowledge graphs (KGs) are a powerful approach for integrating heterogeneous data and making inferences in biology and many other domains, but a coherent solution for constructing, exchanging, and facilitating the downstream use of KGs is lacking. Here we present KG-Hub, a platform that enables standardized construction, exchange, and reuse of KGs. Features include a simple, modular extract-transform-load pattern for producing graphs compliant with Biolink Model (a high-level data model for standardizing biological data), easy integration of any OBO (Open Biological and Biomedical Ontologies) ontology, cached downloads of upstream data sources, versioned and automatically updated builds with stable URLs, web-browsable storage of KG artifacts on cloud infrastructure, and easy reuse of transformed subgraphs across projects. Current KG-Hub projects span use cases including COVID-19 research, drug repurposing, microbial-environmental interactions, and rare disease research. KG-Hub is equipped with tooling to easily analyze and manipulate KGs. KG-Hub is also tightly integrated with graph machine learning (ML) tools which allow automated graph ML, including node embeddings and training of models for link prediction and node classification. https://kghub.org.

Wasserstein Graph Distance Based on L1–Approximated Tree Edit Distance between Weisfeiler–Lehman Subtrees

The Weisfeiler-Lehman (WL) test is a widely used algorithm in graph machine learning, including graph kernels, graph metrics, and graph neural networks. However, it focuses only on the consistency of the graph, which means that it is unable to detect slight structural differences. Consequently, this limits its ability to capture structural information, which also limits the performance of existing models that rely on the WL test. This limitation is particularly severe for traditional metrics defined by the WL test, which cannot precisely capture slight structural differences. In this paper, we propose a novel graph metric called the Wasserstein WL Subtree (WWLS) distance to address this problem. Our approach leverages the WL subtree as structural information for node neighborhoods and defines node metrics using the L1-approximated tree edit distance (L1-TED) between WL subtrees of nodes. Subsequently, we combine the Wasserstein distance and the L1-TED to define the WWLS distance, which can capture slight structural differences that may be difficult to detect using conventional metrics. We demonstrate that the proposed WWLS distance outperforms baselines in both metric validation and graph classification experiments.

Disease Prediction Using Graph Machine Learning Based on Electronic Health Data: A Review of Approaches and Trends

Graph machine-learning (ML) methods have recently attracted great attention and have made significant progress in graph applications. To date, most graph ML approaches have been evaluated on social networks, but they have not been comprehensively reviewed in the health informatics domain. Herein, a review of graph ML methods and their applications in the disease prediction domain based on electronic health data is presented in this study from two levels: node classification and link prediction. Commonly used graph ML approaches for these two levels are shallow embedding and graph neural networks (GNN). This study performs comprehensive research to identify articles that applied or proposed graph ML models on disease prediction using electronic health data. We considered journals and conferences from four digital library databases (i.e., PubMed, Scopus, ACM digital library, and IEEEXplore). Based on the identified articles, we review the present status of and trends in graph ML approaches for disease prediction using electronic health data. Even though GNN-based models have achieved outstanding results compared with the traditional ML methods in a wide range of disease prediction tasks, they still confront interpretability and dynamic graph challenges. Though the disease prediction field using ML techniques is still emerging, GNN-based models have the potential to be an excellent approach for disease prediction, which can be used in medical diagnosis, treatment, and the prognosis of diseases.

Private Graph Extraction via Feature Explanations

Privacy and interpretability are two important ingredients for achieving trustworthy machine learning. We study the interplay of these two aspects in graph machine learning through graph reconstruction attacks. The goal of the adversary here is to reconstruct the graph structure of the training data given access to model explanations. Based on the different kinds of auxiliary information available to the adversary, we propose several graph reconstruction attacks. We show that additional knowledge of post-hoc feature explanations substantially increases the success rate of these attacks. Further, we investigate in detail the differences between attack performance with respect to three different classes of explanation methods for graph neural networks: gradient-based, perturbation-based, and surrogate model-based methods. While gradient-based explanations reveal the most in terms of the graph structure, we find that these explanations do not always score high in utility. For the other two classes of explanations, privacy leakage increases with an increase in explanation utility. Finally, we propose a defense based on a randomized response mechanism for releasing the explanations, which substantially reduces the attack success rate. Our code is available at https://github.com/iyempissy/graph-stealing-attacks-with-explanation.

AI’s 10 to Watch, 2022

AI’s 10 to Watch, 2022

Exploring genetic influences on adverse outcome pathways using heuristic simulation and graph data science

Adverse outcome pathways provide a powerful tool for understanding the biological signaling cascades that lead to disease outcomes following toxicity. The framework outlines downstream responses known as key events, culminating in a clinically significant adverse outcome as a final result of the toxic exposure. Here we use the AOP framework combined with artificial intelligence methods to gain novel insights into genetic mechanisms that underlie toxicity-mediated adverse health outcomes. Specifically, we focus on liver cancer as a case study with diverse underlying mechanisms that are clinically significant. Our approach uses two complementary AI techniques: Generative modeling via automated machine learning and genetic algorithms, and graph machine learning. We used data from the US Environmental Protection Agency’s Adverse Outcome Pathway Database (AOP-DB; aopdb.epa.gov) and the UK Biobank’s genetic data repository. We use the AOP-DB to extract disease-specific AOPs and build graph neural networks used in our final analyses. We use the UK Biobank to retrieve real-world genotype and phenotype data, where genotypes are based on single nucleotide polymorphism data extracted from the AOP-DB, and phenotypes are case/control cohorts for the disease of interest (liver cancer) corresponding to those adverse outcome pathways. We also use propensity score matching to appropriately sample based on important covariates (demographics, comorbidities, and social deprivation indices) and to balance the case and control populations in our machine language training/testing datasets. Finally, we describe a novel putative risk factor for LC that depends on genetic variation in both the aryl-hydrocarbon receptor (AHR) and ATP binding cassette subfamily B member 11 (ABCB11) genes.

Improved environmental chemistry property prediction of molecules with graph machine learning

Rapid prediction of environmental chemistry properties is critical for the green and sustainable development of the chemical industry and drug discovery.

Detecting Arbitrage on Ethereum Through Feature Fusion and Positive-Unlabeled Learning

Due to the lack of supervision in the decentralized exchanges (DEXs), arbitrageurs can utilize information and take advantage of price gap to make profits over such platforms such as Ethereum blockchain. DEX arbitrage poses possibilities and opportunities for defrauding and can seriously impair the operation of the Ethereum ecosystem. It motivates this work to explore and characterize the unique features of arbitrage which differ from other frauds such as money laundering and Ponzi games for better detection. This work makes the first attempt for detecting arbitrage on Ethereum through feature fusion and positive-unlabeled learning (PU learning). We first conduct an in-depth analysis and exploit two-fold arbitrage features by fusion including: 1) statistical features that explicitly represent the node activity levels according to expert knowledge; and 2) structural features that implicitly encode the transactions information by graph machine learning. We then apply PU learning to generate negative instances for compensating the imbalanced arbitrage datasets. We evaluate our proposed method through extensive experiments over a real-world dataset and demonstrate that it can achieve 90% accuracy in detecting arbitrage activities on Ethereum.

BTG: A Bridge to Graph machine learning in telecommunications fraud detection

Telecommunications fraud runs rampant recently around the world. Therefore, how to effectively detect fraudsters has become an increasingly challenging problem. However, previous studies either assume that the samples are independent of each other and use non-graph methods, or use local subgraphs with good connectivity for graph-based anomaly detection. Few prior works have performed graph-based fraud detection on real-world Call Detail Records (CDR) meta data sets with sparse connectivity. To solve this problem, we propose an end-to-end telecommunications fraud detection framework named Bridge To Graph (BTG). BTG leverages the subscriber synergy behavior to reconstruct connectivity, which bridges the gap between sparse connectivity data and graph machine learning. Concretely, we extract multi-model features from meta data and perform Box–Cox transformation first. Then, aiming at the sparse connectivity of real-world CDR meta data, the graph is reconstructed through dimensionally selectable link prediction of node similarity. Finally, the reconstructed graph and node features are input into the graph machine learning module for node embedding representation learning and fraud node classification. Comprehensive experiments on the real-world telecommunications network CDR data set show that our proposed method outperforms the classic methods in many metrics. Beyond telecom fraud detection, our method can also be extended to anomaly detection scenarios with no graph or sparse connectivity graph.

M2STAN: Multi-modal multi-task spatiotemporal attention network for multi-location ultra-short-term wind power multi-step predictions

In recent years, wind power has continued to emerge as a key source of renewable energy. When large-scale wind farm clusters are connected to the grid for power generation, accurate multi-location ultra-short-term wind power predictions carry significant value in terms of ensuring the safety, stability, and economical operation of the power system. However, there are complex temporal and spatial correlations among multiple wind farms in multiple locations, which makes wind power predictions involving wind farm clusters very challenging. The development of artificial intelligence technology, especially graph machine learning, provides new approaches for modeling such spatiotemporal correlations. In addition, compared with single-step forecasting, multi-step forecasting can better reflect the general situation, and thus, it is more widely applicable in reality. To optimize multi-step wind power predictions in multiple locations, this report proposes a Multi-Modal Multi-Task Spatiotemporal Attention Network (M2STAN) model. The developed model employs a graph attention network and a bidirectional gated recurrent unit (Bi-GRU) to model the spatial and temporal dependence, respectively. In addition, the introduction of multi-modal and multi-task learning strategies improves the accuracy and computational efficiency of this predictive model. The results indicate that the proposed method is superior to existing methods, including support vector regression, Bi-GRU, multi-modal multi-task graph spatiotemporal networks, and graph convolutional deep learning architectures in terms of prediction performance.