Graph Learning Research Articles

Self-supervised dynamic stochastic graph network for spatio-temporal wind speed forecasting

Spatio-temporal Forecasting has been implemented in diverse fields, such as energy, traffic, and weather. As one typical paradigm of intelligent power systems, spatio-temporal wind speed forecasting has attracted increasing attention. Recently, some graph-based methods have revealed the great potential in the spatio-temporal wind speed forecasting task. Those methods are usually built on a static graph which is incapable of discovering the dynamic characteristics and the uncertainty of temporal wind patterns. To this end, we propose a novel Dynamic Stochastic Graph Network (DSGN) for spatio-temporal wind speed forecasting. Specifically, we characterize each wind turbine of regional power grids with a dynamic stochastic distribution to incorporate the uncertainty into networks. Then Wasserstein distance is adopted to build the stochastic graph, which can effectively represent the spatial dependencies between wind turbines. Additionally, we devise a reconstruction task to preserve the geographical distribution information and regularize the learning of the stochastic graph. To further exploit global spatial dependencies and enhance graph modeling, we augment the stochastic graph view by diffusion operations and integrate a self-supervised learning framework, serving as an auxiliary task, to maximize mutual information between the wind latent states from different graph views. Empirical experiments on four real-world wind speed datasets demonstrate the superiority of DSGN over some state-of-the-art methods. Some ablation studies further validate the effectiveness of the stochastic graph modeling and the self-supervised framework.

Offloading Strategy Based on Graph Neural Reinforcement Learning in Mobile Edge Computing

In the mobile edge computing (MEC) architecture, base stations with computational capabilities are subject to service coverage limitations, and the mobility of devices leads to dynamic changes in their connections, directly impacting the offloading decisions of agents. The connections between base stations and mobile devices, as well as the connections between base stations themselves, are abstracted into an MEC structural diagram due to the difficulty of deep reinforcement learning (DRL) in capturing the complex relationships between nodes and their multi-order neighboring nodes in the graph; decisions solely generated by DRL have limitations. To address this issue, this study proposes a hierarchical mechanism strategy based on Graph Neural Reinforcement Learning (M-GNRL) under multiple constraints. Specifically, the MEC structural graph constructed with the current device as an observation point aggregates to learn node features, thus comprehensively considering the contextual information of nodes, and the learned graph information serves as the environment for deep reinforcement learning, effectively integrating a graph neural network (GNN) with DRL. In the M-GNRL strategy, edge features from GNN are introduced into the architecture of the DRL network to enhance the accuracy of agents’ decision-making. Additionally, this study proposes an updated algorithm to obtain graph data that change with observation points. Comparative experiments demonstrate that the M-GNRL algorithm outperforms other baseline algorithms in terms of system cost and convergence performance.

On the effectiveness of hybrid pooling in mixup-based graph learning for language processing

Graph neural network (GNN)-based graph learning has been popular in natural language and programming language processing, particularly in text and source code classification. Typically, GNNs are constructed by incorporating alternating layers which learn transformations of graph node features, along with graph pooling layers that use graph pooling operators (e.g., Max-pooling) to effectively reduce the number of nodes while preserving the semantic information of the graph. Recently, to enhance GNNs in graph learning tasks, Manifold-Mixup, a data augmentation technique that produces synthetic graph data by linearly mixing a pair of graph data and their labels, has been widely adopted. However, the performance of Manifold-Mixup can be highly affected by graph pooling operators, and there have not been many studies that are dedicated to uncovering such affection. To bridge this gap, we take an early step to explore how graph pooling operators affect the performance of Mixup-based graph learning. To that end, we conduct a comprehensive empirical study by applying Manifold-Mixup to a formal characterization of graph pooling based on 11 graph pooling operations (9 hybrid pooling operators, 2 non-hybrid pooling operators). The experimental results on both natural language datasets (Gossipcop, Politifact) and programming language datasets (JAVA250, Python800) demonstrate that hybrid pooling operators are more effective for Manifold-Mixup than the standard Max-pooling and the state-of-the-art graph multiset transformer (GMT) pooling, in terms of producing more accurate and robust GNN models.Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Lightweight Federated Graph Learning for Accelerating Classification Inference in UAV-Assisted MEC Systems

Lightweight Federated Graph Learning for Accelerating Classification Inference in UAV-Assisted MEC Systems

Multi-temporal heterogeneous graph learning with pattern-aware attention for industrial chain risk detection

Multi-temporal heterogeneous graph learning with pattern-aware attention for industrial chain risk detection

Dual space-based fuzzy graphs and orthogonal basis clustering for unsupervised feature selection

Unsupervised feature selection (UFS) takes an important position because gaining the class labels is laborious or even impossible. In the domain of UFS, clustering is a major means to exploit label information. The existing methods either could not model a clear clustering structure or could not utilize the local structure of data. Consequently, a new clustering method in combination with graph learning is proposed in this work. Specifically, for clustering, orthogonal basis clustering is introduced, where orthogonal constraints are imposed on the cluster center matrix and the clustering matrix. The clustering indicator matrix is also imposed by a non-negative constraint. A clearer clustering structure and more independent clustering centers are obtained through these constraints. For local preservation, given that traditional graphs for keeping the local manifold are faced with the problem of imbalanced neighbors, the fuzzy graph is introduced to acquire a robust structure, which is applied to both data space and feature space. The topological structure in these spaces can be well maintained. For the choice of salient features, ℓ2,0-norm regularization is imposed on the projection matrix. The object function is solved alternately. Then, a feature selection algorithm is designed. Experiments are designed and performed on nine real-world data sets. The results attest to the effectiveness of the proposed algorithm compared with other relevant algorithms.

Hybrid multimodal fusion for graph learning in disease prediction

Graph neural networks (GNNs) have gained significant attention in disease prediction where the latent embeddings of patients are modeled as nodes and the similarities among patients are represented through edges. The graph structure, which determines how information is aggregated and propagated, plays a crucial role in graph learning. Recent approaches typically create graphs based on patients' latent embeddings, which may not accurately reflect their real-world closeness. Our analysis reveals that raw data, such as demographic attributes and laboratory results, offers a wealth of information for assessing patient similarities and can serve as a compensatory measure for graphs constructed exclusively from latent embeddings. In this study, we first construct adaptive graphs from both latent representations and raw data respectively, and then merge these graphs via weighted summation. Given that the graphs may contain extraneous and noisy connections, we apply degree-sensitive edge pruning and kNN sparsification techniques to selectively sparsify and prune these edges. We conducted intensive experiments on two diagnostic prediction datasets, and the results demonstrate that our proposed method surpasses current state-of-the-art techniques.

SpaceRL-KG: Searching paths automatically combining embedding-based rewards with Reinforcement Learning in Knowledge Graphs

Knowledge Graph Completion seeks to find missing elements in a Knowledge Graph, usually edges representing some relation between two concepts. One possible way to do this is to find paths between two nodes that indicate the presence of a missing edge. This can be achieved through Reinforcement Learning, by training an agent that learns how to navigate through the graph, starting at a node with a missing edge and identifying what edge among the available ones at each step is more promising in order to reach the target of the missing edge. While some approaches have been proposed to this effect, their reward functions only take into account whether the target node was reached or not, and only apply a single Reinforcement Learning algorithm. In this regard, we present a new family of reward functions based on node embeddings and structural distance, leveraging additional information related to semantic similarity and removing the need to reach the target node to obtain a measure of the benefits of an action. Our experimental results show that these functions, as well as the novel use of more modern Reinforcement Learning techniques, are able to obtain better results than the existing strategies in the literature.

Protection of Guizhou Miao batik culture based on knowledge graph and deep learning

In the globalization trend, China’s cultural heritage is in danger of gradually disappearing. The protection and inheritance of these precious cultural resources has become a critical task. This paper focuses on the Miao batik culture in Guizhou Province, China, and explores the application of knowledge graphs, natural language processing, and deep learning techniques in the promotion and protection of batik culture. We propose a dual-channel mechanism that integrates semantic and visual information, aiming to connect batik pattern features with cultural connotations. First, we use natural language processing techniques to automatically extract batik-related entities and relationships from the literature, and construct and visualize a structured batik pattern knowledge graph. Based on this knowledge graph, users can textually search and understand the images, meanings, taboos, and other cultural information of specific patterns. Second, for the batik pattern classification, we propose an improved ResNet34 model. By embedding average pooling and convolutional operations into the residual blocks and introducing long-range residual connections, the classification performance is enhanced. By inputting pattern images into this model, their categories can be accurately identified, and then the underlying cultural connotations can be understood. Experimental results show that our model outperforms other mainstream models in evaluation metrics such as accuracy, precision, recall, and F1-score, achieving 94.46%, 94.47%, 93.62%, and 93.8%, respectively. This research provides new ideas for the digital protection of batik culture and demonstrates the great potential of artificial intelligence technology in cultural heritage protection.

Dissecting tumor microenvironment from spatially resolved transcriptomics data by heterogeneous graph learning

Spatially resolved transcriptomics (SRT) has enabled precise dissection of tumor-microenvironment (TME) by analyzing its intracellular molecular networks and intercellular cell-cell communication (CCC). However, lacking computational exploration of complicated relations between cells, genes, and histological regions, severely limits the ability to interpret the complex structure of TME. Here, we introduce stKeep, a heterogeneous graph (HG) learning method that integrates multimodality and gene-gene interactions, in unraveling TME from SRT data. stKeep leverages HG to learn both cell-modules and gene-modules by incorporating features of diverse nodes including genes, cells, and histological regions, allows for identifying finer cell-states within TME and cell-state-specific gene-gene relations, respectively. Furthermore, stKeep employs HG to infer CCC for each cell, while ensuring that learned CCC patterns are comparable across different cell-states through contrastive learning. In various cancer samples, stKeep outperforms other tools in dissecting TME such as detecting bi-potent basal populations, neoplastic myoepithelial cells, and metastatic cells distributed within the tumor or leading-edge regions. Notably, stKeep identifies key transcription factors, ligands, and receptors relevant to disease progression, which are further validated by the functional and survival analysis of independent clinical data, thereby highlighting its clinical prognostic and immunotherapy applications.

Intelligent substation virtual circuit verification method combining knowledge graph and deep learning

Intelligent substation virtual circuit verification method combining knowledge graph and deep learning

Enhancing Communication in CPS Using Graph-Based Reply Relationship Identification in Multi-Party Conversations

To enhance communication and collaborative work efficiency in cyber–physical systems (CPSs) within the Industry 4.0 environment, this study investigates a graph-based machine learning approach aimed at optimizing information interaction during multi-party conversations. Devices within CPSs must efficiently exchange information in real time to synchronize operations and responses. This research treats these interactions as intricate graph structures and uses graph learning techniques to accurately identify communication links and dependencies among devices. This improvement leads to more accurate decision-making and smoother operations. Our methodology involves a real-time analysis of structural patterns and node attributes within conversations, improving information flow and comprehension. The empirical findings demonstrate that this approach significantly enhances production efficiency, system adaptability, and minimizes delays attributed to communication misunderstandings. Our method can effectively identify the communication relationships between devices, significantly improving the efficiency and accuracy of information transmission. This improved communication capability leads to an enhanced production efficiency of the entire system.

Digital twin-driven discriminative graph learning networks for cross-domain bearing fault recognition

Rolling bearing fault diagnosis holds paramount significance in industrial field, as it is pivotal for ensuring the reliability, safety, and economic viability of mechanical systems. Most of traditional data-driven fault diagnosis methods rely on the availability of pre-collected data sets (containing various failure modes) as training data. Regrettably, such datasets may not always be easy to collect, particularly in critical industrial settings. Consequently, the applicability of data-driven fault diagnostic methods across diverse applications is impeded. In response to this challenge, this research introduces a digital twin-empowered discriminative graph learning network (DT-DGLN) for bearing fault diagnostics with unlabeled industrial data. The primary improvements of this study are twofold: (1) The creation of an elaborate and meticulous digital twin model for bearings. This model encompasses multi-scale Kinetic simulations of the operational parameters of the bearing and relies exclusively on the architectural attributes of the bearing and magnitude of the failure to deduce the vibratory system’s reaction. (2) The establishment of a domain adaption-based framework based on discriminative graph learning strategy that facilitates the transfer of known intelligence from simulated signals to real signals. This framework enables efficient fault recognition in cases where knowledge is limited. Rigorous experiments have been conducted to validate the effectiveness of the proposed approach.

GH-Twin: Graph Learning Empowered Hierarchical Digital Twin for Optimizing Self-Healing Networks

Communication networks are witnessing a fast evolution towards Beyond 5G (B5G), bringing unprecedented complexities and challenges for optimizing networks in guaranteeing self-healing abilities and maintaining quality of services (QoS). To this end, this study presents a Graph Learning-driven Hierarchical Digital Twin framework, called GH-Twin, to build a reliable virtual replica of network components and their communications between different layers, leading to inclusive network representation. The proposed framework introduces graph cross-learning (GCL) distributed across different participants to devise competent predictive modelling of network performance collaboratively and preemptively recognize abnormalities in network settings. To preserve local privacy, differential privacy is applied by injecting some Gaussian into the parameters of local GCL before sharing it with the global coordinator. Proof of concept simulations has demonstrated that GH-Twin can precisely predict flow-level QoS and recognize anomalous links and nodes under different network topologies.

GNN-Based Network Traffic Analysis for the Detection of Sequential Attacks in IoT

This research introduces a novel framework utilizing a sequential gated graph convolutional neural network (GGCN) designed specifically for botnet detection within Internet of Things (IoT) network environments. By capitalizing on the strengths of graph neural networks (GNNs) to represent network traffic as complex graph structures, our approach adeptly handles the temporal dynamics inherent to botnet attacks. Key to our approach is the development of a time-stamped multi-edge graph structure that uncovers subtle temporal patterns and hidden relationships in network flows, critical for recognizing botnet behaviors. Moreover, our sequential graph learning framework incorporates time-sequenced edges and multi-edged structures into a two-layered gated graph model, which is optimized with specialized message-passing layers and aggregation functions to address the challenges of time-series traffic data effectively. Our comparative analysis with the state of the art reveals that our sequential gated graph convolutional neural network achieves substantial improvements in detecting IoT botnets. The proposed GGCN model consistently outperforms the conventional model, achieving improvements in accuracy ranging from marginal to substantial—0.01% for BoT IoT and up to 25% for Mirai. Moreover, our empirical analysis underscores the GGCN’s enhanced capabilities, particularly in binary classification tasks, on imbalanced datasets. These findings highlight the model’s ability to effectively navigate and manage the varying complexity and characteristics of IoT security threats across different datasets.

Non-negative consistency affinity graph learning for unsupervised feature selection and clustering

Feature selection plays a crucial role in data mining and pattern recognition tasks. This paper proposes an efficient method for robust unsupervised feature selection, called joint local preserving and low-rank representation with nonnegative and symmetric constraint (JLPLRNS). This approach utilizes an indicator matrix instead of the row sparsity of the projection matrix to directly select some significant features from the original data and adaptively preserves the local geometric structure of original data into the low-dimensional embedded feature subspace via learned indicator matrix. By simultaneously imposing nonnegative symmetric and low-rank constraints on the representation coefficient matrix, it cannot only make this matrix discriminative, sparse and weight consistency for each pair of data, but also uncover the global structure of original data. These effectively will improve clustering performance. An algorithm based on the augmented Lagrange multiplier method with an alternating direction strategy is designed to resolve this model. Experimental results on various real datasets show that the proposed method can effectively identify some important features in data and outperforms many state-of-the-art unsupervised feature selection methods in terms of clustering performance.

Discriminative Subspace Learning With Adaptive Graph Regularization

Abstract Many subspace learning methods based on low-rank representation employ the nearest neighborhood graph to preserve the local structure. However, in these methods, the nearest neighborhood graph is a binary matrix, which fails to precisely capture the similarity between distinct samples. Additionally, these methods need to manually select an appropriate number of neighbors, and they cannot adaptively update the similarity graph during projection learning. To tackle these issues, we introduce Discriminative Subspace Learning with Adaptive Graph Regularization (DSL_AGR), an innovative unsupervised subspace learning method that integrates low-rank representation, adaptive graph learning and nonnegative representation into a framework. DSL_AGR introduces a low-rank constraint to capture the global structure of the data and extract more discriminative information. Furthermore, a novel graph regularization term in DSL_AGR is guided by nonnegative representations to enhance the capability of capturing the local structure. Since closed-form solutions for the proposed method are not easily obtained, we devise an iterative optimization algorithm for its resolution. We also analyze the computational complexity and convergence of DSL_AGR. Extensive experiments on real-world datasets demonstrate that the proposed method achieves competitive performance compared with other state-of-the-art methods.

GGT-SNN: Graph learning and Gaussian prior integrated spiking graph neural network for event-driven tactile object recognition

Spiking neural networks possess asynchronous, discrete, and sparse characteristics that enable direct processing of event-based tactile sensor data and more efficient transmission of information. These networks have been widely applied in the tactile perception field. However, due to the sparsity of tactile data, these models are prone to generalization issues. Additionally, existing methods for tactile graph construction are constrained when spatially connecting high-dimensional tactile data, making it challenging for the model to fuse and process such information effectively. To address these challenges, we propose the GGT-SNN, which introduces a parameter estimation method with Gaussian priors to alleviate generalization problems caused by sparse tactile data. Furthermore, we design M-tree and Z-tree tactile graph construction methods to compensate for deficiencies of other approaches when handling high-dimensional tactile spatial connections. This approach enhances the model’s ability to efficiently process high-dimensional event-driven tactile information. We introduce an approximate LIF neuron activation function to enable backpropagation within the model and comprehensively compare and evaluate its performance against different approximate functions. The experimental results demonstrate that our proposed method performs significantly better than SOTA methods, exhibiting a 10.83 % improvement over TactileSGNet on the EvTouch-Containers dataset and a 2.92 % improvement over TactileSGNet on the EvTouch-Objects dataset.

BP-MoE: Behavior Pattern-aware Mixture-of-Experts for Temporal Graph Representation Learning

Temporal graph representation learning aims to develop low-dimensional embeddings for nodes in a graph that can effectively capture their structural and temporal properties. Prior approaches primarily focus on devising sophisticated neural architectures to encode the historical interactions of nodes, enabling capturing the specific behavior patterns. However, such methods often overlook the fact that different nodes in a graph may exhibit distinct evolutionary preferences, resulting in a failure to adequately model the variations in the node evolution behavior. To address these issues, we propose a novel temporal graph learning framework called Behavior Pattern-aware Mixture-of-Experts (BP-MoE), which leverages multiple expert encoder networks with diverse architectures to comprehensively capture the behavior patterns for nodes. In detail, we first design a multi-perspective graph encoder that employs experts based on the gated recurrent units, graph attention units, and multilayer perceptron units to encode a node’s long-term memory, high-order neighborhood features, and short-term behavior preference, respectively. Subsequently, to achieve the personalized evolution preference learning for each individual node, we incorporate a behavior pattern-aware MoE mechanism to activate the pattern-specific experts for each node via a gating network. Finally, to optimize the training process of expert networks, we introduce two auxiliary losses that prevent the imbalanced training across experts. Extensive experiments conducted on multiple benchmark datasets verify the superiority of our approach over the state-of-the-art baselines. Specifically, BP-MoE consistently outperforms the baseline models for the link prediction and node classification tasks under various experimental settings.

Real-time outage management in active distribution networks using reinforcement learning over graphs

Self-healing smart grids are characterized by fast-acting, intelligent control mechanisms that minimize power disruptions during outages. The corrective actions adopted during outages in power distribution networks include reconfiguration through switching control and emergency load shedding. The conventional decision-making models for outage mitigation are, however, not suitable for smart grids due to their slow response and computational inefficiency. Here, we present a graph reinforcement learning model for outage management in the distribution network to enhance its resilience. The distinctive characteristic of our approach is that it explicitly accounts for the underlying network topology and its variations with switching control, while also capturing the complex interdependencies between state variables (along nodes and edges) by modeling the task as a graph learning problem. Our model learns the optimal control policy for power restoration using a Capsule-based graph neural network. We validate our model on three test networks, namely the 13, 34, and 123-bus modified IEEE networks where it is shown to achieve near-optimal, real-time performance. The resilience improvement of our model in terms of loss of energy is 607.45 kWs and 596.52 kWs for 13 and 34 buses, respectively. Our model also demonstrates generalizability across a broad range of outage scenarios.