Sequential Graph Research Articles

SSGCNet: A Sparse Spectra Graph Convolutional Network for Epileptic EEG Signal Classification.

In this article, we propose a sparse spectra graph convolutional network (SSGCNet) for epileptic electroencephalogram (EEG) signal classification. The goal is to develop a lightweighted deep learning model while retaining a high level of classification accuracy. To do so, we propose a weighted neighborhood field graph (WNFG) to represent EEG signals. The WNFG reduces redundant edges between graph nodes and has lower graph generation time and memory usage than the baseline solution. The sequential graph convolutional network is further developed from a WNFG by combining sparse weight pruning and the alternating direction method of multipliers (ADMM). Compared with the state-of-the-art method, our method has the same classification accuracy on the Bonn public dataset and the spikes and slow waves (SSW) clinical real dataset when the connection rate is ten times smaller.

GNN-Based Network Traffic Analysis for the Detection of Sequential Attacks in IoT

This research introduces a novel framework utilizing a sequential gated graph convolutional neural network (GGCN) designed specifically for botnet detection within Internet of Things (IoT) network environments. By capitalizing on the strengths of graph neural networks (GNNs) to represent network traffic as complex graph structures, our approach adeptly handles the temporal dynamics inherent to botnet attacks. Key to our approach is the development of a time-stamped multi-edge graph structure that uncovers subtle temporal patterns and hidden relationships in network flows, critical for recognizing botnet behaviors. Moreover, our sequential graph learning framework incorporates time-sequenced edges and multi-edged structures into a two-layered gated graph model, which is optimized with specialized message-passing layers and aggregation functions to address the challenges of time-series traffic data effectively. Our comparative analysis with the state of the art reveals that our sequential gated graph convolutional neural network achieves substantial improvements in detecting IoT botnets. The proposed GGCN model consistently outperforms the conventional model, achieving improvements in accuracy ranging from marginal to substantial—0.01% for BoT IoT and up to 25% for Mirai. Moreover, our empirical analysis underscores the GGCN’s enhanced capabilities, particularly in binary classification tasks, on imbalanced datasets. These findings highlight the model’s ability to effectively navigate and manage the varying complexity and characteristics of IoT security threats across different datasets.

Sequential graph convolutional network and DeepRNN based hybrid framework for epileptic seizure detection from EEG signal.

Automated epileptic seizure detection from ectroencephalogram (EEG) signals has attracted significant attention in the recent health informatics field. The serious brain condition known as epilepsy, which is characterized by recurrent seizures, is typically described as a sudden change in behavior caused by a momentary shift in the excessive electrical discharges in a group of brain cells, and EEG signal is primarily used in most cases to identify seizure to revitalize the close loop brain. The development of various deep learning (DL) algorithms for epileptic seizure diagnosis has been driven by the EEG's non-invasiveness and capacity to provide repetitive patterns of seizure-related electrophysiological information. Existing DL models, especially in clinical contexts where irregular and unordered structures of physiological recordings make it difficult to think of them as a matrix; this has been a key disadvantage to producing a consistent and appropriate diagnosis outcome due to EEG's low amplitude and nonstationary nature. Graph neural networks have drawn significant improvement by exploiting implicit information that is present in a brain anatomical system, whereas inter-acting nodes are connected by edges whose weights can be determined by either temporal associations or anatomical connections. Considering all these aspects, a novel hybrid framework is proposed for epileptic seizure detection by combined with a sequential graph convolutional network (SGCN) and deep recurrent neural network (DeepRNN). Here, DepRNN is developed by fusing a gated recurrent unit (GRU) with a traditional RNN; its key benefit is that it solves the vanishing gradient problem and achieve this hybrid framework greater sophistication. The line length feature, auto-covariance, auto-correlation, and periodogram are applied as a feature from the raw EEG signal and then grouped the resulting matrix into time-frequency domain as inputs for the SGCN to use for seizure classification. This model extracts both spatial and temporal information, resulting in improved accuracy, precision, and recall for seizure detection. Extensive experiments conducted on the CHB-MIT and TUH datasets showed that the SGCN-DeepRNN model outperforms other deep learning models for seizure detection, achieving an accuracy of 99.007%, with high sensitivity and specificity.

Multivariate Time-Series Representation Learning via Hierarchical Correlation Pooling Boosted Graph Neural Network

Representation learning is vital for the performance of Multivariate Time Series (MTS) related tasks. Given high-dimensional MTS data, researchers generally rely on deep learning (DL) models to learn representative features. Among them, the methods that can capture the spatial-temporal dependencies within MTS data generally achieve better performance. However, they ignored hierarchical relations and the dynamic property within MTS data, hindering their performance. To address these problems, we propose a Hierarchical Correlation Pooling boosted graph neural network (HierCorrPool) for MTS data representation learning. First, we propose a novel correlation pooling scheme to learn and capture hierarchical correlations between sensors. Meanwhile, a new assignment matrix is designed to ensure the effective learning of hierarchical correlations by adaptively combining both sensor features and correlations. Second, we learn sequential graphs to represent the dynamic property within MTS data, so that this property can be captured for learning decent representations. We conducted extensive experiments to test our model on various MTS tasks, including remaining useful life prediction, human activity recognition, and sleep stage classification. Experimental results have shown the effectiveness of our proposed model.

Dual Graph for Traffic Forecasting

Traffic forecasting is the task of predicting future traffic based on historical traffic data. It is challenging due to the complex spatial-temporal correlation on road networks. Most existing research works use sequential Graph Neural Networks (GNN) to model traffic inference. However, they only focus on nodes (intersections) or edges (road segments) traffic forecasting alone. As a result, they could hardly provide a complete description of future traffic on road networks. Actually, nodes and edges traffic are interrelated. Both of them are important for traffic safety and efficiency, and neither one is negligible. In this paper, we exploit nodes and edges information together and make traffic forecasting on nodes and edges simultaneously. We propose a novel dual graph framework, called DualGraph, to model the propagation behavior of traffic on road networks. Inside our framework, we develop a DualMap block to simulate the recursive interactions between nodes and edges. The interaction process is realized by a message passing mechanism of nearby information flow. We employ the Simulation of Urban MObility (SUMO) software to generate real-world traffic data to illustrate the effectiveness of our method. We also empirically evaluate our model on public traffic datasets. The results show that even for node or edge traffic forecasting alone, our model still outperforms compared ones, especially for long term (one hour) prediction.

PGRASS-Solver: A Graph Spectral Sparsification-Based Parallel Iterative Solver for Large-Scale Power Grid Analysis

With the increase in the complexity of VLSI chips, power grid analysis has become a challenging task, because linear equations of extremely large size need to be solved. Recent graph sparsification based solvers have shown promising performance for power grid analysis. However, existing graph sparsification algorithms are implemented in serial computing, while factorization and backward/forward substitution of the sparsifier’s Laplacian matrix are hard to parallelize. On the other hand, partition based iterative methods which are inherently parallel lack a direct control of the relative condition number of the preconditioner and consume more memory. In this work, we propose a novel parallel iterative solver called pGRASS-Solver. We first propose a practically-efficient parallel graph sparsification algorithm. Then domain decomposition method (DDM) is utilized to solve the sparsifier’s Laplacian matrix. To further improve the efficiency, a variant of DDM which employs partial Cholesky factorization and Schur complement matrix sparsification is proposed. Thus, we obtain an efficient parallel preconditioner, which not only leads to fast convergence but also enjoys ease of parallelization. Numerous experiments are conducted to illustrate the superior efficiency of the proposed pGRASS-Solver for large-scale power grid analysis, showing an average 6.8X speedup over a recent parallel iterative solver [1]. Moreover, it solves a real-world power grid matrix with 0.36 billion nodes and 8.7 billion nonzeros within 20 minutes on a 16-core machine, which is 10.9X faster than the best result of sequential graph sparsification based solver [2].

A Graph Neural Network (GNN)-Based Approach for Real-Time Estimation of Traffic Speed in Sustainable Smart Cities

Planning effective routes and monitoring vehicle traffic are essential for creating sustainable smart cities. Accurate speed prediction is a key component of these efforts, as it aids in alleviating traffic congestion. While their physical proximity is important, the interconnection of these road segments is what significantly contributes to the increase of traffic congestion. This interconnectedness poses a significant challenge to increasing prediction accuracy. To address this, we propose a novel approach based on Deep Graph Neural Networks (DGNNs), which represent the connectedness of road sections as a graph using Graph Neural Networks (GNNs). In this study, we implement the proposed approach, called STGGAN, for real-time traffic-speed estimation using two different actual traffic datasets: PeMSD4 and PeMSD8. The experimental results validate the prediction accuracy values of 96.67% and 98.75% for the PeMSD4 and PeMSD8 datasets, respectively. The computation of mean squared error (MSE), root mean squared error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE) also shows a progressive decline in these error values with increasing iteration count, demonstrating the success of the suggested technique. To confirm the feasibility, reliability, and applicability of the suggested STGGAN technique, we also perform a comparison analysis, including several statistical, analytical, and machine-learning- and deep-learning-based approaches. Our work contributes significantly to the field of traffic-speed estimation by considering the structure and characteristics of road networks through the implementation of DGNNs. The proposed technique trains a neural network to accurately predict traffic flow using data from the entire road network. Additionally, we extend DGNNs by incorporating Gated Graph Attention Network (GGAN) blocks, enabling the modification of the input and output to sequential graphs. The prediction accuracy of the proposed model based on DGNNs is thoroughly evaluated through extensive tests on real-world datasets, providing a comprehensive comparison with existing state-of-the-art models for traffic-flow forecasting.

Learning From the Future: Light Cone Modeling for Sequential Recommendation.

Modeling sequential behaviors is the core of sequential recommendation. As users visit items in chronological order, existing methods typically capture a user's present interests from his/her past-to-present behaviors, i.e., making recommendations with only the unidirectional past information. This article argues that future information is another critical factor for the sequential recommendation. However, directly learning from future-to-present behaviors inevitably causes data leakage. Here, it is pointed out that future information can be learned from users' collaborative behaviors. Toward this end, this article introduces sequential graphs to depict item transition relationships: where and how each item transits from and will transit to. This temporal evolution information is called the light cone in special and general relativity. Then, a bidirectional sequential graph convolutional network (BiSGCN) is proposed to learn item representations by encoding past and future light cones. Finally, a manifold translating embedding (MTE) method is proposed to model item transition patterns in Riemannian manifolds, which helps to better capture the geometric structures of light cones and item transition patterns. Experimental comparisons and ablation studies verify the outstanding performance of BiSGCN, the benefits of learning from the future, and the improvements of learning in Riemannian manifolds.

Sequential Graph Attention Learning for Predicting Dynamic Stock Trends (Student Abstract)

The stock market is characterized by a complex relationship between companies and the market. This study combines a sequential graph structure with attention mechanisms to learn global and local information within temporal time. Specifically, our proposed “GAT-AGNN” module compares model performance across multiple industries as well as within single industries. The results show that the proposed framework outperforms the state-of-the-art methods in predicting stock trends across multiple industries on Taiwan Stock datasets.

Big graphs

Big data is typically characterized with 4V's: Volume, Velocity, Variety and Veracity. When it comes to big graphs, these challenges become even more staggering. Each and every of the 4V's raises new questions, from theory to systems and practice. Is it possible to parallelize sequential graph algorithms and guarantee the correctness of the parallelized computations? Given a computational problem, does there exist a parallel algorithm for it that guarantees to reduce parallel runtime when more machines are used? Is there a systematic method for developing incremental algorithms with effectiveness guarantees in response to frequent updates? Is it possible to write queries across relational databases and semistructured graphs in SQL? Can we unify logic rules and machine learning, to improve the quality of graph-structured data, and deduce associations between entities? This paper aims to incite interest and curiosity in these topics. It raises as many questions as it answers.

Graph neural network-enabled manufacturing method classification from engineering drawings

While millions of scanned engineering drawings are received every year, the online quotation companies for custom mechanical parts have experienced a surging need to increase their processing efficiency by replacing the currently manual inspection process with an automatic system. Previous work has used traditional, and data-driven computer-vision approaches to detect symbols and text information from the drawings. However, there lacks a unified framework to determine the associated manufacturing processes as a critical step for realizing an automatic quoting system. In this paper, we propose a computational framework to automatically determine the manufacturing method appropriate to produce each queried engineering drawing, such as lathing, sheet metal bending, and milling. We present a data-driven framework that directly processes the raster images with a series of pre-processing steps and accurately determines the corresponding manufacturing methods for the queried part with a graph neural network. We propose a novel line tracing algorithm to transform complex geometries in engineering drawings into vectorized line segments with minimal information loss. To extract the shape contours, we use an efficient image segmentation network to remove the information tables, followed by a sequential graph neural network to detect and eliminate dimension lines. Finally, we propose a novel graph neural network with updated graph connections to hierarchically distill graph descriptors and classify the engineering drawing by its appropriate manufacturing method. Our framework has been validated on industry datasets. We verify that our framework can effectively classify the engineering drawings with an accuracy of 90.78%. We further assess our framework by comparing it against state-of-the-art image classification algorithms(+17.20%).

Sequential EHR-based dynamic graph network for multiple myeloma detection and feature interaction investigation.

e13591 Background: Electronic Health Records (EHR) provide the possibility of leveraging modern machine learning algorithms for disease detection prediction. Since a patient’s medical journey contains a complex pattern and it is difficult to identify interactive feature impacts, we propose to reconstruct patients’ attributes at different time points into dynamic graphs. Then we can use graph-based learning algorithms to enhance the predictive precision and identify leading feature interactions that reveal key services transitions. Methods: Sequential EHR like diagnosis (Dx), procedure (Px) and prescription (Rx) can be transformed into dynamic temporal graphs with nodes representing attributes, services, and medical status and edges representing patient-object interactions. Also, graph based sequential recommendation can use structured historical behavior trajectories as input, aiming to predict patient’s next behavior. Then, the model can investigate patients’ time-sensitive features interactively identifying a potential disease and then output a probability of getting that disease. Dynamic graphs can also help to track patient’s medical treatment/drug use service transitions. Results: We applied our method on a Newly Diagnosed Multiple Myeloma (NDMM) dataset consisting of 1.5 million candidates’ longitudinal medical claims from 2017 to 2020 with diagnosis rate around 0.6%. In this data set, patients’159 aggregated Dx, Rx and Px features, together with 30 demographic attributes are updated monthly. The model was trained on 4 monthly data cohorts from the 12/2017 to 03/2019 period and its performance was tested on 9 monthly cohorts from year 2020. The 1000 patients scored as having highest risk were evaluated for the predictive precision which was around 6.5% This precision compares most favorably relative to the reported 0.76% lifetime risk of getting multiple myeloma in US. We also captured the top, service transitions in NDMM patients having 30% more frequency than those in overall populations. Conclusions: The novel dynamic graph-based learning approach on a large-scale EHR dataset demonstrates the advancement of the disease detection methodology. While the outcome of this model significantly improves the accuracy of predictive precision, it also helps identify more meaningful key drivers attributing to disease. The incremental new patients identified through this novel model will provide a better basis for the evolving identification solution within Biopharma industry.

Powerful molecule generation with simple ConvNet.

Automated molecule generation is a crucial step in in-silico drug discovery. Graph-based generation algorithms have seen significant progress over recent years. However, they are often complex to implement, hard to train and can under-perform when generating long-sequence molecules. The development of a simple and powerful alternative can help improve practicality of automated drug discovery method. We proposed a ConvNet-based sequential graph generation algorithm. The molecular graph generation problem is reformulated as a sequence of simple classification tasks. At each step, a convolutional neural network operates on a sub-graph that is generated at previous step, and predicts/classifies an atom/bond adding action to populate the input sub-graph. The proposed model is pretrained by learning to sequentially reconstruct existing molecules. The pretrained model is abbreviated as SEEM (structural encoder for engineering molecules). It is then fine-tuned with reinforcement learning to generate molecules with improved properties. The fine-tuned model is named SEED (structural encoder for engineering drug-like-molecules). The proposed models have demonstrated competitive performance comparing to 16 state-of-the-art baselines on three benchmark datasets. Code is available at https://github.com/yuh8/SEEM and https://github.com/yuh8/SEED. QM9 dataset is availble at http://quantum-machine.org/datasets/, ZINC250k dataset is availble at https://raw.githubusercontent.com/aspuru-guzik-group/chemical_vae/master/models/zinc_properties/250k_rndm_zinc_drugs_clean_3.csv, and ChEMBL dataset is availble at https://www.ebi.ac.uk/chembl/. Supplementary data are available at Bioinformatics online.

HGNN: Hyperedge-based graph neural network for MOOC Course Recommendation

Previous studies on Course Recommendation (CR) mainly focus on investigating the sequential relationships among courses (RNN is applied) and fail to learn the similarity relationships among learners. Moreover, existing RNN-based methods can only model courses’ short-term sequential patterns due to the inherent shortcomings of RNNs. In light of the above issues, we develop a hyperedge-based graph neural network, namely HGNN, for CR. Specifically, (1) to model the relationships among learners, we treat learners (i.e., hyperedges) as the sets of courses in a hypergraph, and convert the task of learning learners’ representations to induce the embeddings for hyperedges, where a hyperedge-based graph attention network is further proposed. (2) To simultaneously consider courses’ long-term and short-term sequential relationships, we first construct a course sequential graph across learners, and learn courses’ representations via a modified graph attention network. Then, we feed the learned representations into a GRU-based sequence encoder to infer their short-term patterns, and deem the last hidden state as the learned sequence-level learner embedding. After that, we obtain the learners’ final representations by a product pooling operation to retain features from different latent spaces, and optimize a cross-entropy loss to make recommendations. To evaluate our proposed solution HGNN, we conduct extensive experiments on two real-world datasets, XuetangX and MovieLens. We conduct experiments on MovieLens to prove the extensibility of our solution on other collections. From the experimental results, we can find that HGNN evidently outperforms other recent CR methods on both datasets, achieving 11.96% on P@20, 16.01% on NDCG@20, and 27.62% on MRR@20 on XuetangX, demonstrating the effectiveness of studying CR in a hypergraph, and the importance of considering both long-term and short-term sequential patterns of courses.

Multimodal graph neural network for video procedural captioning

Video procedural captioning aims to generate detailed descriptive captions for all steps in a long instructional video. The peculiarity of this problem is the procedural dependency between the events to generate consistent captions among the video. However, existing video (dense) captioning methods only consider intra-event or sequential inter-event context and are hard to model the non-sequential context dependency between events. In this paper, inspired by the recent success of graph neural networks in capturing the relations for structured data, we propose a novel Multimodal Graph Neural Network (MGNN) for dense video procedural captioning in capturing the procedural structure between events. Specifically, we construct temporal sequential graph and semantic non-sequential graph for a multimodal heterogeneous graph. Moreover, we adopt the graph neural network to enhance the visual and text features, and fuse both features for further caption generation. Extensive experiments demonstrate the proposed MGNN is effective in generating coherent captions on both the Youcook2 and Activitynet Captions benchmark.

Deep Learning of Spatiotemporal Patterns for Urban Mobility Prediction Using Big Data

Timely and accurate prediction of human movement in urban areas offers instructive insights into transportation management, public safety, and location-based services, to name a few. Yet, modeling urban mobility is challenging and complex because of the spatiotemporal dynamics of movement behavior and the influence of exogenous factors such as weather, holidays, and local events. In this paper, we use bus transportation as a proxy to mine spatiotemporal travel patterns. We propose a deep-learning-based urban mobility prediction model that collectively forecasts passenger flows between pairs of city regions in an origin-destination (OD) matrix. We first process OD matrices in a convolutional neural network to capture spatial correlations. Intermediate results are reconstructed into three multivariate time series: hourly, daily, and weekly time series. Each time series is aggregated in a long short-term memory (LSTM) network with a novel attention mechanism to guide the aggregation. In addition, our model is context-aware by using contextual embeddings learned from exogenous factors. We dynamically merge results from LSTM components and context embeddings in a late fusion network to make a final prediction. The proposed model is implemented and evaluated using a large-scale transportation data set of more than 200 million bus trips with a suite of Big Data technologies developed for data processing. Through performance comparison, we show that our approach achieves sizable accuracy improvements in urban mobility prediction. Our work has major implications for efficient transportation system design and performance improvement. The proposed deep neural network structure is generally applicable for sequential graph data prediction.

Sequential graph collaborative filtering

Understanding collaborative and sequential information plays a major role in next-item recommendation. Although previous models have achieved considerable results, their capabilities have been limited by the basic paradigm of characterizing each user’s profile with only his/her sequential behaviors. This paper considers collaborative information over users to solve the problem. Specifically, we first reorganize all interactions as a multiplex directed heterogeneous graph (named sequential graph) to depict each user’s sequential behaviors and all users’ collaborative interactions. Second, we propose a generic vectorization algorithm to address the challenge of multiplex edges. The algorithm can implement arbitrary graph attention networks on complex graphs without graph simplification. Finally, we propose a novel sequential graph attention network (SGAT) that controls information to attentively propagate through directed heterogeneous edges. Based on the sequential graph and the vectorization algorithm, SGAT infers each user’s dynamic preference by capturing high-order sequential information among items and exploiting both latitudinally and longitudinally collaborative information over users. Extensive experiments and ablation studies on various datasets demonstrate the superiority of SGAT. The sequential graph provides an alternative way to organize users’ sequential behaviors and the vectorization algorithm can promote the development of graph mining by retaining topological structures of complex graphs.

The Moran Spectrum as a Geoinformatic Tupu: implications for the First Law of Geography

ABSTRACT Geoinformatic Tupu, or Geoinformatic graph spectrum, is a theoretical as well as a technical framework for generalizing geographic knowledge and solving real world problems. Geoinformatic Tupu is a promising platform for capitalizing on the technical advances of Geographic Information Systems, and to integrate the Chinese traditional way of thinking with modern information technology. It has been one of the major research topics in the Chinese GIScience community in recent decades, with an evolving epistemological development. A core objective of Geoinformatic Tupu is to recover and represent geographic principles with the Tupu approach, which is adopted in this paper to formulate the First Law of Geography (FLG) [i.e. the law of spatial autocorrelation] as the Moran Spectrum – a combination of sequential diagrams, graphs, and numeric components. Using the Moran Spectrum as a conduit, we present the theory of Moran Eigenvector Spatial Filtering (MESF), a distinct branch of spatial statistics that has demonstrable advantages in statistical modelling and machine learning, but has yet to be widely disseminated due to its conceptual and computational complexity. This paper demonstrates the effectiveness of the Tupu approach in enriching the representation of the FLG as well as deepening its applications. It also suggests inclusion of the Moran Spectrum as a core component in Geoinformatic Tupu.

A Sequential Graph Neural Network for Short Text Classification

Short text classification is an important problem of natural language processing (NLP), and graph neural networks (GNNs) have been successfully used to solve different NLP problems. However, few studies employ GNN for short text classification, and most of the existing graph-based models ignore sequential information (e.g., word orders) in each document. In this work, we propose an improved sequence-based feature propagation scheme, which fully uses word representation and document-level word interaction and overcomes the limitations of textual features in short texts. On this basis, we utilize this propagation scheme to construct a lightweight model, sequential GNN (SGNN), and its extended model, ESGNN. Specifically, we build individual graphs for each document in the short text corpus based on word co-occurrence and use a bidirectional long short-term memory network (Bi-LSTM) to extract the sequential features of each document; therefore, word nodes in the document graph retain contextual information. Furthermore, two different simplified graph convolutional networks (GCNs) are used to learn word representations based on their local structures. Finally, word nodes combined with sequential information and local information are incorporated as the document representation. Extensive experiments on seven benchmark datasets demonstrate the effectiveness of our method.

Strategy on Disaster Recovery Management based on Graph Theory Concepts

The COVID-19 pandemic has asserted major baseline facts from disaster anthropology during the last three decades. Resilience could be based on the solution to the question: "What is the maximum amount of destruction, if any, that the graph (a network) can sustain while ensuring that at least one of each technology type remains and that the remaining induced subgraph is properly colored?" The concept of a graph's Chromatic Core Subgraph is a solution to the stated problem. In this paper, the pandemic graphs and certain sequential graphs are developed. For these graphs, the Chromatic core subgraph is obtained. The results of the pandemic graphs' Chromatic core subgraph are used to develop a disaster recovery strategy for the COVID-19 pandemic.