Graph Convolutional Long Short-Term Memory Research Articles

A3GC-IP: Attention-oriented adjacency adaptive recurrent graph convolutions for human pose estimation from sparse inertial measurements

Conventional methods for human pose estimation either require a high degree of instrumentation, by relying on many inertial measurement units (IMUs), or constraint the recording space, by relying on extrinsic cameras. These deficits are tackled through the approach of human pose estimation from sparse IMU data. We define attention-oriented adjacency adaptive graph convolutional long-short term memory networks (A3GC-LSTM), to tackle human pose estimation based on six IMUs, through incorporating the human body graph structure directly into the network. The A3GC-LSTM combines both spatial and temporal dependency in a single network operation, more memory efficiently than previous approaches. The recurrent graph learning on arbitrarily long sequences is made possible by equipping graph convolutions with adjacency adaptivity, which eliminates the problem of information loss in deep or recurrent graph networks, while it also allows for learning unknown dependencies between the human body joints. To further boost accuracy, a spatial attention formalism is incorporated into the recurrent LSTM cell. With our presented approach, we are able to utilize the inherent graph nature of the human body, and thus can outperform the state-of-the-art for human pose estimation from sparse IMU data.

Attack Graph Model for Cyber-Physical Power Systems Using Hybrid Deep Learning

Electrical power grids are vulnerable to cyber attacks, as seen in Ukraine in 2015 and 2016. However, existing attack detection methods are limited. Most of them are based on power system measurement anomalies that occur when an attack is successfully executed at the later stages of the cyber kill chain. In contrast, the attacks on the Ukrainian power grid show the importance of system-wide, early-stage attack detection through communication-based anomalies. Therefore, in this paper, we propose a novel method for online cyber attack situational awareness that enhances the power grid resilience. It supports power system operators in the identification and localization of active attack locations in Operational Technology (OT) networks in near real-time. The proposed method employs a hybrid deep learning model of Graph Convolutional Long Short-Term Memory (GC-LSTM) and a deep convolutional network for time series classification-based anomaly detection. It is implemented as a combination of software defined networking, anomaly detection in communication throughput, and a novel attack graph model. Results indicate that the proposed method can identify active attack locations, e.g., within substations, control center, and wide area network, with an accuracy above 96%. Hence, it outperforms existing state-of-the-art deep learning-based time series classification methods.

EEG-Based Emotion Recognition Using Spatial-Temporal Graph Convolutional LSTM With Attention Mechanism.

The dynamic uncertain relationship among each brain region is a necessary factor that limits EEG-based emotion recognition. It is a thought-provoking problem to availably employ time-varying spatial and temporal characteristics from multi-channel electroencephalogram (EEG) signals. Although deep learning has made remarkable achievements in emotion recognition, the biological topological information among brain regions does not fully exploit, which is vital for EEG-based emotion recognition. In response to this problem, we design a hybrid model called ST-GCLSTM, which comprises a spatial-graph convolutional network (SGCN) module and an attention-enhanced bi-directional Long Short-Term Memory (LSTM) module. The main advantage of ST-GCLSTM is that it can consider the biological topology information of each brain region to extract representative spatial-temporal features from multiple EEG channels. Specifically, we construct two layers SGCN by introducing adjacency matrices to adaptively learn the intrinsic connection among different EEG channels. Moreover, an attention-enhanced mechanism is placed into a bi-directional LSTM module to extract the crucial spatial-temporal features from sequential EEG data, and then these features serve as the input layer of the classifier to learn discriminative emotion-related features. Extensive experiments on the DEAP, SEED, and SEED-IV datasets demonstrate the effectiveness of the proposed ST-GCLSTM model, revealing that our model had an absolute performance improvement over state-of-the-art strategies.

Spatio-Temporal Graph Neural Networks for Multi-Site PV Power Forecasting

Accurate forecasting of solar power generation with fine temporal and spatial resolution is vital for the operation of the power grid. However, state-of-the-art approaches that combine machine learning with numerical weather predictions (NWP) have coarse resolution. In this paper, we take a graph signal processing perspective and model multi-site photovoltaic (PV) production time series as signals on a graph to capture their spatio-temporal dependencies and achieve higher spatial and temporal resolution forecasts. We present two novel graph neural network models for deterministic multi-site PV forecasting dubbed the graph-convolutional long short term memory (GCLSTM) and the graph-convolutional transformer (GCTrafo) models. These methods rely solely on production data and exploit the intuition that PV systems provide a dense network of virtual weather stations. The proposed methods were evaluated in two data sets for an entire year: 1) production data from 304 real PV systems, and 2) simulated production of 1000 PV systems, both distributed over Switzerland. The proposed models outperform state-of-the-art multi-site forecasting methods for prediction horizons of six hours ahead. Furthermore, the proposed models outperform state-of-the-art single-site methods with NWP as inputs on horizons up to four hours ahead.

Driver activity recognition using spatial‐temporal graph convolutional LSTM networks with attention mechanism

Driver activity engagement while driving plays a vital role that leads to negative outcomes of driving safety. To reduce traffic accidents and ensure driving safety, real-time driver activity recognition architecture is proposed in this study. Specifically, a total of eight kinds of common driving-related activities are identified, which include the normal driving, left or right checking, texting, answering the phone, using media, drinking, and picking up objects. Raw experiment videos are collected via onboard monocular cameras, which are used for the upper body skeleton information extraction of the driver. Then, the graph convolutional networks (GCN) are constructed for spatial structure feature reasoning in a single frame, which is consecutively followed by long short-term memory (LSTM) networks for temporal motion feature learning within the sequence. Moreover, the attention mechanism is further utilised to emphasise the keyframes to select discriminative sequential information. Finally, a large-scale driver activity dataset, consisting of both naturalistic driving data and simulative driving data, is collected for model training and evaluations. Experimental results show that the general recall ratio of those eight driving-related activities reaches up to 88.8% and the real-time recognition efficiency can reach up to 24 fps, which would satisfy the real-time requirements of engineering applications.

Attention-Based Multilevel Co-Occurrence Graph Convolutional LSTM for 3-D Action Recognition

Action recognition is essential for many human-centered applications in the Internet of Things (IoT). Especially, in the Internet of Medical Things (IoMT), action recognition shows great importance in surgical assistance, patient monitoring, etc. Recently, 3-D skeleton sequence-based action recognition draws broad attention. It is a challenging task that needs effective modeling on intraframe skeleton representations and interframe temporal dynamics. Standard long short-term memory (LSTM)-based models are widely used for sequence modeling due to its long-term memory, yet they are unable to fully model the relationship between different body joints or persons to extract crucial co-occurrence features from different levels. To handle this shortcoming, we propose an attention-based multilevel co-occurrence graph convolutional LSTM (AMCGC-LSTM). By integrating graph convolutional networks (GCNs) into LSTM, the proposed model is capable of leveraging body structural information from skeletons and strengthening the multilevel co-occurrence (MC) feature learning. Specifically, we first design the spatial attention module for feature enhancement of key joints from skeleton inputs. Second, we design MC memory units coupled with GCN to automatically model the spatial relationship between joints, and simultaneously capture the co-occurrence features from different joints, persons, and frames. Finally, we construct aggregated features of MCs (AFMCs) from MC memory units to better represent the intraframe action context encoding, and leverage a concurrent LSTM (Co-LSTM) to further model their temporal dynamics for action recognition. Our model significantly outperforms mainstream methods on NTU RGB+D 60/120 data set, mutual action subset of NTU RGB+D 60/120 data set, and Northewestern-UCLA data set.

Mobile Demand Forecasting via Deep Graph-Sequence Spatiotemporal Modeling in Cellular Networks

The demand forecasting plays a crucial role in the predictive physical and virtualized network management in cellular networks, which can effectively reduce both the capital and operational expenditures by fully exploiting the network infrastructure. In this paper, we study the per-cell demand forecasting in cellular networks. The success of demand forecasting relies on the effective modeling of both the spatial and temporal aspects of the per-cell demand time series. However, the main challenge of the spatial relevancy modeling in the per-cell demand forecasting is the irregular spatial distribution of cells in a network, where applying grid-based models (e.g., convolutional neural networks) would lead to degradation of spatial granularity. In this paper, we propose to model the spatial relevancy among cells by a dependency graph based on spatial distances among cells without the loss of spatial granularity. Such spatial distance-based graph modeling is confirmed by the spatiotemporal analysis via semivariogram, which suggests that the relevancy between any two cells declines as their spatial distance increases. Hence, the graph convolutional networks and long short-term memory (LSTM) from deep learning are employed to model the spatial and temporal aspects, respectively. In addition, the deep graph-sequence model, graph convolutional LSTM, is further employed to simultaneously characterize both the spatial and temporal aspects of mobile demand forecasting. Experiments demonstrate that our proposed graph-sequence demand forecasting model could achieve a superior forecasting performance compared with the other two proposed models as well as the traditional auto regression integrated moving average time series model.