Temporal Neural Network Research Articles

A deep learning approach to secure vehicle to road side unit communications in intelligent transportation system

Vehicle and road side unit communications are crucial to the information network in Intelligent Transportation System (ITS). There are two fundamental problems with the current communication security solutions: (1) encryption technology alone is not sufficient to verify the authenticity of the messages transmitted from vehicles to road side units, (2) existing solutions fail to build a complete framework in the way that they are case-dependent and apply to limited scenarios. This paper first presents a cloud-vehicle-road architecture that explains the precise message contents as well as the message generation and transmission process. A binary classification model is deployed to assess uploaded traffic-related messages to improve traffic efficiency based on cross-sectional data. A novel graph temporal neural network with attention is designed for misbehavior detection of vehicles based on time-sequential data. According to simulation results, the system's overall performance can be effectively improved regarding security and availability.

An integrated spatial-temporal neural network for proactive throughput bottleneck prediction in high-variety shops with complex job routings

ABSTRACT Throughput bottlenecks remain a main concern for managers in practice since they affect production output and throughput times. A large literature on bottleneck detection and prediction consequently emerged. Bottleneck prediction is specifically important in context where bottlenecks shift since it allows for counteracting the potential impact. The literature on throughput bottleneck prediction largely focusses on temporal aspects. Although this reflects the relation among stations if the routing of jobs is fairly directed, the relative station position constantly changes for more complex routings. A station maybe upstream, downstream or have no relation to another station dependent on the mix of jobs currently on the shop floor. In these high-variety contexts, both temporal and spatial features should be considered when predicting bottlenecks. In response, this study proposes a new neural network model that systematically connects multiple independent workstations into a system by extracting the spatial features between workstations. The new approach is different from traditional stacking mechanisms applied in the literature, and it allows for a better integration of spatial and temporal neural networks. Experimental results show that the proposed model outperforms alternative models and provides good prediction performance. Findings have important implications for research and practice.

A temporal and spatial prediction method for urban pipeline network based on deep learning

Water pipeline is one of the important components of urban infrastructure and plays a key role in residential life. An accurate pressure prediction could help improve the resilience of the system. In recent years, some studies have found that the massive pressure monitoring data have complex temporal and spatial correlations. It issues some new challenges to traditional prediction models. In this study, a pressure prediction method based on spatial–temporal neural network (PP-STNN) is proposed. Before the modeling, the pipeline network is mapped into a graph. In the method, Graph Convolutional Network (GCN) is used to capture the spatial correlation of the pipeline network and Gated Recurrent Unit (GRU) is used to capture the temporal correlation. The proposed method is evaluated using real-world dataset and compared with some benchmark methods. The results show that the proposed method could reach the highest accuracy among all methods for different prediction steps. Moreover, the comparison indicates that simultaneously considering temporal and spatial correlation can contribute to the prediction, especially for multiple steps prediction. Compared with GRU for 3-step, 12-step, and 24-step prediction, the proposed method can improve the Root Mean Square Error (RMSE) by about 19%, 8%, and 8%, respectively. Using deep learning methods, this study can improve the accuracy of the pressure prediction, thus increasing the resilience of the cities and promoting safety and sustainable development in the area.

Probabilistic air quality forecasting using deep learning spatial–temporal neural network

Probabilistic air quality forecasting using deep learning spatial–temporal neural network

A Comprehensive Spatio-Temporal Model for Subway Passenger Flow Prediction

Accurate subway passenger flow prediction is crucial to operation management and line scheduling. It can also promote the construction of intelligent transportation systems (ITS). Due to the complex spatial features and time-varying traffic patterns of subway networks, the prediction task is still challenging. Thus, a hybrid neural network model, GCTN (graph convolutional and comprehensive temporal neural network), is proposed. The model combines the Transformer network and long short-term memory (LSTM) network to capture the global and local temporal dependency. Besides, it uses a graph convolutional network (GCN) to capture the spatial features of the subway network. For the sake of the stability and accuracy for long-term passenger flow prediction, we enhance the influence of the station itself and the global station and combine the convolutional neural networks (CNN) and Transformer. The model is verified by the passenger flow data of the Shanghai Subway. Compared with some typical data-driven methods, the results show that the proposed model improves the prediction accuracy in different time intervals and exhibits superiority in prediction stability and robustness. Besides, the model has a better performance in the peak value and the period when passenger flow changes quickly.

MST-RNN: A Multi-Dimension Spatiotemporal Recurrent Neural Networks for Recommending the Next Point of Interest

With the increasing popularity of location-aware Internet-of-Vehicle services, the next-Point-of-Interest (POI) recommendation has gained significant research interest, predicting where drivers will go next from their sequential movements. Many researchers have focused on this problem and proposed solutions. Machine learning-based methods (matrix factorization, Markov chain, and factorizing personalized Markov chain) focus on a POI sequential transition. However, they do not recommend the user’s position for the next few hours. Neural network-based methods can model user mobility behavior by learning the representations of the sequence data in the high-dimensional space. However, they just consider the influence from the spatiotemporal dimension and ignore many important influences, such as duration time at a POI (Point of Interest) and the semantic tags of the POIs. In this paper, we propose a novel method called multi-dimension spatial–temporal recurrent neural networks (MST-RNN), which extends the ST-RNN and exploits the duration time dimension and semantic tag dimension of POIs in each layer of neural networks. Experiments on real-world vehicle movement data show that the proposed MST-RNN is effective and clearly outperforms the state-of-the-art methods.

GTAD: Graph and Temporal Neural Network for Multivariate Time Series Anomaly Detection

The rapid development of smart factories, combined with the increasing complexity of production equipment, has resulted in a large number of multivariate time series that can be recorded using sensors during the manufacturing process. The anomalous patterns of industrial production may be hidden by these time series. Previous LSTM-based and machine-learning-based approaches have made fruitful progress in anomaly detection. However, these multivariate time series anomaly detection algorithms do not take into account the correlation and time dependence between the sequences. In this study, we proposed a new algorithm framework, namely, graph attention network and temporal convolutional network for multivariate time series anomaly detection (GTAD), to address this problem. Specifically, we first utilized temporal convolutional networks, including causal convolution and dilated convolution, to capture temporal dependencies, and then used graph neural networks to obtain correlations between sensors. Finally, we conducted sufficient experiments on three public benchmark datasets, and the results showed that the proposed method outperformed the baseline method, achieving detection results with F1 scores higher than 95% on all datasets.

Wi-Exercise: An Indoor Human Movement Detection Method Based on Bidirectional LSTM Attention

Human motion recognition has become a hot topic in the field of human-computer interaction. Due to the traditional method of detecting human movement in indoor environments, there are still problems such as high hardware costs, influenced by the environment and the need for experimenters to be equipped with relevant equipment. With the large-scale deployment of wireless devices and the establishment of wireless network infrastructure, indoor personnel movement recognition systems based on wireless signals are gradually proposed. In this paper, we propose a device-independent motion-detection method Wi-exercise based on temporal recurrent neural networks. The acquired channel state information (CSI) is preprocessed by wavelet transform, and the output CSI matrix is filtered by the principal component analysis (PCA) to extract the feature values. The wavelet function is processed to further remove the noise also to make the action data more distinctive, and then PCA is used to track the entire CSI time series to improve and maintain its dimensional energy. The preprocessed data are subjected to feature extraction to obtain CSI of human motion and establish feature sequences, and then the action model is trained using the bidirectional long short-term memory attention (ABLSTM) mechanism to finally realize human motion recognition. Finally, the robustness of Wi-exercise is tested experimentally, and its performance is compared with that of existing recognition methods. The experimental results show that the average recognition rate of the method is 94.43% in the complex indoor environment, and the method is better than the traditional classifier method for action recognition, with lower and higher recognition accuracy.

A new approach for product evaluation based on integration of EEG and eye-tracking

Exploiting biometric measures, especially neurophysiological data of evaluator for product evaluation is advantageous at avoiding bias and subjectivity in expert scoring process. This paper proposes an approach that integrates electroencephalograph (EEG) and eye-tracking (ET) data in a new way to derive multi-faceted supportive information for product evaluation. Firstly, emotion recognition from EEG signals of evaluator is carried out with a spatial–temporal neural network. Then, based on correlations between emotions and preferential judgement, general customer preference toward product design scheme is inferred from emotions by fuzzy system. Finally, general preference is integrated with ET data at application-level to quantify fine-grained customer preferences toward design modules and visual attractiveness. This approach is verified with a case study which evaluates six designs of frontal area of automotive interior, and valuable supportive information for design decision-making is yielded. Also, comprehensive analysis is conducted and the results verify the effectiveness of proposed approach.

Gas leakage detection using spatial and temporal neural network model

Natural gas leakage can impose significant danger on a facility and its surrounding communities. Methods for early detection and diagnosis of such leakages have been developed and widely used for gas pipelines and storage tanks. Most techniques include inspection of sensor-aided mathematical models. Application of machine learning techniques to gas leakage detection has been rarely explored. In the present work, convolutional network (to model spatial likelihood of leak) is combined with bi-directional long short-term memory layer network, or BiLSTM (to model temporal dependence of leak likelihood) to perform leak detection and diagnosis. The developed model was trained and tested using sequence of concentration profiles generated using open-source simulated data. The model learned successfully to predict gas leakage and classify its size. The study also explores the flexibility of this network to perform quick detection and diagnose with the limited data. While the networks did not require parameter adjustments to achieve high prediction accuracy, further optimization is possible through data selection and pre-processing. The model needs to be further tested for wide range of leak scenarios. At its present condition, the combined application of convolutional network and BiLSTM shows promising results for early and accurate leak detection in natural gas facilities. Experimental results are needed to confirm the effectiveness of the model and data uncertainty.

A Subgrid-Oriented Privacy-Preserving Microservice Framework Based on Deep Neural Network for False Data Injection Attack Detection in Smart Grids

False data injection attacks (FDIAs) have recently become a major threat to smart grids. Most of the existing FDIA detection methods have focused on modeling the temporal relationship of time-series measurement data but have paid less attention to the spatial relationship between bus/line measurement data and have failed to consider the relationship between subgrids. To address these issues, in this article, we propose a subgrid-oriented microservice framework by integrating a well-designed spatial–temporal neural network for FDIA detection in ac-model power systems. First, a well-designed neural network is developed to model the spatial–temporal relationship of bus/line measurements for subgrids. A microservice-based supervising network is then proposed for integrating the representation features obtained from subgrids for the collaborative detection of FDIAs. To evaluate the proposed framework, three types of FDIA datasets are generated based on a public benchmark power grid. Case studies on the FDIA datasets show that our method outperforms state-of-the-art methods for FDIA detection in these datasets.

Attention-based spatial–temporal neural network for accurate phase recognition in minimally invasive surgery: feasibility and efficiency verification

Abstract Laparoscopic surgery, as a representative minimally invasive surgery (MIS), is an active research area of clinical practice. Automatic surgical phase recognition of laparoscopic videos is a vital task with the potential to improve surgeons’ efficiency and has gradually become an integral part of computer-assisted intervention systems in MIS. However, the performance of most methods currently employed for surgical phase recognition is deteriorated by optimization difficulties and inefficient computation, which hinders their large-scale practical implementation. This study proposes an efficient and novel surgical phase recognition method using an attention-based spatial–temporal neural network consisting of a spatial model and a temporal model for accurate recognition by end-to-end training. The former subtly incorporates the attention mechanism to enhance the model’s ability to focus on the key regions in video frames and efficiently capture more informative visual features. In the temporal model, we employ independently recurrent long short-term memory (IndyLSTM) and non-local block to extract long-term temporal information of video frames. We evaluated the performance of our method on the publicly available Cholec80 dataset. Our attention-based spatial–temporal neural network purely produces the phase predictions without any post-processing strategies, achieving excellent recognition performance and outperforming other state-of-the-art phase recognition methods.

Age group classification and gender recognition from speech with temporal convolutional neural networks

This paper analyses the performance of different types of Deep Neural Networks to jointly estimate age and identify gender from speech, to be applied in Interactive Voice Response systems available in call centres. Deep Neural Networks are used, because they have recently demonstrated discriminative and representation capabilities in a wide range of applications, including speech processing problems based on feature extraction and selection. Networks with different sizes are analysed to obtain information on how performance depends on the network architecture and the number of free parameters. The speech corpus used for the experiments is Mozilla’s Common Voice dataset, an open and crowdsourced speech corpus. The results are really good for gender classification, independently of the type of neural network, but improve with the network size. Regarding the classification by age groups, the combination of convolutional neural networks and temporal neural networks seems to be the best option among the analysed, and again, the larger the size of the network, the better the results. The results are promising for use in IVR systems, with the best systems achieving a gender identification error of less than 2% and a classification error by age group of less than 20%.

Genes Expression Classification Through Histone Modification Using Temporal Neural Network

Background: Genes expression is high dimensional data, so it is very difficult to classify high dimensional data through traditional machine learning approaches. In this work we have proposed a model based on combined approach of Convolutional Neural Network and Recurrent Neural Network, both belong to deep learning model. The prediction has shown improved result than other machine learning algorithms. Expressions are generated through histone modification. Methods: To improve the accuracy deep learning model is proposed i.e. based on Convolutional and Recurrent neural network. This proposed model uses filter, causal convolutional layers and Residual Block for predictions. Results: In this work we have implemented the machine learning algorithms and deep learning algorithms like Logistic Regression, SVM, CNN, Deep Chrome and the proposed Temporal Neural Network. The performance is measured on the basis of parameters like accuracy, precision and AUC on the training and testing set. Conclusion: The proposed Temporal Neural Network model has shown better performance than other machine learning and deep learning algorithms. Due to this proposed deep learning algorithm can be successfully applied on the genes expression dataset.

Nozzle Thermal Estimation for Fused Filament Fabricating 3D Printer Using Temporal Convolutional Neural Networks

A preventive maintenance embedded for the fused deposition modeling (FDM) printing technique is proposed. A monitoring and control integrated system is developed to reduce the risk of having thermal degradation on the fabricated products and prevent printing failure; nozzle clogging. As for the monitoring program, the proposed temporal neural network with a two-stage sliding window strategy (TCN-TS-SW) is utilized to accurately provide the predicted thermal values of the nozzle tip. These estimated thermal values are utilized to be the stimulus of the control system that performs countermeasures to prevent the anomaly that is bound to happen. The performance of the proposed TCN-TS-SW is presented in three case studies. The first scenario is when the proposed system outperforms the other existing machine learning algorithms namely multi-look back LSTM, GRU, LSTM, and the generic TCN architecture in terms of obtaining the highest training accuracy and lowest training loss. TCN-TS-SW also outperformed the mentioned algorithms in terms of prediction accuracy measured by the performance metrics like RMSE, MAE, and R2 scores. In the second case, the effect of varying the window length and the changing length of the forecasting horizon. This experiment reveals the optimized parameters for the network to produce an accurate nozzle thermal estimation.

A new reconstruction algorithm based on temporal neural network and its application in power quality disturbance data

The traditional reconstruction algorithms based on p-norm, limited by their reconstruction model and data processing mode, are prone to reconstruction failure or long reconstruction time. In order to break through the limitations, this paper proposes a reconstruction algorithm based on the temporal neural network (TCN). A new reconstruction model based on TCN is first established, which does not need sparse representation and has large-scale parallel processing. Next, a TCN with a fully connected layer and symmetrical zero-padding operation is designed to meet the reconstruction requirements, including non-causality and length-inconsistency. Moreover, the proposed algorithm is constructed and applied to power quality disturbance (PQD) data. Experimental results show that the proposed algorithm can implement the reconstruction task, demonstrating better reconstruction accuracy and less reconstruction time than OMP, ROMP, CoSaMP, and SP. Therefore, the proposed algorithm is more attractive when dictionary design is complicated, or real-time reconstruction is required.

Analysis of the Impact of Medical Features and Risk Prediction of Acute Kidney Injury for Critical Patients Using Temporal Electronic Health Record Data With Attention-Based Neural Network.

Acute kidney injury (AKI) is one of the most severe consequences of kidney injury, and it will also cause or aggravate the complications by the fast decline of kidney excretory function. Accurate AKI prediction, including the AKI case, AKI stage, and AKI onset time interval, can provide adequate support for effective interventions. Besides, discovering how the medical features affect the AKI result may also provide supporting information for disease treatment. An attention-based temporal neural network approach was employed in this study for AKI prediction and for the analysis of the impact of medical features from temporal electronic health record (EHR) data of patients before AKI diagnosis. We used the publicly available dataset provided by the Medical Information Mart for Intensive Care (MIMIC) for model training, validation, and testing, and then the model was applied in clinical practice. The improvement of AKI case prediction is around 5% AUC (area under the receiver operating characteristic curve), and the AUC value of AKI stage prediction on AKI stage 3 is over 82%. We also analyzed the data by two steps: the associations between the medical features and the AKI case (positive or inverse) and the extent of the impact of medical features on AKI prediction result. It shows that features, such as lactate, glucose, creatinine, blood urea nitrogen (BUN), prothrombin time (PT), and partial thromboplastin time (PTT), are positively associated with the AKI case, while there are inverse associations between the AKI case and features such as platelet, hemoglobin, hematocrit, urine, and international normalized ratio (INR). The laboratory test features such as urine, glucose, creatinine, sodium, and blood urea nitrogen and the medication features such as nonsteroidal anti-inflammatory drugs, agents acting on the renin–angiotensin system, and lipid-lowering medication were detected to have higher weights than other features in the proposed model, which may imply that these features have a great impact on the AKI case.

Gesture Based Real-time Indian Sign Language Interpreter

Hand gesture is one of the methods used in sign language for non-verbal communication. It is most commonly used by hearing & speech impaired people who have hearing or speech problems to communicate among themselves or with normal people. Developing sign language applications for hearing impaired people can be very important, as hearing & speech impaired people will be able to communicate easily with even those who don’t understand sign language. This project aims at taking the basic step in bridging the communication gap between normal people, deaf and dumb people using sign language. The main focus of this work is to create a vision based system to identify sign language gestures from the video sequences. The reason for choosing a system based on vision relates to the fact that it provides a simpler and more intuitive way of communication between a human and a computer. Video sequences contain both temporal as well as spatial features. In this project, two different models are used to train the temporal as well as spatial features. To train the model on the spatial features of the video sequences a deep Convolutional Neural Network. Convolutional Neural Network was trained on the frames obtained from the video sequences of train data. To train the model on the temporal features Recurrent Neural Network is used. The Trained Convolutional Neural Network model was used to make predictions for individual frames to obtain a sequence of predictions. Now this sequence of prediction outputs was given to the Recurrent Neural Network to train on the temporal features. Collectively both the trained models i.e. Convolutional Neural Network and Recurrent Neural Network will produce the text output of the respective gesture.

Day-ahead hourly photovoltaic power forecasting using attention-based CNN-LSTM neural network embedded with multiple relevant and target variables prediction pattern

Accurate forecasting of photovoltaic power plays a pivotal role in the integration, operation, and scheduling of smart grid systems. Notably, volatility and intermittence of solar energy are the primary constraints influencing the accuracy of photovoltaic power prediction. This work proposes, an attention-based long-term and short-term temporal neural network prediction model (ALSM) assembled using the convolutional neural network (CNN), long short-term memory neural network (LSTM), and attention mechanism under the multiple relevant and target variables prediction pattern (MRTPP). This is geared towards capturing the short-term and long-term temporal modes and achieving the day-ahead hourly photovoltaic power forecasting. The proposed method is verified by the historical data of the photovoltaic system downloaded from the DKASC website. Consequently, the results indicate that the forecasting accuracy using the MRTPP pattern is better than those common input-output prediction patterns. Moreover, the proposed ALSM model under the MRTPP pattern demonstrates more superiority compared to a few PV power forecasting methods including the statistical methods as well as artificial intelligence methods. Subsequently, different important parameters affecting the accuracy of forecasting range of the model are analyzed, and suggestions on memory lengths corresponding to the divergent prediction range are provided.

The Pulse of Urban Transport: Exploring the Co-evolving Pattern for Spatio-temporal Forecasting

Transportation demand forecasting is a topic of large practical value. However, the model that fits the demand of one transportation by only considering the historical data of its own could be vulnerable since random fluctuations could easily impact the modeling. On the other hand, common factors like time and region attribute, drive the evolution demand of different transportation, leading to a co-evolving intrinsic property between different kinds of transportation. In this work, we focus on exploring the co-evolution between different modes of transport, e.g., taxi demand and shared-bike demand. Two significant challenges impede the discovery of the co-evolving pattern: (1) diversity of the co-evolving correlation, which varies from region to region and time to time. (2) Multi-modal data fusion. Taxi demand and shared-bike demand are time-series data, which have different representations with the external factors. Moreover, the distribution of taxi demand and bike demand are not identical. To overcome these challenges, we propose a novel method, known as co-evolving spatial temporal neural network (CEST). CEST learns a multi-view demand representation for each mode of transport, extracts the co-evolving pattern, then predicts the demand for the target transportation based on multi-scale representation, which includes fine-scale demand information and coarse-scale pattern information. We conduct extensive experiments to validate the superiority of our model over the state-of-art models.