Time Series Neural Network Research Articles

A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection.

Time series are the primary data type used to record dynamic system measurements and generated in great volume by both physical sensors and online processes (virtual sensors). Time series analytics is therefore crucial to unlocking the wealth of information implicit in available data. With the recent advancements in graph neural networks (GNNs), there has been a surge in GNN-based approaches for time series analysis. These approaches can explicitly model inter-temporal and inter-variable relationships, which traditional and other deep neural network-based methods struggle to do. In this survey, we provide a comprehensive review of graph neural networks for time series analysis (GNN4TS), encompassing four fundamental dimensions: forecasting, classification, anomaly detection, and imputation. Our aim is to guide designers and practitioners to understand, build applications, and advance research of GNN4TS. At first, we provide a comprehensive task-oriented taxonomy of GNN4TS. Then, we present and discuss representative research works and introduce mainstream applications of GNN4TS. A comprehensive discussion of potential future research directions completes the survey. This survey, for the first time, brings together a vast array of knowledge on GNN-based time series research, highlighting foundations, practical applications, and opportunities of graph neural networks for time series analysis.

A Physics- and Data-Driven Study on the Ground Effect on the Propulsive Performance of Tandem Flapping Wings

In this paper, we present a physics- and data-driven study on the ground effect on the propulsive performance of tandem flapping wings. With numerical simulations, the impact of the ground effect on the aerodynamic force, energy consumption, and efficiency is analyzed, revealing a unique coupling effect between the ground effect and the wing–wing interference. It is found that, for smaller phase differences between the front and rear wings, the thrust is higher, and the boosting effect due to the ground on the rear wing (maximum of 12.33%) is lower than that on a single wing (maximum of 43.83%) For a larger phase difference, a lower thrust is observed, and it is also found that the boosting effect on the rear wing is above that on a single wing. Further, based on the bidirectional gate recurrent units (BiGRUs) time-series neural network, a surrogate model is further developed to predict the unsteady aerodynamic characteristics of tandem flapping wings under the ground effect. The surrogate model exhibits high predictive precision for aerodynamic forces, energy consumption, and efficiency. On the test set, the relative errors of the time-averaged values range from −4% to 2%, while the root mean squared error of the transient values is less than 0.1. Meanwhile, it should be pointed out that the established surrogate model also demonstrates strong generalization capability. The findings contribute to a comprehensive understanding of the ground effect mechanism and provide valuable insights for the aerodynamic design of tandem flapping-wing air vehicles operating near the ground.

A prototype early warning system for diarrhoeal disease to combat health threats of climate change in the asia-pacific region

Abstract Ongoing climate variability and change are increasing the burden of diarrhoeal disease worldwide. Meaningful early warning systems with adequate lead times (weeks to months) are needed to guide public health decision–making and enhance community resilience against health threats posed by climate change. Toward this goal, we trained various machine-learning models to predict diarrhoeal disease rates in Nepal (2002–2014), Taiwan (2008–2019), and Vietnam (2000–2015) using temperature, precipitation, previous disease rates, and El Niño Southern Oscillation phases. We also compared the performance of shallow time-series neural network (NN), Random Forest Regressor, artificial nn, gradient boosting regressor, and long short-term memory–based methods for their effectiveness in predicting diarrhoeal disease burden across multiple countries. We evaluated model performance using a test dataset and assessed the accuracy of predicted diarrhoeal disease incidence rates for the last year of available data in each district. Our results suggest that even in the absence of the most recent disease surveillance data, a likely scenario in most low- and middle-income countries, our NN-based early warning system using historical data performs reasonably well. However, future studies are needed to perform prospective evaluations of such early warning systems in real-world settings.

Enhancing public opinion monitoring for social hot events with a time series neural network-based logic map

A method for constructing a commercial map of social hotspots based on time-series neural networks is proposed. It explores map construction through three stages: atomic event graph building, fusion, and public opinion analysis. The neural network captures event dynamics and public opinion evolution. An unsupervised learning approach trains the model to explore deep event connections. A time map reflects event trends and opinion changes. A sentence-level event extraction model provides rich information. An empirical mode decomposition method analyzes IMF data to understand intrinsic patterns. Time-series neural networks classify and reconstruct event graphs, constructing a knowledge graph. Experiments show this method reduces storage costs, simplifies prediction, and exhibits robustness. It offers new insights for commercial mapping and contributes to public opinion monitoring.

Oriented to a multi-learning mode: Establishing trend-fuzzy-granule-based LSTM neural networks for time series forecasting

In the construction of information granule based neural networks for time series multi-step forecasting, the existing works tend to focus on consecutive-learning mode while rarely explore multi-learning mode. In fact, only under the multi-learning mode can diversified associations among data collected over time granules be well learned. Also, the existing works exhibit limited time interpretability. Here the problem centers around how to endow information granule based neural networks with a multi-learning mode to learn diversified associations simultaneously, and well-articulated trend semantics. To solve these problems, the first originality of this paper stems from a scale equalization method for multilinear-trend fuzzy information granules to track complex trend changes of data in a more accurate and explainable manner from both global and local views. Furthermore, an adaptive rather than empirical or traversal method, which is trend-driven in nature, is tailored for mining diversified associations. The resulting model can give forecasts in the form of granules as well as numerical values, being interpretable and accurate in the sense that: (a) its inputs and output are granules which come with well-defined trend semantics under a customary time concept, and (b) a clump of data is considered in a concise granule whilst roles of diversified associations are ware of during forecasting, making the model less prone to cumulative errors. Appealing experimental results corroborate the effectiveness of the proposed model.

Short-term load forecasting: cascade intuitionistic fuzzy time series—univariate and bivariate models

AbstractShort-term load forecasting (STLF) is essential for developing reliable and sustainable economic and operational strategies for power systems. This study presents a forecasting model combining cascade forward neural network (CFNN) and intuitionistic fuzzy time series (IFTS) models for STLF. The proposed cascading intuitionistic fuzzy time series forecasting model (C-IFTS-FM) offers the advantage of CFNN using the links of both linear and nonlinear to model fuzzy relations between inputs and outputs. Moreover, it offers a more reliable and realistic approach to uncertainty, taking notice of also the degree of hesitation. C-IFTS-FM works in univariate structure when it uses only hourly load data, and in bivariate structure when it uses hourly load data and hourly temperature time series together. The conversion of time series into IFTS is realized with intuitionistic fuzzy c-means (IFCM). Thus, the membership and non-membership values for each data point are produced. In modelling process, membership and non-membership values, in addition to actual lagged observations, are used as input of the CFNNs. The effectiveness of C-IFTS-FM on test sets for both structures was discussed comparatively via different error criteria, in addition, the convergence time was examined, and also the fit of forecasts and observations was presented with different illustrations. Among different combinations of hyperparameters, in the best case, approximately 86% better accuracy is achieved than the best of the others, while even in the case of the worst of hyperparameters combination, the accuracy was improved by over 20% for the PSJM data sets. For HEXING, CHENGNAN, and EUNITE data sets, these progress rates reached approximately 90% in the best case.

A method for phase estimation of X-ray pulsar signals: Combining a transformer network structure and a two-dimensional profile map

The high-precision phase estimation of X-ray pulsar signals is crucial for X-ray pulsar navigation. This paper introduces a novel feature modeling method named Two-Dimensional Profile Map (2D-PM), which enhances the time-domain information of the period axis using the epoch folding algorithm. Compared to traditional profiles, the newly proposed 2D-PM model exhibits richer information content, and greater robustness in terms of folding period precision. Furthermore, a new time-series neural network structure, inspired by groundbreaking advancements in the field of Natural Language Processing (NLP), is developed using the TensorFlow framework. This structure is designed to learn the proposed features and achieve precise phase estimation. The paper also outlines training schemes and debugging strategies for hyperparameters. To test the method, sample sets were created using simulated data and Rossi X-ray Timing Explorer (RXTE) observational data. These samples are specifically designed to enhance feature consistency and the generalization capability of the network. The effectiveness of the proposed method is demonstrated through a comparative analysis with traditional cross-correlation algorithms. The results confirm a high degree of alignment between the network outputs and the feature model, highlighting the significant potential of the proposed method for application in X-ray pulsar navigation.

Insights into Flexible Bioinspired Fins for Unmanned Underwater Vehicle Systems through Deep Learning.

The last few decades have led to the rise of research focused on propulsion and control systems for bio-inspired unmanned underwater vehicles (UUVs), which provide more maneuverable alternatives to traditional UUVs in underwater missions. Recent work has explored the use of time-series neural network surrogate models to predict thrust and power from vehicle design and fin kinematics. We expand upon this work, creating new forward neural network models that encapsulate the effects of the material stiffness of the fin on its kinematic performance, thrust, and power, and are able to interpolate to the full spectrum of kinematic gaits for each material. Notably, we demonstrate through testing of holdout data that our developed forward models capture the thrust and power associated with each set of parameters with high resolution, enabling highly accurate predictions of previously unseen gaits and thrust and FOM gains through proper materials and kinematics selection. As propulsive efficiency is of utmost importance for flapping-fin UUVs in order to extend their range and endurance for essential operations, a non-dimensional figure of merit (FOM), derived from measures of propulsive efficiency, is used to evaluate different fin designs and kinematics and allow for comparison with other bio-inspired platforms. We use the developed FOM to analyze optimal gaits and compare the performance between different fin materials. The forward model demonstrates the ability to capture the highest thrust and FOM with good precision, which enables us to improve thrust generation by 83.89% and efficiency by 137.58% with proper fin stiffness and kinematics selection, allowing us to improve material selection for bio-inspired fin design.

Prediction of Freezing of Gait in Parkinson’s disease based on multi-channel time-series neural network

Freezing of Gait (FOG) is a noticeable symptom of Parkinson’s disease, like being stuck in place and increasing the risk of falls. The wearable multi-channel sensor system is an efficient method to predict and monitor the FOG, thus warning the wearer to avoid falls and improving the quality of life. However, the existing approaches for the prediction of FOG mainly focus on a single sensor system and cannot handle the interference between multi-channel wearable sensors. Hence, we propose a novel multi-channel time-series neural network (MCT-Net) approach to merge multi-channel gait features into a comprehensive prediction framework, alerting patients to FOG symptoms in advance. Owing to the causal distributed convolution, MCT-Net is a real-time method available to give optimal prediction earlier and implemented in remote devices. Moreover, intra-channel and inter-channel transformers of MCT-Net extract and integrate different sensor position features into a unified deep learning model. Compared with four other state-of-the-art FOG prediction baselines, the proposed MCT-Net obtains 96.21% in accuracy and 80.46% in F1-score on average 2 s before FOG occurrence, demonstrating the superiority of MCT-Net.

Comparative analysis of time series neural network methods for three-way catalyst modeling

Relative Oxygen Level of the Three-Way Catalyst is an important parameter that affects the conversion efficiency of pollutants. ROL is a time-varying hidden state variable that is difficult to directly observe in practice. Therefore, it is common to use a method of clearing oxygen storage to simplify control in vehicles. However, this method negates the positive effects of ROL on pollutant treatment. ROL can be indirectly observed through modeling methods. Chemical modeling methods involve extensive computational requirements that cannot meet the demands of practical control. In contrast, time-series neural networks offer computational speed advantages when dealing with similar problems. Therefore, the ROL observation models using both NARX and LSTM neural networks are developed and compared in this study. The results indicate that the NARX neural network exhibits higher precision with a smaller number of neurons and time steps. The LSTM neural network demonstrates greater stability when dealing with data error fluctuations. In practical applications, the ROL model can monitor the TWC operating status and assist in the development of intelligent pollutant aftertreatment control strategies.

Towards real-time earthquake forecasting in Chile: Integrating intelligent technologies and machine learning

This paper proposes an innovative approach towards real-time earthquake forecasting in Chile by integrating intelligent technologies and machine learning methods. Earthquakes pose significant risks to communities and infrastructure in Chile, making accurate and timely forecasting crucial for disaster preparedness and mitigation. Traditional forecasting methods have limitations in providing real-time insights into seismic activity. In contrast, intelligent technologies such as artificial intelligence (AI) and machine learning offer promising avenues for enhancing prediction accuracy and speed. A new earthquake forecasting method using a modified clustering approach LMSCAN(Local Maxima-based Spatio-Cluster Analysis Network) and enhanced neural network time series analysis LSTM-IC (Long Short Term Memory Inverse Correlation) is presented in this paper. This neural network-based technology has been utilized to forecast earthquakes in the Chile region, one of the countries with the largest seismic activity. This study explores the integration of intelligent systems with machine learning algorithms to analyze seismic data and predict earthquake occurrences in Chile. By leveraging historical seismic data, sensor networks, and advanced predictive models, our approach aims to provide timely warnings and insights into seismic events, thereby improving disaster response and resilience. The proposed framework holds the potential to revolutionize earthquake forecasting by enabling real-time monitoring and proactive measures to safeguard communities and infrastructure in Chile and beyond. The remarkable 95 % accuracy achieved by this model is a testament to its exceptional learning process, which sets it apart from other models. Its ability to learn and adapt to new data is unparalleled, allowing it to forecast incredibly precisely and produce highly reliable results.

On-the-fly training architecture for a time-series neural network on ADITYA/ADITYA-U data

Plasma disruption prediction using the data drive technique is popular for fusion experiments. Auto-encoder LSTM neural network is implemented on ADITYA/ADITYA-U data to predict the plasma disruption. There is a generic way of processing time series from a dataset that facilitates efficient generalization on new time-series data using a transfer learning technique. This technique enables task-specific parameter improvement with residual correction on univariate time series. This provides an on-the-fly architecture to expand the training model for new data without retraining the whole model. It also provides layer updates based on linearization learning for new input data This architecture enables us to update real-time saved models with new features of data that provide performance improvement during prediction. The demo is tested on ADITYA/ADITYA-U data which save time and run time updates to improve the accuracy of data. The improved results are derived by following the trend while predicting the result after adding new features via this architecture to the existing saved model. This technique allows automated on-the-fly training architecture at time-to-time basis or explicit need of retraining with new data.

Neural network time-series classifiers for gravitational-wave searches in single-detector periods

The search for gravitational-wave (GW) signals is limited by non-Gaussian transient noises that mimic astrophysical signals. Temporal coincidence between two or more detectors is used to mitigate contamination by these instrumental glitches. However, when a single detector is in operation, coincidence is impossible, and other strategies have to be used. We explore the possibility of using neural network classifiers and present the results obtained with three types of architectures: convolutional neural network, temporal convolutional network, and inception time. The last two architectures are specifically designed to process time-series data. The classifiers are trained on a month of data from the LIGO Livingston detector during the first observing run (O1) to identify data segments that include the signature of a binary black hole merger. Their performances are assessed and compared. We then apply trained classifiers to the remaining three months of O1 data, focusing specifically on single-detector times. The most promising candidate from our search is 4 January 2016 12:24:17 UTC. Although we are not able to constrain the significance of this event to the level conventionally followed in GW searches, we show that the signal is compatible with the merger of two black holes with masses m1=50.7−8.9+10.4M⊙ and m2=24.4−9.3+20.2M⊙ at the luminosity distance of dL=564−338+812Mpc .

Multidimensional forecasting of precipitation and potential evapotranspiration in the Paranapanema river basin using neural network time series

Spatial and temporal forecasts of the hydrological cycle compartments aiming projections of extreme drought scenarios represent a challenge for the planning, management and monitoring of water resources in order to mitigate potential impacts on the natural environment, civil society and wildlife under climate change. Machine Learning (ML) methods can help in this task, combining constant updating of model information and further scenarios evaluation. This study investigated the application of multidimensional forecast of precipitation and potential evapotranspiration at the Paranapanema River Basin (PRB) for the years 2023–2025. PRB is a region that provides hydrological, energy and agricultural resources, located in the southeast of Brazil that has suffered several problems related to water deficit and stress as well as droughts in the last 10 years. For these reasons, geospatial technologies such as remote sensing and Geographic Information Systems (GIS) were applied to generate time series between 2001 and early 2023 for a total of 22 Hydrological Planning Units (HPUs) in the PRB. Subsequently, a Neural Network Auto Regression (NNAR) was used to forecast precipitation and potential evapotranspiration of the HPUs in the period 2023–2025, finding for the months of May, June, July and August of 2024 and later in 2025 possible periods of water deficit in the central and northern regions. Finally, a comparative analysis of possible impacts on the agricultural, energy and social sectors based on the forecast developed by the NNAR network is presented, showing possible scenarios for short and mid-term water planning in the PRB.

News-driven stock market index prediction based on trellis network and sentiment attention mechanism

Applying deep learning, especially time series neural network, to stock market prediction, has become one of the important applications in the quantitative finance field. However, due to the multi-correlation and volatility of the stock market, how to timely and accurately predict it has become a challenging issue. In order to cope with this challenge, a news-driven stock market index prediction model based on TrellisNet and a sentiment attention mechanism (SA-TrellisNet) is proposed. A sentiment analysis model based on CNN and LSTM is presented to obtain the sentiment index of massive news crawled from authoritative financial websites. Furthermore, a sentiment attention mechanism is designed for data fusion of stock data and news sentiment index as the input of the simple and efficient TrellisNet network for model training and prediction. The performance of our model is systematically evaluated using seven major international stock market indices including S&P500, NYSE, DJI, NASDAQ, FTSE 100, Nikkei 225 and SSE, and comparative experiments demonstrate that SA-TrellisNet is competitive to the other state-of-the-art methods in predicting stock market indices.

Power-Aware Inverse-Search Machine Learning for Low Resource Multi-Objective Unmanned Underwater Vehicle Control (Student Abstract)

Flapping-fin unmanned underwater vehicle (UUV) propulsion systems enable high maneuverability for tasks ranging from station-keeping to surveillance but are often constrained by their limited computational power and battery capacity. Previous research has demonstrated that time-series neural network models can accurately predict the thrust and power of certain fin kinematics based on the specified gait coupled with the fin configuration, but can not fit an inverse neural network that takes a thrust request and tunes the kinematics by weighting thrust generation, smooth movement transitions, and power attributes. We study various combinations of the three weights and fin materials to create different ‘modes’ of movement for a multi-objective UUV, based on controller intent using an inverse neural network. Finally, we implement and validate an enhanced power-aware inverse model by benchmarking on the Raspberry Pi Model 4B system and testing through generated simulated movements.

Meta-Transfer Metric Learning for Time Series Classification in 6G-Supported Intelligent Transportation Systems

Deep learning-based time series classification in 6G-supported Intelligent Transportation Systems (ITS) helps transport decision-making. Deep learning classifier training necessitates a large amount of labeled data for feature extraction. Labeling time series data in 6G-supported ITS is tough. Meta-learning can be used to train deep classifiers with limited data. However, in meta-learning, the tasks are frequently modeled by a low-complexity base learner. It is unable to use more complicate and powerful structures. The meta-learning-pretrained classifier can only perform new classification problems with the same number of classes. Most pre-training strategies do not prioritize enhancing the pre-training phase’s convergence rate and lowering the computational cost. Most research work aims to improve classification performance by increasing the complexity of the classification model. However, this raises computing costs. In this paper, we propose a one-dimensional Multi-Scale Dilated Convolution Neural Network time series classifier (MSDCNN). MSDCNN combines multi-scale CNN and dilated CNN. It can extract multi-scale characteristics from time series and reduce the complexity of the classifier. Furthermore, we propose a pre-training strategy, called Meta-transfer metric Learning using Scale function (MLS). MLS allows the classifier to gain experience from different tasks with various numbers of classes. Experiments show that MLS reduces pre-training computation costs during the pre-training phase. The pre-trained classifier, without using any fine tuning techniques, achieves the highest accuracy by comparing with the state-of-the-art methods. Finally, we present a case study of applying MSDCNN and MLS to detect road accidents in 6G-supported transportation systems.

Predicting the Posture of High-Rise Building Machines Based on Multivariate Time Series Neural Network Models.

High-rise building machines (HBMs) play a critical role in the successful construction of super-high skyscrapers, providing essential support and ensuring safety. The HBM's climbing system relies on a jacking mechanism consisting of several independent jacking cylinders. A reliable control system is imperative to maintain the smooth posture of the construction steel platform (SP) under the action of the jacking mechanism. Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Temporal Convolutional Network (TCN) are three multivariate time series (MTS) neural network models that are used in this study to predict the posture of HBMs. The models take pressure and stroke measurements from the jacking cylinders as inputs, and their outputs determine the levelness of the SP and the posture of the HBM at various climbing stages. The development and training of these neural networks are based on historical on-site data, with the predictions subjected to thorough comparative analysis. The proposed LSTM and GRU prediction models have similar performances in the prediction process of HBM posture, with medians R2 of 0.903 and 0.871, respectively. However, the median MAE of the GRU prediction model is more petite at 0.4, which exhibits stronger robustness. Additionally, sensitivity analysis showed that the change in the levelness of the position of the SP portion of the HBM exhibited high sensitivity to the stroke and pressure of the jacking cylinder, which clarified the position of the cylinder for adjusting the posture of the HBM. The results show that the MTS neural network-based prediction model can change the HBM posture and improve work stability by adjusting the jacking cylinder pressure value of the HBM.

A time-series neural network for pig feeding behavior recognition and dangerous detection from videos

With the development of modern animal husbandry, especially pig farming, large-scale, intensive, and automated farming has become the trend in the industry. The realization of accurate recognition and warning of dangerous actions in feeding scenarios for sows and piglets holds high research and practical value in this field. This paper addresses this issue by proposing a Transformer-based Neural Network (TNN) model. This model emphasizes the extraction of global features and handling of long-distance dependencies, significantly improving the accuracy of behavior recognition. Compared with traditional neural network models, TNN demonstrates superiority in dealing with animal behavior recognition in farming scenarios. Furthermore, an in-depth study of the attention mechanism in the TNN model was conducted in this paper. By visualizing the attention heat map of the TNN model, it was found that TNN could effectively focus on key areas in the image, thereby accurately identifying the behavior of piglets. Finally, this paper proposes a unique model lightweighting strategy that allows the TNN model to run efficiently on edge devices. In the experimental part, the performance of the TNN model on five behavior recognition tasks was evaluated first. The results showed that the TNN model achieved high scores in both Precision and Recall, far exceeding traditional neural network models. Then, by visualizing the attention heat map of the TNN model, the superiority of the TNN model in focusing on key image areas was further confirmed. In the end, the effect of the model lightweighting strategy was demonstrated, and even with a significant reduction in parameters and computational complexity, the performance of the TNN model remained excellent. This research not only promotes the development of animal behavior recognition technology but also provides new insights and tools for the precise management and efficient operation of the animal husbandry industry.

Nonlinear autoregressive with exogeneous input neural network time series model performance: bitcoin price prediction

Nonlinear autoregressive with exogeneous input neural network time series model performance: bitcoin price prediction