Time Series Prediction Model Research Articles

Double-level optimal fuzzy association rules prediction model for time series based on DTW-i[formula omitted] fuzzy C-means

Information granulation theory has been widely used in short-term time-series forecasting research and holds significant weight. However, the error accumulation due to the lack of granular accuracy, along with information redundancy or deficiency in predictions, significantly affects short-term prediction accuracy. To compensate for these shortcomings, this paper proposes a double-level optimal fuzzy association rules prediction model for short-term time-series forecasting, which can strengthen the performance of information granulation in prediction. Firstly, this paper proposes a concept of breakpoints, which can accurately segment complex linear trends in time series and thus obtain a granular time series with highly accurate linear fuzzy information granules (LFIGs). Secondly, a improved distance is proposed to more accurately reflect the similarity between LFIGs by addressing counter-intuitive problems in the original distance. Theoretical analysis shows that the improved distance can effectively reduce errors in granular calculation. Then, a granule-suited fuzzy c-means algorithm is proposed for clustering LFIGs. Finally, this paper proposes a double-level optimal fuzzy association rules prediction model, which establishes the optimal rules for each cluster and selects the optimal two rules for prediction by the contribution of the clusters. The experimental results show that the prediction method effectively avoids the problems of information redundancy and information deficiency, and increases forecast accuracy. The model’s exceptional performance is demonstrated through comparative analysis with existing models in experimental investigations.

A Time Series Prediction Model for Wind Power Based on the Empirical Mode Decomposition–Convolutional Neural Network–Three-Dimensional Gated Neural Network

In response to the global challenge of climate change and the shift away from fossil fuels, the accurate prediction of wind power generation is crucial for optimizing grid operations and managing energy storage. This study introduces a novel approach by integrating the proportional–integral–derivative (PID) control theory into wind power forecasting, employing a three-dimensional gated neural (TGN) unit designed to enhance error feedback mechanisms. The proposed empirical mode decomposition (EMD)–convolutional neural network (CNN)–three-dimensional gated neural network (TGNN) framework starts with the pre-processing of wind data using EMD, followed by feature extraction via a CNN, and time series forecasting using the TGN unit. This setup leverages proportional, integral, and differential control within its architecture to improve adaptability and response to dynamic wind patterns. The experimental results show significant improvements in forecasting accuracy; the EMD–CNN–TGNN model outperforms both traditional models like autoregressive integrated moving average (ARIMA) and support vector regression (SVR), and similar neural network approaches, such as EMD–CNN–GRU and EMD–CNN–LSTM, across several metrics including mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), and coefficient of determination (R2). These advancements substantiate the model’s effectiveness in enhancing the precision of wind power predictions, offering substantial implications for future renewable energy management and storage solutions.

A reservoir computing model based on nonlinear spiking neural P systems for time series forecasting

Nonlinear spiking neural P (NSNP) systems are a novel class of neural-like computing models that exhibit nonlinear dynamic behaviours. Further, reservoir computing (RC) is a new type of recurrent neural network (RNN) whose dynamic behaviour is provided by a reservoir of neurones. An interesting question is whether NSNP systems can be embedded into an RC framework to develop an efficient prediction model for time series forecasting problems. To address this problem, this paper first defines a variant of NSNP systems with two outputs, called NSNP-TO systems, in which a nonlinear spiking mechanism with two outputs is the key feature. Based on NSNP-TO systems, a new variant of RC models is proposed based on NSNP systems, called the RC-NSNP model. Owing to the use of spiking neurones, the RC-NSNP model comprises state-updating and output equations that are different from those of the existing RC models. Using a similar structure, the proposed RC-NSNP model can be implemented in the RC framework, and a similar training method can be utilised to train it. Time series forecasting tasks are performed to verify the effectiveness of the proposed RC-NSNP model. The proposed RC-NSNP model achieved the best performance on nine of the twelve time series problems and obtained the second-best performance on the other two time series.

Multi-robot Source Navigation Method Based on Coordination Graph Monte Carlo Tree Search

To address the dynamic cooperative multi-robot sequential decision problem, we propose a multi-robot source navigation method based on coordination graph Monte Carlo tree search. The Monte Carlo tree search online planning is designed with the algorithm system's structure in mind. We combine the upper confidence bound strategy with the coordination graph, which represents multi-robot cooperative communication, and solve the coordination graph using the max-sum algorithm to select joint actions. This paper proposes a time series prediction model for optimizing multi-robot movement trends using historical data, which is then used to solve the exploration and exploitation problem in Monte Carlo tree search. The experimental analysis and comparison show that our proposed method outperforms the comparison algorithms in terms of source search steps, task execution time, and source search success rate, as well as robustness and efficiency for multi-robot source search.

New perspective of environmental impact research: predicting bus exhaust emissions using the ETSformer based on collaborative perception

Accurate prediction of urban traffic exhaust emissions plays a crucial role in controlling motor vehicle exhaust pollution. Urban public transport vehicles often operate in stop-and-go traffic conditions, emitting significant exhaust pollution. Existing approaches primarily focus on assessing vehicle emissions in the past or present, which may not adequately address long-term planning needs. To enhance the accuracy of bus exhaust emissions prediction, we introduce a novel time series transformer architecture named ETSformer for forecasting future vehicle emissions. This framework integrates exponential smoothing principles to refine the transformer-based time series prediction model, utilizing historical pollutant emissions data to forecast future exhaust emissions. The experiment collects emissions data and driving status information from public transport vehicles in Zhenjiang City, Jiangsu Province, considering factors such as vehicle runtime, route, and operational state to standardize response variables and enhance model precision and consistency. Experimental results demonstrate a determination coefficient (R2) of 70.7% and a mean square error (MSE) of 41.1% for exhaust gas prediction, signifying improved accuracy and efficiency compared to other models. Applying the exhaust gas prediction model to bus operations enables advanced forecasting of future exhaust emissions, addressing current challenges and potentially reducing Carbon Dioxide (CO2) emissions by over 3% during bus operations.

ENSO dataset & comparison of deep learning models for ENSO forecasting

Forecasting the El Nino-Southern Oscillation (ENSO) is a challenging task in climatology. It is one of the main factors responsible for the Earth’s interannual climatic fluctuation and can result in many climatic anomalies. The impacts include natural disasters (floods, droughts), low & high agriculture yields, price fluctuation, energy demand, availability of water resources, animal movement, and many more. This study presents a comprehensive ENSO dataset containing standard indicators and other relevant data to facilitate ENSO analysis and forecasting. To ensure the dataset's validity and reliability, we performed extensive data analysis and trained four basic deep models for time series forecasting (i.e. CNN, RNN, LSTM, and hybrids). The data analysis confirmed the accuracy and suitability of the dataset for ENSO forecasting. The LSTM model achieved the best fit to the data, leading to superior performance in forecasting ENSO events.

Reliability improvement of the dredging perception system: A sensor fault-tolerant strategy

In the dredging industry, the automation and accuracy of the Dredging Perception System (DPS) are vital for operational efficiency and environmental safety. Current DPS implementations face challenges with sensor fault tolerance, leading to system unreliability and increased false alarm rates that can disrupt dredging operations. We propose a Hybrid Redundancy Sensor Fault Tolerance (HRSFT) strategy that integrates matching physical sensors (PS) with two distinct types of virtual sensors (VS) driven by multi-sensor association and time-series prediction models. The HRSFT employs a voting-cold storage strategy to address the false alarm issues commonly associated with single virtual sensor systems. Through experimental validation, the HRSFT strategy has demonstrated its capability to provide accurate replacement information during both single and multi-sensor failure scenarios, effectively managing abnormal sensor data and enhancing the operational reliability of the DPS. The implementation of the HRSFT strategy significantly improves the accuracy and stability of the DPS, suggesting a substantial advancement in sensor fault tolerance that could be applied to similar systems in various industries, leading to safer and more reliable operations.

TrajectoFormer: Transformer-Based Trajectory Prediction of Autonomous Vehicles with Spatio-temporal Neighborhood Considerations

Accurate trajectory prediction of autonomous vehicles is crucial for ensuring road safety. Predicting precise and accurate trajectories is still considered a challenging problem because of the intricate spatio-temporal dependencies among the vehicles. Our study primarily focuses on resolving this issue by introducing a comprehensive system called “TrajectoFormer”, which can effectively represent the spatio-temporal dependency between vehicles. In this system, we have conducted preprocessing on the NGSIM dataset by constructing an 8-neighborhood for each vehicle that represents the spatio-temporal dependency between vehicles effectively. Second, we have deployed a transformer network that captures dependencies between the target vehicle and its neighbor from the constructed neighborhood and predicts future trajectories for the target vehicle with notably reduced training times and significant accuracy compared to existing methods. Experiments on both NGSIM US-101 and US-I80 show that our proposed approach outperforms the other benchmarks in terms of showing low RMSE value for the 5-s prediction horizon of trajectory prediction. Our conducted ablation study also underscores the effectiveness of each component of our proposed TrajectoFormer model relative to traditional time-series prediction models.

Forecasting carbon peaking in China using data-driven rule-base model: An in-depth analysis across regional and economic scenarios

At the 2020 United Nations Climate Summit, China officially announced the goal to achieve carbon peaking by 2030. Exploring whether it is possible to reach the peak of carbon emissions earlier necessitates an urgent and imperative need for precise long-term forecasting of China's carbon emissions dynamics. However, the current carbon peaking predictions mostly depend on mechanical or mathematical models, which failed to consider the interdependence between carbon emissions and the time series-based patterns existed in carbon emission data. Therefore, this study presents a novel carbon peaking prediction method based on the data-driven rule-base model, which is implemented by the adaption of the extended belief rule base (EBRB) model for time series forecasting (TSF), and thus the proposed method is referred to as TSF-EBRB model. The TSF-EBRB model not only captures and measures the temporal correlations within the data throughout the processes of modeling and inference, but also consists of a novel parameter optimization model based on the temporal correlations. The study collected carbon emission data from 30 provinces in China for empirical analysis. It computed and predicted the carbon peaking trajectories of each province under three different scenarios from 2022 to 2030, validating the effectiveness and superiority of the TSF-EBRB model better than other existing carbon peaking prediction methods. The results indicated that, with policy interventions, the majority of provinces are projected to reach carbon peaking before 2030.

MaxEnt SeismoSense Model: Ionospheric Earthquake Anomaly Detection Based on the Maximum Entropy Principle

In our exploration, we aimed at identifying seismic anomalies using limited ionospheric data for earthquake forecasting and we meticulously compiled datasets under conditions of minimal geomagnetic disturbance. Our systematic evaluation affirmed the ITransformer as a potent tool for the feature extraction of ionospheric data, standing out within the domain of transformer-based time series prediction models. We integrated the maximum entropy principle to fully leverage the available information, while minimizing the influence of presuppositions on our predictions. This led to the creation of the MaxEnt SeismoSense Model, a novel composite model that combines the strengths of the transformer architecture with the maximum entropy principle to improve prediction accuracy. The application of this model demonstrated a proficient capability to detect seismic disturbances in the ionosphere, showcasing an improvement in both recall rate and accuracy to 71% and 69%, respectively, when compared to conventional baseline models. This indicates that the combined use of transformer technology and the maximum entropy principle could allow pre-seismic anomalies in the ionosphere to be sensed more efficiently and could offer a more reliable and precise approach to earthquake prediction.

Time Series Prediction Based on Multi-Scale Feature Extraction

Time series data are prevalent in the real world, particularly playing a crucial role in key domains such as meteorology, electricity, and finance. Comprising observations at historical time points, these data, when subjected to in-depth analysis and modeling, enable researchers to predict future trends and patterns, providing support for decision making. In current research, especially in the analysis of long time series, effectively extracting and integrating long-term dependencies with short-term features remains a significant challenge. Long-term dependencies refer to the correlation between data points spaced far apart in a time series, while short-term features focus on more recent changes. Understanding and combining these two features correctly are crucial for constructing accurate and reliable predictive models. To efficiently extract and integrate long-term dependencies and short-term features in long time series, this paper proposes a pyramid attention structure model based on multi-scale feature extraction, referred to as the MSFformer model. Initially, a coarser-scale construction module is designed to obtain coarse-grained information. A pyramid data structure is constructed through feature convolution, with the bottom layer representing the original data and each subsequent layer containing feature information extracted across different time step lengths. As a result, nodes higher up in the pyramid integrate information from more time points, such as every Monday or the beginning of each month, while nodes lower down retain their individual information. Additionally, a Skip-PAM is introduced, where a node only calculates attention with its neighboring nodes, parent node, and child nodes, effectively reducing the model’s time complexity to some extent. Notably, the child nodes refer to nodes selected from the next layer by skipping specific time steps. In this study, we not only propose an innovative time series prediction model but also validate the effectiveness of these methods through a series of comprehensive experiments. To comprehensively evaluate the performance of the designed model, we conducted comparative experiments with baseline models, ablation experiments, and hyperparameter studies. The experimental results demonstrate that the MSFformer model improves by 35.87% and 42.6% on the MAE and MSE indicators, respectively, compared to traditional Transformer models. These results highlight the outstanding performance of our proposed deep learning model in handling complex time series data, particularly in capturing long-term dependencies and integrating short-term features.

Learning from Polar Representation: An Extreme-Adaptive Model for Long-Term Time Series Forecasting

In the hydrology field, time series forecasting is crucial for efficient water resource management, improving flood and drought control and increasing the safety and quality of life for the general population. However, predicting long-term streamflow is a complex task due to the presence of extreme events. It requires the capture of long-range dependencies and the modeling of rare but important extreme values. Existing approaches often struggle to tackle these dual challenges simultaneously. In this paper, we specifically delve into these issues and propose Distance-weighted Auto-regularized Neural network (DAN), a novel extreme-adaptive model for long-range forecasting of stremflow enhanced by polar representation learning. DAN utilizes a distance-weighted multi-loss mechanism and stackable blocks to dynamically refine indicator sequences from exogenous data, while also being able to handle uni-variate time-series by employing Gaussian Mixture probability modeling to improve robustness to severe events. We also introduce Kruskal-Wallis sampling and gate control vectors to handle imbalanced extreme data. On four real-life hydrologic streamflow datasets, we demonstrate that DAN significantly outperforms both state-of-the-art hydrologic time series prediction methods and general methods designed for long-term time series prediction.

AutoMixer for Improved Multivariate Time-Series Forecasting on Business and IT Observability Data

The efficiency of business processes relies on business key performance indicators (Biz-KPIs), that can be negatively impacted by IT failures. Business and IT Observability (BizITObs) data fuses both Biz-KPIs and IT event channels together as multivariate time series data. Forecasting Biz-KPIs in advance can enhance efficiency and revenue through proactive corrective measures. However, BizITObs data generally exhibit both useful and noisy inter-channel interactions between Biz-KPIs and IT events that need to be effectively decoupled. This leads to suboptimal forecasting performance when existing multivariate forecasting models are employed. To address this, we introduce AutoMixer, a time-series Foundation Model (FM) approach, grounded on the novel technique of channel-compressed pretrain and finetune workflows. AutoMixer leverages an AutoEncoder for channel-compressed pretraining and integrates it with the advanced TSMixer model for multivariate time series forecasting. This fusion greatly enhances the potency of TSMixer for accurate forecasts and also generalizes well across several downstream tasks. Through detailed experiments and dashboard analytics, we show AutoMixer's capability to consistently improve the Biz-KPI's forecasting accuracy (by 11-15%) which directly translates to actionable business insights.

Predicting quantum emitter fluctuations with time-series forecasting models

2D materials have important fundamental properties allowing for their use in many potential applications, including quantum computing. Various Van der Waals materials, including Tungsten disulfide (WS2), have been employed to showcase attractive device applications such as light emitting diodes, lasers and optical modulators. To maximize the utility and value of integrated quantum photonics, the wavelength, polarization and intensity of the photons from a quantum emission (QE) must be stable. However, random variation of emission energy, caused by the inhomogeneity in the local environment, is a major challenge for all solid-state single photon emitters. In this work, we assess the random nature of the quantum fluctuations, and we present time series forecasting deep learning models to analyse and predict QE fluctuations for the first time. Our trained models can roughly follow the actual trend of the data and, under certain data processing conditions, can predict peaks and dips of the fluctuations. The ability to anticipate these fluctuations will allow physicists to harness quantum fluctuation characteristics to develop novel scientific advances in quantum computing that will greatly benefit quantum technologies.

A novel solar irradiance forecasting method based on multi-physical process of atmosphere optics and LSTM-BP model

Prediction of solar irradiance is crucial for minimizing energy costs and ensuring high power quality in electrical power grids that incorporate distributed solar photovoltaic generation. Traditional methods have focused mainly on time series prediction of irradiance or studies of the relationship between irradiance and environmental factors at a given moment. This paper presents a novel global horizontal irradiance forecasting method based on multi-physical process of atmosphere optics and LSTM-BP model. In the proposed method, cloud fraction, cloud albedo and aerosol optical thickness are selected as critical characteristic factors in the sunlight-atmosphere interaction. Long short-term memory (LSTM) is utilized to extract temporal features of the critical characteristic factors in various physical processes of atmospheric optics and predict their values. Subsequently, back propagation (BP) neural network is constructed to investigate the relationship between these characteristic factors and global horizontal irradiance under a given point in time. The global horizontal irradiance is predicted by combining the predicted values of critical characteristic factors and BP neural network for modelling irradiance decay process. To demonstrate the superior performance of the proposed model, the results obtained by the proposed method are compared with those of the LSTM network model for time series forecasting, in which six error evaluation indicators are used in the comparison. The proposed method for solar irradiance forecasting is verified in different time scales and under various weather conditions and shows better accuracy comparing with time series forecasting methods.

Tesla Stock Prediction Based on Twitter Comments’ Sentiment Feature Extraction and Ensemble Networks

As people become more aware of environmental issues, Tesla, a pillar of the global electric vehicle market, has become a hot commodity in recent years. The prediction of Tesla’s share price is also a hot topic in the investment market. This experiment extracts sentiment factors of tweets about Tesla’s comments, combined with Tesla’s historical stock price as the dataset for training, testing and inspection. This experiment builds time series prediction models based on LSTM, XGBoost and RF algorithms to predict Tesla’s stock price, and evaluates the prediction effectiveness of the three algorithms based on the fit and error of the prediction results. The analysis of the data shows that XGBoost has the best fit and the lowest error among the three algorithms, and that the sentiment factor has its unique utility as raw data. The experimental results also empirically demonstrate the applicability of sentiment factor analysis and the three algorithms LSTM, XGBoost and RF in the field of stock price prediction.

Ensemble dynamics and information flow deduction from whole-brain imaging data.

The recent advancements in large-scale activity imaging of neuronal ensembles offer valuable opportunities to comprehend the process involved in generating brain activity patterns and understanding how information is transmitted between neurons or neuronal ensembles. However, existing methodologies for extracting the underlying properties that generate overall dynamics are still limited. In this study, we applied previously unexplored methodologies to analyze time-lapse 3D imaging (4D imaging) data of head neurons of the nematode Caenorhabditis elegans. By combining time-delay embedding with the independent component analysis, we successfully decomposed whole-brain activities into a small number of component dynamics. Through the integration of results from multiple samples, we extracted common dynamics from neuronal activities that exhibit apparent divergence across different animals. Notably, while several components show common cooperativity across samples, some component pairs exhibited distinct relationships between individual samples. We further developed time series prediction models of synaptic communications. By combining dimension reduction using the general framework, gradient kernel dimension reduction, and probabilistic modeling, the overall relationships of neural activities were incorporated. By this approach, the stochastic but coordinated dynamics were reproduced in the simulated whole-brain neural network. We found that noise in the nervous system is crucial for generating realistic whole-brain dynamics. Furthermore, by evaluating synaptic interaction properties in the models, strong interactions within the core neural circuit, variable sensory transmission and importance of gap junctions were inferred. Virtual optogenetics can be also performed using the model. These analyses provide a solid foundation for understanding information flow in real neural networks.

A Novel Ensemble Machine Learning Model for Oil Production Prediction with Two-Stage Data Preprocessing

Petroleum production forecasting involves the anticipation of fluid production from wells based on historical data. Compared to traditional empirical, statistical, or reservoir simulation-based models, machine learning techniques leverage inherent relationships among historical dynamic data to predict future production. These methods are characterized by readily available parameters, fast computational speeds, high precision, and time–cost advantages, making them widely applicable in oilfield production. In this study, time series forecast models utilizing robust and efficient machine learning techniques are formulated for the prediction of production. We have fused the two-stage data preprocessing methods and the attention mechanism into the temporal convolutional network-gated recurrent unit (TCN-GRU) model. Firstly, the random forest (RF) algorithm is employed to extract key dynamic production features that influence output, serving to reduce data dimensionality and mitigate overfitting. Next, the mode decomposition algorithm, complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN), is introduced. It employs a decomposition–reconstruction approach to segment production data into high-frequency noise components, low-frequency regular components and trend components. These segments are then individually subjected to prediction tasks, facilitating the model’s ability to capture more accurate intrinsic relationships among the data. Finally, the TCN-GRU-MA model, which integrates a multi-head attention (MA) mechanism, is utilized for production forecasting. In this model, the TCN module is employed to capture temporal data features, while the attention mechanism assigns varying weights to highlight the most critical influencing factors. The experimental results indicate that the proposed model achieves outstanding predictive performance. Compared to the best-performing comparative model, it exhibits a reduction in RMSE by 3%, MAE by 1.6%, MAPE by 12.7%, and an increase in R2 by 2.6% in Case 1. Similarly, in Case 2, there is a 7.7% decrease in RMSE, 7.7% in MAE, 11.6% in MAPE, and a 4.7% improvement in R2.

Nonlinear Spiking Neural Systems With Autapses for Predicting Chaotic Time Series.

Spiking neural P (SNP) systems are a class of distributed and parallel neural-like computing models that are inspired by the mechanism of spiking neurons and are 3rd-generation neural networks. Chaotic time series forecasting is one of the most challenging problems for machine learning models. To address this challenge, we first propose a nonlinear version of SNP systems, called nonlinear SNP systems with autapses (NSNP-AU systems). In addition to the nonlinear consumption and generation of spikes, the NSNP-AU systems have three nonlinear gate functions, which are related to the states and outputs of the neurons. Inspired by the spiking mechanisms of NSNP-AU systems, we develop a recurrent-type prediction model for chaotic time series, called the NSNP-AU model. As a new variant of recurrent neural networks (RNNs), the NSNP-AU model is implemented in a popular deep learning framework. Four datasets of chaotic time series are investigated using the proposed NSNP-AU model, five state-of-the-art models, and 28 baseline prediction models. The experimental results demonstrate the advantage of the proposed NSNP-AU model for chaotic time series forecasting.

Attention-Based Models for Multivariate Time Series Forecasting: Multi-step Solar Irradiation Prediction

Bangladesh's subtropical climate with an abundance of sunlight throughout the greater portion of the year results in increased effectiveness of solar panels. Solar irradiance forecasting is an essential aspect of grid-connected photovoltaic systems to efficiently manage solar power's variation and uncertainty and to assist in balancing power supply and demand. This is why it is essential to forecast solar irradiation accurately. Many meteorological factors influence solar irradiation, which has a high degree of fluctuation and uncertainty. Predicting solar irradiance multiple steps ahead makes it difficult for forecasting models to capture long-term sequential relationships. Attention-based models are widely used in the field of Natural Language Processing for their ability to learn long-term dependencies within sequential data. In this paper, our aim is to present an attention-based model framework for multivariate time series forecasting. Using data from two different locations in Bangladesh with a resolution of 30 min, the Attention-based encoder-decoder, Transformer, and Temporal Fusion Transformer (TFT) models are trained and tested to predict over 24 steps ahead and compared with other forecasting models. According to our findings, adding the attention mechanism significantly increased prediction accuracy and TFT has shown to be more precise than the rest of the algorithms in terms of accuracy and robustness. The obtained mean square error (MSE), the mean absolute error (MAE), and the coefficient of determination (R2) values for TFT are 0.151, 0.212, and 0.815, respectively. In comparison to the benchmark and sequential models (including the Naive, MLP, and Encoder-Decoder models), TFT has a reduction in the MSE and MAE of 8.4–47.9% and 6.1–22.3%, respectively, while R2 is raised by 2.13–26.16%. The ability to incorporate long-distance dependency increases the predictive power of attention models.