Chaotic Time Series Forecasting Research Articles

Dynamic adaptive graph convolutional transformer with broad learning system for multi-dimensional chaotic time series prediction

Chaotic time series data is extensively applied in financial stocks, climate monitoring, and sea clutter, in which data fusion from various sources and multi-sensor information make accurate predictions of chaotic time series challenging under complex nonlinear conditions. Previous works focus on designing different model frameworks to capture the temporal dependence and extract richer nonlinear features to improve the accuracy of univariate chaotic time series prediction, which ignores the spatial dependence of multivariable. However, in this paper, we argue that spatial correlation among multiple variables is essential to improve the prediction accuracy of chaotic time series. To fill the gap, we innovatively propose a Dynamic Adaptive Graph Convolutional Transformer with a Broad Learning System (DAGCT-BLS), a GCN and Transformer-based model utilizing multivariate spatial dependence for multi-dimensional chaotic time series forecasting. In DAGCT-BLS, the multivariate chaotic time series are reconstructed into the phase space, and the reconstructed data are rapidly feature-extracted using a cascade network BLS with frozen weights to maximize the retention of chaotic properties and nonlinear relationships. Then, the Dynamic Adaptive Graph Convolutional Network (DAGCN) is proposed to capture the spatial correlation among the multiple variables. Finally, improved multi-head attention of the Transformer Encoder is used to capture the temporal dependence of the phase point sequence. Experiments of our proposed model on three datasets (Lorenz, Rossler, and Sea clutter) show that DAGCT-BLS can achieve the best prediction performance and have strong interpretability, and multivariate-based joint modeling of chaotic time series helps to improve the prediction performance.

Nonlinear Spiking Neural Systems With Autapses for Predicting Chaotic Time Series.

Spiking neural P (SNP) systems are a class of distributed and parallel neural-like computing models that are inspired by the mechanism of spiking neurons and are 3rd-generation neural networks. Chaotic time series forecasting is one of the most challenging problems for machine learning models. To address this challenge, we first propose a nonlinear version of SNP systems, called nonlinear SNP systems with autapses (NSNP-AU systems). In addition to the nonlinear consumption and generation of spikes, the NSNP-AU systems have three nonlinear gate functions, which are related to the states and outputs of the neurons. Inspired by the spiking mechanisms of NSNP-AU systems, we develop a recurrent-type prediction model for chaotic time series, called the NSNP-AU model. As a new variant of recurrent neural networks (RNNs), the NSNP-AU model is implemented in a popular deep learning framework. Four datasets of chaotic time series are investigated using the proposed NSNP-AU model, five state-of-the-art models, and 28 baseline prediction models. The experimental results demonstrate the advantage of the proposed NSNP-AU model for chaotic time series forecasting.

Mackey–Glass Chaotic Time Series Forecasting by Using Self-Organizing Fuzzy Inference Network

Mackey–Glass Chaotic Time Series Forecasting by Using Self-Organizing Fuzzy Inference Network

Chaotic Time Series Forecasting Approaches Using Machine Learning Techniques: A Review

Traditional statistical, physical, and correlation models for chaotic time series prediction have problems, such as low forecasting accuracy, computational time, and difficulty determining the neural network’s topologies. Over a decade, various researchers have been working with these issues; however, it remains a challenge. Therefore, this review paper presents a comprehensive review of significant research conducted on various approaches for chaotic time series forecasting, using machine learning techniques such as convolutional neural network (CNN), wavelet neural network (WNN), fuzzy neural network (FNN), and long short-term memory (LSTM) in the nonlinear systems aforementioned above. The paper also aims to provide issues of individual forecasting approaches for better understanding and up-to-date knowledge for chaotic time series forecasting. The comprehensive review table summarizes the works closely associated with the mentioned issues. It includes published year, research country, forecasting approach, application, forecasting parameters, performance measures, and collected data area in this sector. Future improvements and current studies in this field are broadly examined. In addition, possible future scopes and limitations are closely discussed.

A hybrid model of stacked autoencoder and modified particle swarm optimization for multivariate chaotic time series forecasting

Phase space reconstruction (PSR) is an effective method for chaotic system modeling, which can reveal the implicit evolution information in a complex system. However, the reconstructed time series tend to have a high dimension and contain some redundant information. It is difficult for a traditional simple model to directly forecast the reconstructed time series. In this paper, we propose a hybrid model of stacked autoencoder (SAE) and modified particle swarm optimization (MPSO) for multivariate chaotic time series forecasting. We utilize SAE to extract the reconstructed time series and adopt feedforward neural network (FNN) to forecast time series. In the proposed hybrid model, the SAE is followed by FNN, and we make the MPSO to train the output weights of the model, which is a large-scale optimization problem. To enhance the generalization ability and prevent over-fitting, we add a regularization item to the objective function when MPSO trains the weights of the model. Experimental results show that MPSO algorithm has advantages in the exploration and exploitation in large-scale optimization problems. Then, experiments on Lorenz time series and two real-world time series datasets verify the effectiveness of the hybrid model in multivariate chaotic time series forecasting.

Entanglement-Structured LSTM Boosts Chaotic Time Series Forecasting.

Traditional machine-learning methods are inefficient in capturing chaos in nonlinear dynamical systems, especially when the time difference between consecutive steps is so large that the extracted time series looks apparently random. Here, we introduce a new long-short-term-memory (LSTM)-based recurrent architecture by tensorizing the cell-state-to-state propagation therein, maintaining the long-term memory feature of LSTM, while simultaneously enhancing the learning of short-term nonlinear complexity. We stress that the global minima of training can be most efficiently reached by our tensor structure where all nonlinear terms, up to some polynomial order, are treated explicitly and weighted equally. The efficiency and generality of our architecture are systematically investigated and tested through theoretical analysis and experimental examinations. In our design, we have explicitly used two different many-body entanglement structures—matrix product states (MPS) and the multiscale entanglement renormalization ansatz (MERA)—as physics-inspired tensor decomposition techniques, from which we find that MERA generally performs better than MPS, hence conjecturing that the learnability of chaos is determined not only by the number of free parameters but also the tensor complexity—recognized as how entanglement entropy scales with varying matricization of the tensor.

Prediction of chaotic time series using hybrid neural network and attention mechanism

Chaotic time series forecasting has been widely used in various domains, and the accurate predicting of the chaotic time series plays a critical role in many public events. Recently, various deep learning algorithms have been used to forecast chaotic time series and achieved good prediction performance. In order to improve the prediction accuracy of chaotic time series, a prediction model (Att-CNN-LSTM) is proposed based on hybrid neural network and attention mechanism. In this paper, the convolutional neural network (CNN) and long short-term memory (LSTM) are used to form a hybrid neural network. In addition, a attention model with <i>softmax</i> activation function is designed to extract the key features. Firstly, phase space reconstruction and data normalization are performed on a chaotic time series, then convolutional neural network (CNN) is used to extract the spatial features of the reconstructed phase space, then the features extracted by CNN are combined with the original chaotic time series, and in the long short-term memory network (LSTM) the combined vector is used to extract the temporal features. And then attention mechanism captures the key spatial-temporal features of chaotic time series. Finally, the prediction results are computed by using spatial-temporal features. To verify the prediction performance of the proposed hybrid model, it is used to predict the Logistic, Lorenz and sunspot chaotic time series. Four kinds of error criteria and model running times are used to evaluate the performance of predictive model. The proposed model is compared with hybrid CNN-LSTM model, the single CNN and LSTM network model and least squares support vector machine(LSSVM), and the experimental results show that the proposed hybrid model has a higher prediction accuracy.

Robustness of LSTM neural networks for multi-step forecasting of chaotic time series

Abstract Recurrent neurons (and in particular LSTM cells) demonstrated to be efficient when used as basic blocks to build sequence to sequence architectures, which represent the state-of-the-art approach in many sequential tasks related to natural language processing. In this work, these architectures are proposed as general purposes, multi-step predictors for nonlinear time series. We analyze artificial, noise-free data generated by chaotic oscillators and compare LSTM nets with the benchmarks set by feed-forward, one-step-recursive and multi-output predictors. We focus on two different training methods for LSTM nets. The traditional one makes use of the so-called teacher forcing, i.e. the ground truth data are used as input for each time step ahead, rather than the outputs predicted for the previous steps. Conversely, the second feeds the previous predictions back into the recurrent neurons, as it happens when the network is used in forecasting. LSTM predictors robustly show the strengths of the two benchmark competitors, i.e., the good short-term performance of one-step-recursive predictors and greatly improved mid-long-term predictions with respect to feed-forward, multi-output predictors. Training LSTM predictors without teacher forcing is recommended to improve accuracy and robustness, and ensures a more uniform distribution of the predictive power within the chaotic attractor. We also show that LSTM architectures maintain good performances when the number of time lags included in the input differs from the actual embedding dimension of the dataset, a feature that is very important when working on real data.

Deep Hybrid Neural Network and Improved Differential Neuroevolution for Chaotic Time Series Prediction

Chaos is widespread in non-linear systems such as finance, energy, and weather. In the chaos system, a variable changing with time generates a chaotic time series, which contains a wealth of information about the non-linear system, and it is helpful for us to analyze and understand chaos systems. Traditional hybrid models for chaotic time series prediction based on neural networks have problems such as low prediction accuracy and difficulty in determining the network topologies. In recent years, the chaotic time series prediction has attached the attention of researchers in the area of deep learning. In this paper, we use a deep hybrid neural network (DHNN) based on convolutional neural network (CNN), gated recurrent unit (GRU) network, and attention mechanism to predict chaotic time series. Besides, we use the idea of neuroevolution to optimize the topologies of the DHNN. In the DHNN, we use CNN to capture spatial features from phase space reconstruction of chaotic time series. Then, we combine spatial features with the original chaotic time series. GRU extracts the spatio-temporal features from the combined sequence, and an attention mechanism with a non-linear activation function is designed to capture critical spatio-temporal features. Besides, we propose an improved differential evolution (IDE) algorithm to optimize the topologies of the DHNN, including the filter sizes of CNN and the number of hidden neurons of GRU. We develop the IDE with an adaptive mutation operator and dynamic chaos crossover operator, which can improve convergence speed and reduce optimization time. In this paper, we use the theoretical Lorenz datasets, monthly mean total sunspot datasets, and the actual coal-mine gas concentration datasets to verify the prediction accuracy of the proposed prediction model. Experimental results have shown that the proposed prediction model performs well in chaotic time series forecasting.

Low dimensional mid-term chaotic time series prediction by delay parameterized method

How to predict the future behavior of complex systems with insufficient information, i.e., low dimensional mid-term chaotic time series prediction in mathematical terms, is not only a significant theoretical problem, but a more intricate practical problem. To address this issue, a Delay Parameterized Method (DPM) for low dimensional mid-term chaotic time series forecasting is presented. The correlation function, which immerses the low dimensional information into reconstructed space, is introduced to bridge time series and hidden order of system in DPM. Traversal algorithm and intelligent algorithm including particle swarm optimization or genetic algorithm, are used to obtain the optimal parameters for prediction. In addition, the applications of the proposed method on Lorenz chaotic time series, stress-strain signals and stock K-line maps show that it produces high quality predictions.

Performance of back-propagation neural network in chaotic data time series forecasting and evaluation over parametric forecast: a case study for rainfall-runoff modelling over a river basin

Back-propagation neural network (BPN) is sufficiently suitable for forecasting of Chaotic data time series by two approaches i.e., parametric forecasting and time-series forecasting. However, conformation of its optimum architecture for a specific case is pre-requisite. Total rainfall-runoff (R–R) from Basantpur station over Mahanadi river basin was in under study. Initially modelling of R–R by parametric forecast approach was done by optimum architecture of BPN. It is found that BPN performs excellent for the months of July, August, and October. For the September performance was drastically unfortunate. Modelling was denied by hypothesis \(MAD\,\left( {\% \,of\,LPA} \right)\,<\,{\raise0.7ex\hbox{$1$} \!\mathord{\left/ {\vphantom {1 {2 }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${2 }$}} SD (\% \,of\,LPA)\). By 20,000,000 epochs of training \(MAD\,\left( {\% \,of\,LPA} \right)\) were 23.40 and \(SD\,\left( {\% \,of\,LPA} \right)\) were 33.84. Therefore second approach of modelling i.e., time-series forecasting was applied. In which, ‘n’ years monthly past recorded R–R over the station is used to forecast of \((n + 1)\)th year monthly R–R over the station. This was significantly found appropriate and better evaluated over parametric approach. The model was highly acceptable under the hypothesis wherein \(MAD\,\left( {\% \,of\,LPA} \right)\) was 2.4516164772862348E-6 and \(SD\,\left( {\% \,of\,LPA} \right)\) was 33.45 and corelation coefficient (CC) between actual and predicted R–R is obtained 1.0 in training and 0.82 in testing independently. In this paper detail design of BPN in time-series forecasting, optimization of its architecture, training with 35 years data sets, testing with 7 years data sets, its evaluation over BPN in parametric forecast is discussed.

Noisy Chaotic time series forecast approximated by combining Reny's entropy with Energy associated to series method: application to rainfall series

This article proposes that the combination of smoothing approach considering the entropic information provided by Renyi's method, has an acceptable performance in term of forecasting errors. The methodology of the proposed scheme is examined through benchmark chaotic time series, such as Mackey Glass, Lorenz, Henon maps, the Lynx and rainfall from Santa Francisca - Cordoba, with addition of white noise by using neural networks-based energy associated (EAS) predictor filter modified by Renyi's entropy of the series. When the time series is short or long, the underlying dynamical system is nonlinear and temporal dependencies span long time intervals, in which this are also called long memory process. In such cases, the inherent nonlinearity of neural networks models and a higher robustness to noise seem to partially explain their better prediction performance when entropic information is extracted from the series. Then, to demonstrate that permutation entropy is computationally efficient, robust to outliers, and effective to measure complexity of time series, computational results are evaluated against several non-linear ANN predictors to show the predictability of noisy rainfall and chaotic time series reported in the literature.

A Multi Time Scale Wind Power Forecasting Model of a Chaotic Echo State Network Based on a Hybrid Algorithm of Particle Swarm Optimization and Tabu Search

The uncertainty and regularity of wind power generation are caused by wind resources’ intermittent and randomness. Such volatility brings severe challenges to the wind power grid. The requirements for ultrashort-term and short-term wind power forecasting with high prediction accuracy of the model used, have great significance for reducing the phenomenon of abandoned wind power , optimizing the conventional power generation plan, adjusting the maintenance schedule and developing real-time monitoring systems. Therefore, accurate forecasting of wind power generation is important in electric load forecasting. The echo state network (ESN) is a new recurrent neural network composed of input, hidden layer and output layers. It can approximate well the nonlinear system and achieves great results in nonlinear chaotic time series forecasting. Besides, the ESN is simpler and less computationally demanding than the traditional neural network training, which provides more accurate training results. Aiming at addressing the disadvantages of standard ESN, this paper has made some improvements. Combined with the complementary advantages of particle swarm optimization and tabu search, the generalization of ESN is improved. To verify the validity and applicability of this method, case studies of multitime scale forecasting of wind power output are carried out to reconstruct the chaotic time series of the actual wind power generation data in a certain region to predict wind power generation. Meanwhile, the influence of seasonal factors on wind power is taken into consideration. Compared with the classical ESN and the conventional Back Propagation (BP) neural network, the results verify the superiority of the proposed method.

Local Functional Coefficient Autoregressive Model for Multistep Prediction of Chaotic Time Series

A new methodology, which combines nonparametric method based on local functional coefficient autoregressive (LFAR) form with chaos theory and regional method, is proposed for multistep prediction of chaotic time series. The objective of this research study is to improve the performance of long-term forecasting of chaotic time series. To obtain the prediction values of chaotic time series, three steps are involved. Firstly, the original time series is reconstructed inm-dimensional phase space with a time delayτby using chaos theory. Secondly, select the nearest neighbor points by using local method in them-dimensional phase space. Thirdly, we use the nearest neighbor points to get a LFAR model. The proposed model’s parameters are selected by modified generalized cross validation (GCV) criterion. Both simulated data (Lorenz and Mackey-Glass systems) and real data (Sunspot time series) are used to illustrate the performance of the proposed methodology. By detailed investigation and comparing our results with published researches, we find that the LFAR model can effectively fit nonlinear characteristics of chaotic time series by using simple structure and has excellent performance for multistep forecasting.

Research on Power Load Forecasting Model Based on Hybrid Algorithm Optimizing BP Neural Network

Short time load forecasting is essential for daily planning and operation of electric power system. It is the important basis for economic dispatching, scheduling and safe operation. Neural network, which has strong nonlinear fitting capability, is widely used in the load forecasting and obtains good prediction effect in nonlinear chaotic time series forecasting. However, the neural network is easy to fall in local optimum, unable to find the global optimal solution. This paper will integrate the traditional optimization algorithm and propose the hybrid intelligent optimization algorithm based on particle swarm optimization algorithm and ant colony optimization algorithm (ACO-PSO) to improve the generalization of the neural network. In the empirical analysis, we select electricity consumption in a certain area for validation. Compared with the traditional BP neutral network and statistical methods, the experimental results demonstrate that the performance of the improved model with more precise results and stronger generalization ability is much better than the traditional methods.

Optimal Selection of Parameters for Nonuniform Embedding of Chaotic Time Series Using Ant Colony Optimization

The optimal selection of parameters for time-delay embedding is crucial to the analysis and the forecasting of chaotic time series. Although various parameter selection techniques have been developed for conventional uniform embedding methods, the study of parameter selection for nonuniform embedding is progressed at a slow pace. In nonuniform embedding, which enables different dimensions to have different time delays, the selection of time delays for different dimensions presents a difficult optimization problem with combinatorial explosion. To solve this problem efficiently, this paper proposes an ant colony optimization (ACO) approach. Taking advantage of the characteristic of incremental solution construction of the ACO, the proposed ACO for nonuniform embedding (ACO-NE) divides the solution construction procedure into two phases, i.e., selection of embedding dimension and selection of time delays. In this way, both the embedding dimension and the time delays can be optimized, along with the search process of the algorithm. To accelerate search speed, we extract useful information from the original time series to define heuristics to guide the search direction of ants. Three geometry- or model-based criteria are used to test the performance of the algorithm. The optimal embeddings found by the algorithm are also applied in time-series forecasting. Experimental results show that the ACO-NE is able to yield good embedding solutions from both the viewpoints of optimization performance and prediction accuracy.

Chaos Optimization Method of SVM Parameters Selection for Chaotic Time Series Forecasting

Abstract For support vector regression (SVR), the setting of key parameters is very important, which determines the regression accuracy and generalization performance of SVR model. In this paper, an optimal selection approach for SVR parameters was put forward based on mutative scale optimization algorithm(MSCOA), the key parameters C and ɛ of SVM and the radial basis kernel parameter g were optimized within the global scopes. The support vector regression model was established for chaotic time series prediction by using the optimum parameters . The time series of Lorenz system was used to testify the effectiveness of the model. The root mean square error of prediction reached RMSE = 3.0335 × 10 −3 . Simulation results show that the optimal selection approach based on MSCOA is an effective approach and the MSCOA-SVR model has a good performance for chaotic time series forecasting.

A Neural Network Scheme for Long-Term Forecasting of Chaotic Time Series

The accuracy of a model to forecast a time series diminishes as the prediction horizon increases, in particular when the prediction is carried out recursively. Such decay is faster when the model is built using data generated by highly dynamic or chaotic systems. This paper presents a topology and training scheme for a novel artificial neural network, named "Hybrid-connected Complex Neural Network" (HCNN), which is able to capture the dynamics embedded in chaotic time series and to predict long horizons of such series. HCNN is composed of small recurrent neural networks, inserted in a structure made of feed-forward and recurrent connections and trained in several stages using the algorithm back-propagation through time (BPTT). In experiments using a Mackey-Glass time series and an electrocardiogram (ECG) as training signals, HCNN was able to output stable chaotic signals, oscillating for periods as long as four times the size of the training signals. The largest local Lyapunov Exponent (LE) of predicted signals was positive (an evidence of chaos), and similar to the LE calculated over the training signals. The magnitudes of peaks in the ECG signal were not accurately predicted, but the predicted signal was similar to the ECG in the rest of its structure.

Study on Grey Forecast Method of Chaotic Time Series

In this paper, the research evolving on chaotic time series forecast is firstly analyzed, and then the grey forecast model of chaotic time series is set up and is applied to chaotic time series forecast to solve the grey forecast problem of chaotic time series.

Training algorithm for radial basis function neural network based on quantum-behaved particle swarm optimization

Radial basis function (RBF) networks are widely applied in function approximation, system identification, chaotic time series forecasting, etc. To use a RBF network, a training algorithm is absolutely necessary for determining the network parameters. The existing training algorithms, such as orthogonal least squares (OLS) algorithm, clustering and gradient descent algorithm, have their own shortcomings respectively. In this paper, we propose a training algorithm based on a novel population-based evolutionary technique, quantum-behaved particle swarm optimization (QPSO), to train RBF neural network. The proposed QPSO-trained RBF network was tested on non-linear system identification problem and chaotic time series forecasting problem, and the results show that it can identify the system and forecast the chaotic time series more quickly and precisely than that trained by the particle swarm algorithm.