Chaotic Time Series Prediction Research Articles

Chaotic time series prediction based on selection of important input variables1

Chaotic systems are dynamic systems with aperiodic and pseudo-random properties, and systems in many fields exhibit chaotic time-series properties. Aiming at the fuzzy modeling problem of chaotic time series, this paper proposes a new fuzzy identification method considering the selection of important input variables. The purpose is to achieve higher model modeling and prediction accuracy by constructing a model with a simple structure. The relevant input variable was swiftly chosen in accordance with the input variable index after the Two Stage Fuzzy Curves method was used to determine the weight of the correlation between each input variable and the output from a large number of selectable input variables. The center and width of the irregular Gaussian membership function were then optimized using the fuzzy C-means clustering algorithm and the particle swarm optimization technique, which led to the determination of the fuzzy model’s underlying premise parameters. Finally, the fuzzy model’s conclusion parameters were determined using the recursive least squares method. This model is used to simulate three chaotic time series, and the outcomes of the simulation are contrasted and examined. The outcomes demonstrate that the fuzzy identification system has higher prediction accuracy based on a simpler structure, demonstrating its validity.

On the Optimization of Machine Learning Techniques for Chaotic Time Series Prediction

Interest in chaotic time series prediction has grown in recent years due to its multiple applications in fields such as climate and health. In this work, we summarize the contribution of multiple works that use different machine learning (ML) methods to predict chaotic time series. It is highlighted that the challenge is predicting the larger horizon with low error, and for this task, the majority of authors use datasets generated by chaotic systems such as Lorenz, Rössler and Mackey–Glass. Among the classification and description of different machine learning methods, this work takes as a case study the Echo State Network (ESN) to show that its optimization can lead to enhance the prediction horizon of chaotic time series. Different optimization methods applied to different machine learning ones are given to appreciate that metaheuristics are a good option to optimize an ESN. In this manner, an ESN in closed-loop mode is optimized herein by applying Particle Swarm Optimization. The prediction results of the optimized ESN show an increase of about twice the number of steps ahead, thus highlighting the usefulness of performing an optimization to the hyperparameters of an ML method to increase the prediction horizon.

Parallel and deep reservoir computing using semiconductor lasers with optical feedback

Abstract Photonic reservoir computing has been intensively investigated to solve machine learning tasks effectively. A simple learning procedure of output weights is used for reservoir computing. However, the lack of training of input-node and inter-node connection weights limits the performance of reservoir computing. The use of multiple reservoirs can be a solution to overcome this limitation of reservoir computing. In this study, we investigate parallel and deep configurations of delay-based all-optical reservoir computing using semiconductor lasers with optical feedback by combining multiple reservoirs to improve the performance of reservoir computing. Furthermore, we propose a hybrid configuration to maximize the benefits of parallel and deep reservoirs. We perform the chaotic time-series prediction task, nonlinear channel equalization task, and memory capacity measurement. Then, we compare the performance of single, parallel, deep, and hybrid reservoir configurations. We find that deep reservoirs are suitable for a chaotic time-series prediction task, whereas parallel reservoirs are suitable for a nonlinear channel equalization task. Hybrid reservoirs outperform other configurations for all three tasks. We further optimize the number of reservoirs for each reservoir configuration. Multiple reservoirs show great potential for the improvement of reservoir computing, which in turn can be applied for high-performance edge computing.

Intergroup Cascade Broad Learning System With Optimized Parameters for Chaotic Time Series Prediction

The prediction of chaotic time series can well describe the physical properties and influencing factors of a sequence of numerical data points in successive order and has played an increasingly crucial role in various scientific and engineering communities. These time series data require the prediction model to have strong dynamic feature representation ability, which often leads to excessive calculation burden. As an emerging and effective learning method, a broad learning system (BLS) can be trained quickly by incremental learning, and the system can be reconstructed without requiring a lengthy retraining process, which has been shown to facilitate superior time-saving performance. However, the historical information of time series data is often ignored, resulting in the lack of dynamic feature representation ability of the original BLS algorithm. In this article, a novel intergroup cascade BLS (ICBLS) method and its variant with optimized parameters, termed MOICBLS, are proposed, which can combine the previous knowledge with current information to determine the output results by constructing a novel intergroup cascade structure for the enhancement nodes. The new algorithms try to retain the time-saving advantage of BLS and greatly enhance the extraction ability of dynamic features of chaotic time series data. In addition, it cannot be ignored that the network structure parameters have an unquestionable impact on the performance of a BLS. In response, two conflicting goals of prediction accuracy and diversity are formulated as multiobjective optimization functions in the training phase to generate the optimized network structure parameters needed by multiple candidate models. The experimental results on two benchmark chaotic time series dataset, the Lorenz system and Rossler system, and a geomagnetic disturbance storm time chaotic time series dataset demonstrate that our proposed algorithms are effective for prediction of chaotic time series, and the prediction accuracy is better than some existing approaches.

Modified echo state network for prediction of nonlinear chaotic time series

Modified echo state network for prediction of nonlinear chaotic time series

Reservoir computing based on an external-cavity semiconductor laser with optical feedback modulation.

We numerically and experimentally investigate reservoir computing based on a single semiconductor laser with optical feedback modulation. In this scheme, an input signal is injected into a semiconductor laser via intensity or phase modulation of the optical feedback signal. We perform a chaotic time-series prediction task using the reservoir and compare the performances of intensity and phase modulation schemes. Our results indicate that the feedback signal of the phase modulation scheme outperforms that of the intensity modulation scheme. Further, we investigate the performance dependence of reservoir computing on parameter values and observe that the prediction error improves for large injection currents, unlike the results in a semiconductor laser with an optical injection input. The physical origin of the superior performance of the phase modulation scheme is analyzed using external cavity modes obtained from steady-state analysis in the phase space. The analysis indicates that high-dimensional mapping can be achieved from the input signal to the trajectory of the response laser output by using phase modulation of the feedback signal.

MFRFNN: Multi-Functional Recurrent Fuzzy Neural Network for Chaotic Time Series Prediction

Chaotic time series prediction, a challenging research topic in dynamic system modeling, has drawn great attention from researchers around the world. In recent years extensive researches have been done on developing chaotic time series prediction methods, and various models have been proposed. Among them, recurrent fuzzy neural networks (RFNNs) have shown significant potential in this area. Most of the proposed RFNNs learn a single function, but when dealing with chaotic time series, different outputs may be generated for a specific input based on the system’s state. So, a network is required that can learn multiple functions simultaneously. Based on this concept, a novel multi-functional recurrent fuzzy neural network (MFRFNN) is proposed in this paper. MFRFNN consists of two fuzzy neural networks with Takagi-Sugeno-Kang fuzzy rules, one is used to produce the output, and the other to determine the system’s state. There is a feedback loop between these two networks, which makes MFRFNN capable of learning and memorizing historical information of past observations. Employing the states allows the proposed network to learn multiple functions simultaneously. Moreover, a new learning algorithm, which employs the particle swarm optimization algorithm, is developed to train the networks’ weights. The effectiveness of MFRFNN is validated using the Lorenz and Rossler chaotic time series and four real-world datasets, including Box–Jenkins gas furnace, wind speed prediction, Google stock price prediction, and air quality index prediction. Based on the root mean square error, the proposed method shows a decrease of 35.12%,13.95%, and 49.62% from the second best methods in the Lorenz time series, Box–Jenkins gas furnace, and wind speed prediction dataset, respectively.

On the Prediction of Chaotic Time Series using Neural Networks

Prediction techniques have the challenge of guaranteeing large horizons for chaotic time series. For instance, this paper shows that the majority of techniques can predict one step ahead with relatively low root-mean-square error (RMSE) and Symmetric Mean Absolute Percentage Error (SMAPE). However, some techniques based on neural networks can predict more steps with similar RMSE and SMAPE values. In this manner, this work provides a summary of prediction techniques, including the type of chaotic time series, predicted steps ahead, and the prediction error. Among those techniques, the echo state network (ESN), long short-term memory, artificial neural network and convolutional neural network are compared with similar conditions to predict up to ten steps ahead of Lorenz-chaotic time series. The comparison among these prediction techniques include RMSE and SMAPE values, training and testing times, and required memory in each case. Finally, considering RMSE and SMAPE, with relatively few neurons in the reservoir, the performance comparison shows that an ESN is a good technique to predict five to fifteen steps ahead using thirty neurons and taking the lowest time for the tracking and testing cases.

FD-LSTM: A Fuzzy LSTM Model for Chaotic Time-Series Prediction

The main issue of time-series prediction is to determine the grade of uncertainty in knowledge, with its essential vagueness and haziness in complex problems. In this study, a deep fuzzy long short-term memory (LSTM) architecture has been proposed to handle the high-order uncertainty associated with time-series applications. The LSTM and type-2 fuzzy logic combination aim to make a more transparent, interpretable, and accurate predictive system. The experiments of this study with real data contain global standard and real-valued benchmarks, including Mackey–Glass (MG), sunspot, and English Premier League seasonal datasets. The obtained performance shows the superiority of the proposed fuzzy–deep model in predicting time series with an average AUC = 0.96 in sunspot, 0.93 for football match time series, and 0.95 on a chaotic equation of MG.

Point and interval prediction of crude oil futures prices based on chaos theory and multiobjective slime mold algorithm.

Crude oil is the most important energy source in the world, and fluctuations in oil prices can significantly influence investors, companies, and governments. However, crude oil prices have numerous characteristics, including randomness, sudden structural changes, intrinsic nonlinearity, volatility, and chaotic nature. This makes the accurate forecasting of crude oil prices a difficult and challenging task. In this paper, a hybrid prediction model for crude oil futures prices is proposed, the accuracy and robustness of which are demonstrated via controlled experiments and sensitivity analysis. This study uses a new data denoising method for data processing to improve the accuracy and stability of the predictions of crude oil prices. Furthermore, the chaotic time-series prediction method, shallow neural networks, linear model prediction methods, and deep learning methods are adopted as submodels. The results of interval forecasts with narrow widths and high prediction accuracies are derived by introducing a confidence interval adjustment coefficient. The results of the simulation experiments indicate that the proposed hybrid prediction model exhibits higher accuracy and efficiency, as well as better robustness of the forecasting than the control models. In summary, the proposed forecasting framework can derive accurate point and interval forecasts and provide a valuable reference for the price forecasting of crude oil futures.

Hybrid Short-Term Wind Power Prediction Based on Markov Chain

This article proposes a combined prediction method based on the Markov chain to realize precise short-term wind power predictions. First, three chaotic models are proposed for the prediction of chaotic time series, which can master physical principles in wind power processes and guide long-term prediction. Then, considering a mechanism switching between different physical models via a Markov chain, a combined model is constructed. Finally, the industrial data from a Chinese wind farm were taken as a study case, and the results validated the feasibility and superiority of the proposed prediction method.

High-Speed Reservoir Computing Based on Circular-Side Hexagonal Resonator Microlaser with Optical Feedback

In the current environment of the explosive growth in the amount of information, the demand for efficient information-processing methods has become increasingly urgent. We propose and numerically investigate a delay-based high-speed reservoir computing (RC) using a circular-side hexagonal resonator (CSHR) microlaser with optical feedback and injection. In this RC system, a smaller time interval can be obtained between virtual nodes, and a higher information processing rate (Rinf) can also be achieved, due to the ultra-short photon lifetime and wide bandwidth of the CSHR microlaser. The performance of the RC system was tested with three benchmark tasks (Santa-Fe chaotic time series prediction task, the 10th order Nonlinear Auto Regressive Moving Average task and Nonlinear channel equalization task). The results show that the system achieves high-accuracy prediction, even with a small number of virtual nodes (25), and is more feasible, with lower requirements for arbitrary waveform generators at the same rate. Significantly, at the high rate of 10 Gbps, low error predictions can be achieved over a large parameter space (e.g., frequency detuning in the interval 80 GHz, injected strength in the range of 0.9 variation and 2% range for feedback strength). Interestingly, it has the potential to achieve Rinf of 25 Gbps under technical advancements. Additionally, its shorter external cavity length and cubic micron scale size make it an excellent choice for large-scale photonic integration reservoir computing.

Chaotic time series prediction using DTIGNet based on improved temporal-inception and GRU

To improve the prediction accuracy of chaotic time series, deep extraction of the system evolutionary patterns is a key problem in modeling. In this paper, we propose a deep learning model of automatic multi-scale feature extraction for chaotic time series prediction. A hybrid deep neural network named deep temporal-inception module and gated recurrent unit network (DTIGNet) is designed. The improved temporal-inception module stacks dilated causal convolution of different depth to increase the network adaptability to different scales and improve the network nonlinear characterization ability, and an optional 1 × 1 convolutional kernel as shared residual connection. The model is applied to the Mackey-Glass system, Rossler system, Lorenz system and sunspots time series to verify the applicability and effectiveness in chaotic time series prediction. The results show that the DTIGNet proposed has higher accuracy and better performance compared with other methods according to the 6 prediction evaluation metrics adopted.

Chaotic Time Series Forecasting Approaches Using Machine Learning Techniques: A Review

Traditional statistical, physical, and correlation models for chaotic time series prediction have problems, such as low forecasting accuracy, computational time, and difficulty determining the neural network’s topologies. Over a decade, various researchers have been working with these issues; however, it remains a challenge. Therefore, this review paper presents a comprehensive review of significant research conducted on various approaches for chaotic time series forecasting, using machine learning techniques such as convolutional neural network (CNN), wavelet neural network (WNN), fuzzy neural network (FNN), and long short-term memory (LSTM) in the nonlinear systems aforementioned above. The paper also aims to provide issues of individual forecasting approaches for better understanding and up-to-date knowledge for chaotic time series forecasting. The comprehensive review table summarizes the works closely associated with the mentioned issues. It includes published year, research country, forecasting approach, application, forecasting parameters, performance measures, and collected data area in this sector. Future improvements and current studies in this field are broadly examined. In addition, possible future scopes and limitations are closely discussed.

Dynamic effects on reservoir computing with a Hopf oscillator.

Limit cycle oscillators have the potential to be resourced as reservoir computers due to their rich dynamics. Here, a Hopf oscillator is used as a physical reservoir computer by discarding the delay line and time-multiplexing procedure. A parametric study is used to uncover computational limits imposed by the dynamics of the oscillator using parity and chaotic time-series prediction benchmark tasks. Resonance, frequency ratios from the Farey sequence, and Arnold tongues were found to strongly affect the computation ability of the reservoir. These results provide insights into fabricating physical reservoir computers from limit cycle systems.

Prediction of chaotic time series using recurrent neural networks and reservoir computing techniques: A comparative study

In recent years, machine-learning techniques, particularly deep learning, have outperformed traditional time-series forecasting approaches in many contexts, including univariate and multivariate predictions. This study aims to investigate the capability of (i) gated recurrent neural networks, including long short-term memory (LSTM) and gated recurrent unit (GRU) networks, (ii) reservoir computing (RC) techniques, such as echo state networks (ESNs) and hybrid physics-informed ESNs, and (iii) the nonlinear vector autoregression (NVAR) approach, which has recently been introduced as the next generation RC, for the prediction of chaotic time series and to compare their performance in terms of accuracy, efficiency, and robustness. We apply the methods to predict time series obtained from two widely used chaotic benchmarks, the Mackey–Glass and Lorenz-63 models, as well as two other chaotic datasets representing a bursting neuron and the dynamics of the El Niño Southern Oscillation, and to one experimental dataset representing a time series of cardiac voltage with complex dynamics. We find that even though gated RNN techniques have been successful in forecasting time series generally, they can fall short in predicting chaotic time series for the methods, datasets, and ranges of hyperparameter values considered here. In contrast, for the chaotic datasets studied, we found that reservoir computing and NVAR techniques are more computationally efficient and offer more promise in long-term prediction of chaotic time series.

Processing-Speed Enhancement in a Delay-Laser-Based Reservoir Computer by Optical Injection

A delay-laser-based reservoir computer (RC) usually has its processing speed limited by the transient response of laser dynamics. Here, we study a simple all-optical approach to enhancing the processing speed by introducing optical injection to the reservoir layer of conventional RC that consists of a semiconductor laser with a delay loop. Using optical injection, the laser’s transient response effectively accelerates due to the speeded carrier-photon resonance. In the chaotic time-series prediction task, the proposed RC achieves good performance in a flexible range of injection detuning frequency under sufficient injection rate. Using proper injection parameters, the prediction error is significantly reduced and stabilized when using high processing speed. For achieving a prediction error below 0.006, the optical injection enhances the processing speed by an order of magnitude of about 5 GSample/s. Moreover, the proposed RC extends the advantage to the handwritten digit recognition task by achieving better word error rate.

Rotating neurons for all-analog implementation of cyclic reservoir computing

Hardware implementation in resource-efficient reservoir computing is of great interest for neuromorphic engineering. Recently, various devices have been explored to implement hardware-based reservoirs. However, most studies were mainly focused on the reservoir layer, whereas an end-to-end reservoir architecture has yet to be developed. Here, we propose a versatile method for implementing cyclic reservoirs using rotating elements integrated with signal-driven dynamic neurons, whose equivalence to standard cyclic reservoir algorithm is mathematically proven. Simulations show that the rotating neuron reservoir achieves record-low errors in a nonlinear system approximation benchmark. Furthermore, a hardware prototype was developed for near-sensor computing, chaotic time-series prediction and handwriting classification. By integrating a memristor array as a fully-connected output layer, the all-analog reservoir computing system achieves 94.0% accuracy, while simulation shows >1000× lower system-level power than prior works. Therefore, our work demonstrates an elegant rotation-based architecture that explores hardware physics as computational resources for high-performance reservoir computing.

Wind Power Prediction Based on a Hybrid Granular Chaotic Time Series Model

For realizing high-accuracy short-term wind power prediction, a hybrid model considering physical features of data is proposed in this paper, with consideration of chaotic analysis and granular computing. First, considering the chaotic features of wind power time series physically, data reconstruction in chaotic phase space is studied to provide a low-dimensional input with more information in modeling. Second, considering that meteorological scenarios of wind development are various, complicated, and uncertain, typical chaotic time series prediction models and wind scenarios are analyzed correspondingly via granular computing (GrC). Finally, through granular rule-based modeling, a hybrid model combining reconstructed wind power data and different models is constructed for short-term wind power prediction. Data from real wind farms is taken for experiments, validating the feasibility and effectiveness of the proposed wind power prediction model.

Prediction of chaotic time series based on Nyström Cauchy kernel conjugate gradient algorithm

Chaotic time series can well reflect the nonlinearity and non-stationarity of real environment changes. The traditional kernel adaptive filter (KAF) with second-order statistical characteristics suffers performance degeneration dramatically for predicting chaotic time series containing noises and outliers. In order to improve the robustness of adaptive filters in the presence of impulsive noise, a nonlinear similarity measure named Cauchy kernel loss (CKL) is proposed, and the global convexity of CKL is guaranteed by the half-quadratic (HQ) method. To improve the convergence rate of stochastic gradient descent and avoid a local optimum simultaneously, the conjugate gradient (CG) method is used to optimize CKL. Furthermore, to address the issue of kernel matrix network growth, the Nyström sparse strategy is adopted to approximate the kernel matrix and then the probability density rank-based quantization (PRQ) is used to improve the approximation accuracy. To this end, a novel Nyström Cauchy kernel conjugate gradient with PRQ (NCKCG-PRQ) algorithm is proposed for the prediction of chaotic time series in this paper. Simulations on prediction of synthetic and real-world chaotic time series validate the advantages of the proposed algorithm in terms of filtering accuracy, robustness, and computational storage complexity.