Chaotic Time Series Prediction Research Articles

BLSTM convolution and self-attention network enabled recursive and direct prediction for optical chaos.

Chaotic time series prediction has attracted much attention in recent years because of its important applications, such as security analysis for random number generators and chaos synchronization in private communications. Herein, we propose a BLSTM convolution and self-attention network model to predict the optical chaos. We validate the model's capability for direct and recursive prediction, and the model dramatically reduces the accumulation of errors. Moreover, the time duration prediction of optical chaos is increased with comparative accuracy where the predicted sequence length reaches 4 ns with normalized mean squared error (NMSE) of less than 0.01.

A chaotic time series combined prediction model for improving trend lagging

AbstractChaotic time series prediction is a prediction method based on chaos theory, and has important theoretical and application value. At present, most prediction methods only pursue digital fitting and do not consider the directional trend. In addition, using the single model will not achieve better prediction results. Therefore, a chaotic time series combined prediction model for improving trend lagging (ITL) is proposed. An improved dual‐stage attention‐based long short‐term memory model with the improved training objective fuction is designed to solve the trend lagging problem. Then, an auto regressive moving average model with the sliding window is established to mine other characteristics of the time series except nonlinear characteristic. Finally, the idea of optimization algorithm is introduced to construct a time series combined prediction model with high accuracy based on the above two models, so as to perform the chaotic time series prediction from multiple perspectives. Multiple datasets are selected as experimental datasets, and the proposed method is compared with common prediction methods. The results show that the proposed method can achieve single‐step prediction with high accuracy and effectively improve the lagging of chaotic time series prediction. This research can provide theoretical support for the complex chaotic time series prediction.

Experimental Demonstration of Reservoir Computing with Self-Assembled Percolating Networks of Nanoparticles.

The complex self-assembled network of neurons and synapses that comprises the biological brain enables natural information processing with remarkable efficiency. Percolating networks of nanoparticles (PNNs) are complex self-assembled nanoscale systems that have been shown to possess many promising brain-like attributes and which are therefore appealing systems for neuromorphic computation. Here experiments are performed that show that PNNs can be utilized as physical reservoirs within a nanoelectronic reservoir computing framework and demonstrate successful computation for several benchmark tasks (chaotic time series prediction, nonlinear transformation, and memory capacity). For each task, relevant literature results are compiled and it is shown that the performance of the PNNs compares favorably to that previously reported from nanoelectronic reservoirs. It is then demonstrated experimentally that PNNs can be used for spoken digit recognition with state-of-the-art accuracy. Finally, a parallel reservoir architecture is emulated, which increases the dimensionality and richness of the reservoir outputs and results in further improvements in performance across all tasks.

Benchmarking reservoir computing for residential energy demand forecasting

In the energy sector, accurate demand forecasts are vital but often limited by the available computational power. Reservoir computing (RC) or echo-state networks excel in chaotic time series prediction, with lower computational requirements compared to other recurrent network based methods like LSTMs. Next-generation reservoir computing (NG-RC) is a newer, more efficient variant of classical RC originating from nonlinear vector autoregression and therefore missing the randomness of classical RC. In our study, we evaluate RC and NG-RC for day-ahead energy demand predictions on four data sets and compare it to LSTMs and a naive persistence approach. We find that NG-RC outperforms all other methods when considering the root mean squared error on all data sets but struggles with very small or zero demands. Additionally, it offers a very computationally effective hyperparameter optimization and excels in replicating the inherent volatility and the erratic behavior of energy demands.

CNN-LSTM based incremental attention mechanism enabled phase-space reconstruction for chaotic time series prediction

To improve the prediction accuracy of chaotic time series and reconstruct a more reasonable phase space structure of the prediction network, we propose a convolutional neural network (CNN)-long short-term memory (LSTM) prediction model based on the incremental attention mechanism. Firstly, a traversal search is conducted through the traversal layer for finite parameters in the phase space. Then, an incremental attention layer is utilized for parameter judgment based on the dimension weight criteria (DWC). The phase space parameters that best meet DWC are selected and fed into the input layer. Finally, the constructed CNN-LSTM network extracts spatiotemporal features and provides the final prediction results. The model is verified using Logistic, Lorenz, and sunspot chaotic time series, and the performance is compared from the two dimensions of prediction accuracy and network phase space structure. Additionally, the CNN-LSTM network based on incremental attention is compared with LSTM, CNN, recurrent neural network (RNN), and support vector regression (SVR) for prediction accuracy. The experiment results indicate that the proposed composite network model possesses enhanced capability in extracting temporal features and achieves higher prediction accuracy. Also, the algorithm to estimate the phase space parameter is compared with the traditional CAO, false nearest neighbor, and C–C, three typical methods for determining the chaotic phase space parameters. The experiments reveal that the phase space parameter estimation algorithm based on the incremental attention mechanism is superior in prediction accuracy compared with the traditional phase space reconstruction method in five networks, including CNN-LSTM, LSTM, CNN, RNN, and SVR.

Photonic implementation of the input and reservoir layers for a reservoir computing system based on a single VCSEL with two Mach-Zehnder modulators

Hardware implementation of reservoir computing (RC), which could reduce the power consumption of machine learning and significantly enhance data processing speed, holds the potential to develop the next generation of machine learning hardware devices and chips. Due to the existing solution only implementing reservoir layers, the information processing speed of photonics RC system are limited. In this paper, a photonic implementation of a VMM-RC system based on single Vertical Cavity Surface Emitting Laser (VCSEL) with two Mach Zehnder modulators (MZMs) has been proposed. Unlike previous work, both the input and reservoir layers are realized in the optical domain. Additionally, the impact of various mask signals, such as Two-level mask, Six-level mask, and chaos mask signal, employed in system, has been investigated. The system's performance improves with the use of more complex mask(t). The minimum Normalized mean square error (NMSE) can reach 0.0020 (0.0456) for Santa-Fe chaotic time series prediction in simulation (experiment), while the minimum Word Error Rate (WER) can 0.0677 for handwritten digits recognition numerically. The VMM-RC proposed is instrumental in advancing the development of photonic RC by overcoming the long-standing limitations of photonic RC systems in reservoir implementation. Linear matrix computing units (the input layer) and nonlinear computing units (the reservoir layer) are simultaneously implemented in the optical domain, significantly enhancing the information processing speed of photonic RC systems.

A Modified Sine Cosine Algorithm for Numerical Optimization

Sine cosine algorithm (SCA) is a recently developed meta-heuristic method based on the characteristics of sine and cosine functions. However, SCA algorithm may suffer premature convergence in solving complex optimization problems. To mitigate this limitation, a modified SCA (MSCA) is developed to achieve an effective balance between exploration and exploitation. Specifically, the conversion parameter can be configured adaptively by using the individual fitness information in the evolution process to guide the individual search behavior. After that, an inertia weight is embedded in the position updating equation to further improve the search accuracy and accelerate the convergence speed. The performance of MSCA algorithm is investigated on 13 benchmark functions and the chaotic time series prediction problem. Experimental results demonstrate that MSCA algorithm is more effective than the other optimization methods.

Reservoir computing system based on mutually delay-coupled semiconductor lasers with optical feedback

Reservoir Computing (RC), an evolution from Recurrent Neural Networks (RNN), not only represents a unique machine learning paradigm, but also serves as a neuromorphic framework that mirrors the intricate cortical circuits of the human brain. This paper proposes another new photonic RC system based on four basic photonic reservoir computing architectures (single photonic RC system, the parallel photonic RC system, the dual-feedback loop-based photonic RC system and the mutually coupled photonic RC system). System proposed uses optical injection for signal input and retains two parallel responsive semiconductor lasers (R-SLs) with self-feedback loops. Meanwhile, two relatively independent R-SLs are mutually coupled via two coupling lines. The new photonic RC system adds only two sections of fiber compared to the parallel photonic RC system and the mutually coupled photonic RC system. The experiments show that the system proposed has significant advantages on the nonlinear auto regressive moving average series tasks, the chaotic time series prediction tasks and the waveform classification task. More importantly, the memory capacity of system proposed can be adjust by controlling the delay time of the self-feedback loops, so it has higher memory capacity to handle the higher order nonlinear auto regressive moving average tasks (NARMA20 and NARMA30) after optimizing the parameters.

Dynamic adaptive graph convolutional transformer with broad learning system for multi-dimensional chaotic time series prediction

Chaotic time series data is extensively applied in financial stocks, climate monitoring, and sea clutter, in which data fusion from various sources and multi-sensor information make accurate predictions of chaotic time series challenging under complex nonlinear conditions. Previous works focus on designing different model frameworks to capture the temporal dependence and extract richer nonlinear features to improve the accuracy of univariate chaotic time series prediction, which ignores the spatial dependence of multivariable. However, in this paper, we argue that spatial correlation among multiple variables is essential to improve the prediction accuracy of chaotic time series. To fill the gap, we innovatively propose a Dynamic Adaptive Graph Convolutional Transformer with a Broad Learning System (DAGCT-BLS), a GCN and Transformer-based model utilizing multivariate spatial dependence for multi-dimensional chaotic time series forecasting. In DAGCT-BLS, the multivariate chaotic time series are reconstructed into the phase space, and the reconstructed data are rapidly feature-extracted using a cascade network BLS with frozen weights to maximize the retention of chaotic properties and nonlinear relationships. Then, the Dynamic Adaptive Graph Convolutional Network (DAGCN) is proposed to capture the spatial correlation among the multiple variables. Finally, improved multi-head attention of the Transformer Encoder is used to capture the temporal dependence of the phase point sequence. Experiments of our proposed model on three datasets (Lorenz, Rossler, and Sea clutter) show that DAGCT-BLS can achieve the best prediction performance and have strong interpretability, and multivariate-based joint modeling of chaotic time series helps to improve the prediction performance.

Short-term prediction for chaotic time series based on photonic reservoir computing using VCSEL with feedback loop

Short-term prediction for chaotic time series based on photonic reservoir computing using VCSEL with feedback loop

Hybrid parallel photonic reservoir computing with accelerated data processing speed

Reservoir computing is a bio-inspired approach especially suited for processing time-dependent information. In this work, we propose a hybrid reservoir computing scheme based on parallel photonic time-delay reservoirs with optical injection. The state of virtual nodes are evaluated from the combination of temporal outputs of parallel lasers. It is demonstrated that our proposed hybrid scheme enables photonic reservoir computer with shortened time interval θ and less virtual nodes number N, which could break through the limitation of short external cavity length in miniaturization of reservoir computing with photonic integrated circuits. The computational efficiency of photonic reservoir is numerically evaluated by chaotic time-series prediction task and memory capacity test. State-of-the-art speed performances reach 20 GSa/s with prediction error rate below 0.01. Our scheme bridges the areas of photonic integrated circuits and photonic information processing.

CTF-former: A novel simplified multi-task learning strategy for simultaneous multivariate chaotic time series prediction

Multivariate chaotic time series prediction is a challenging task, especially when multiple variables are predicted simultaneously. For multiple related prediction tasks typically require multiple models, however, multiple models are difficult to keep synchronization, making immediate communication between predicted values challenging. Although multi-task learning can be applied to this problem, the principles of allocation and layout options between shared and specific representations are ambiguous. To address this issue, a novel simplified multi-task learning method was proposed for the precise implementation of simultaneous multiple chaotic time series prediction tasks. The scheme proposed consists of a cross-convolution operator designed to capture variable correlations and sequence correlations, and an attention module proposed to capture the information embedded in the sequence structure. In the attention module, a non-linear transformation was implemented with convolution, and its local receptive field and the global dependency of the attention mechanism achieve complementarity. In addition, an attention weight calculation was devised that takes into account not only the synergy of time and frequency domain features, but also the fusion of series and channel information. Notably the scheme proposed a purely simplified design principle of multi-task learning by reducing the specific network to single neuron. The precision of the proposed solution and its potential for engineering applications were verified with the Lorenz system and power consumption. The mean absolute error of the proposed method was reduced by an average of 82.9% in the Lorenz system and 19.83% in power consumption compared to the Gated Recurrent Unit.

Photonic time-delayed reservoir computing based on series-coupled microring resonators with high memory capacity.

On-chip microring resonators (MRRs) have been proposed to construct time-delayed reservoir computing (RC) systems, which offer promising configurations available for computation with high scalability, high-density computing, and easy fabrication. A single MRR, however, is inadequate to provide enough memory for the computation task with diverse memory requirements. Large memory requirements are satisfied by the RC system based on the MRR with optical feedback, but at the expense of its ultralong feedback waveguide. In this paper, a time-delayed RC is proposed by utilizing a silicon-based nonlinear MRR in conjunction with an array of linear MRRs. These linear MRRs possess a high quality factor, providing enough memory capacity for the RC system. We quantitatively analyze and assess the proposed RC structure's performance on three classical tasks with diverse memory requirements, i.e., the Narma 10, Mackey-Glass, and Santa Fe chaotic timeseries prediction tasks. The proposed system exhibits comparable performance to the system based on the MRR with optical feedback, when it comes to handling the Narma 10 task, which requires a significant memory capacity. Nevertheless, the dimension of the former is at least 350 times smaller than the latter. The proposed system lays a good foundation for the scalability and seamless integration of photonic RC.

An encoder–decoder architecture with Fourier attention for chaotic time series multi-step prediction

Multi-step prediction of chaotic time series has been a challenging problem. An encoder–decoder architecture based on novel Fourier attention was proposed, called FGNet, applied to chaotic time series multi-step prediction. Two perspectives were carefully re-considered in modeling, one was the integrated use of frequency and temporal domain information, the other was the fusion of channel information and series information. The encoder was a self-attention architecture with Fourier attention module for capturing dynamic information in time series. Correspondingly, the decoder used the Gated Recurrent Unit (GRU) module with identity mapping, and the decoder was used to model the representation based on temporal dependency. For the interpretability deficiency of Fourier attention in FNet, which was proposed by google, a novel Fourier attention module was proposed. Specifically, the frequency domain features were extracted using Fourier transform, series and channel features were integrated capture through channel swapping coupling. Then these features were fused in the frequency domain space to enhance feature representation and their interpretability is achieved through inverse Fourier transform. Simulated data (Mackey–Glass system and Lorenz) and real data (sunspot time series) were used to verify that FGNet proposed achieves pleasantly surprising multi-step prediction performance in practice.

A data-driven implicit deep adaptive neuro-fuzzy inference system capable of manifold learning for function approximation

Fuzzy Neural Networks (FNN) have the ability of decision-making based on constructing semi-ellipsoidal clusters in the input space as the antecedent parts of their fuzzy rules. To determine the output value for each input instance, FNNs consider its membership degree to different sub-regions of the input space. However, forming such meaningful sub-regions is not possible in all applications due to the nonlinear interactions among input variables and their low information gain. Indeed, the samples could be distributed on a manifold in the input space. Therefore, to cover the input space, we need lots of rules, each representing a small region of input space. This issue decreases the generalization ability of the model along with its explainability. Consequently, to efficiently form fuzzy rules, first, it is necessary to unfold the manifold by mapping the samples to an appropriate embedding space. Next, the fuzzy rules in the form of semi-ellipsoidal regions should be constructed in this extracted feature space. Deep Fuzzy Neural Networks address this problem by representation learning through stacking multiple cascade mapping layers. In this paper, we propose a novel approach for nonlinear function approximation and time-series prediction problems, based on using the kernel trick to implicitly learn the mapping function to the new feature space. Moreover, to initialize the fuzzy rules, a KNN-based method using the kernel trick is proposed. A hierarchical Levenberg–Marquardt approach is applied to learn the model’s parameters. The performance and structure of the proposed method are studied and compared with some other relevant methods in synthetic and real-world benchmarks. Based on these experiments, the proposed method has the best performance with the most parsimonious architecture. According to these experiments, the test RMSE of the proposed method is 0.002 for Mc-Glass chaotic time-series prediction, 0.015 for a Nonlinear dynamic system identification, 0.0345 for Box–Jenkins nonlinear system identification, 0.0609 for Fuel consumption prediction of automobiles, 10.24 for Sydney stock price tracking, and 0.595 for California housing.

A hybrid proper orthogonal decomposition and next generation reservoir computing approach for high-dimensional chaotic prediction: Application to flow-induced vibration of tube bundles.

Chaotic time series prediction is a central science problem in diverse areas, ranging from engineering, economy to nature. Classical chaotic prediction techniques are limited to short-term prediction of low- or moderate-dimensional systems. Chaotic prediction of high-dimensional engineering problems is notoriously challenging. Here, we report a hybrid approach by combining proper orthogonal decomposition (POD) with the recently developed next generation reservoir computing (NGRC) for the chaotic forecasting of high-dimensional systems. The hybrid approach integrates the synergistic features of the POD for model reduction and the high efficiency of NGRC for temporal data analysis, resulting in a new paradigm on data-driven chaotic prediction. We perform the first chaotic prediction of the nonlinear flow-induced vibration (FIV) of loosely supported tube bundles in crossflow. Reducing the FIV of a continuous beam into a 3-degree-of-freedom system using POD modes and training the three time coefficients via a NGRC network with three layers, the hybrid approach can predict time series of a weakly chaotic system with root mean square prediction error less than 1% to 19.3 Lyapunov time, while a three Lyapunov time prediction is still achieved for a highly chaotic system. A comparative study demonstrates that the POD-NGRC outperforms the other existing methods in terms of either predictability or efficiency. The efforts open a new avenue for the chaotic prediction of high-dimensional nonlinear dynamic systems.

Roll Prediction and Parameter Identification of Marine Vessels Under Unknown Ocean Disturbances

Abstract This paper deals with two topics: roll predictions of marine vessels with machine-learning methods and parameter estimation of unknown ocean disturbances when the amplitude, frequency, offset, and phase are difficult to estimate. This paper aims to prevent the risky roll motions of marine vessels exposed to harsh circumstances. First of all, this study demonstrates complex dynamic phenomena by utilising a bifurcation diagram, Lyapunov exponents, and a Poincare section. Without any observers, an adaptive identification applies these four parameters to the globally exponential convergence using linear second-order filters and parameter estimation errors. Then, a backstepping controller is employed to make an exponential convergence of the state variables to zero. Finally, this work presents the prediction of roll motion using reservoir computing (RC). As a result, the RC process shows good performance for chaotic time series prediction in future states. Thus, the poor predictability of Lyapunov exponents may be overcome to a certain extent, with the help of machine learning. Numerical simulations validate the dynamic behaviour and the efficacy of the proposed scheme.

Simplification of ANFIS based on importance-confidence-similarity measures

Adaptive-network-based fuzzy inference system (ANFIS) is a well-known neuro-fuzzy system, which is widely implemented in nonlinear system approximation, prediction, control, and pattern classification. However, either too many fuzzy sets or fuzzy rules are indispensable to establish an accurate ANFIS for a high-dimensional system, which increases the complexity and limits the widespread application of the ANFIS. In this paper, an approach to simplify the ANFIS is developed based on importance-confidence-similarity (ICS) measures. The simplification process consists of selection of fuzzy rules with large importance-confidence (IC) values and reduction of similar fuzzy rules. The importance is defined as the average normalized firing strength of the rule. The confidence which reflects the approximation performance of a T-S fuzzy rule is estimated by the weighted orthogonal distance from the output data to the rule hyperplane (the consequent of a linear T-S fuzzy rule). The IC measures provide a comprehensive basis for the selection of the appropriate fuzzy rules. Furthermore, to the best of our knowledge, an approach to measure similarities of the T-S fuzzy rules including the rule premises and consequents is first proposed in this paper. We also develop a direct and simple method to remove the similar fuzzy rules based on a hierarchical tree and the IC measures. The proposed simplification approach is applied to Mackey-Glass chaotic time series prediction problem and the strip temperature increment prediction problem of the indirect-fired furnace of the annealing heating line. The results show that our proposed method achieves a balance between simplicity and accuracy of the ANFIS for the high-dimensional system.

Reducing reservoir computer hyperparameter dependence by external timescale tailoring

Task specific hyperparameter tuning in reservoir computing is an open issue, and is of particular relevance for hardware implemented reservoirs. We investigate the influence of directly including externally controllable task specific timescales on the performance and hyperparameter sensitivity of reservoir computing approaches. We show that the need for hyperparameter optimisation can be reduced if timescales of the reservoir are tailored to the specific task. Our results are mainly relevant for temporal tasks requiring memory of past inputs, for example chaotic timeseries prediction. We consider various methods of including task specific timescales in the reservoir computing approach and demonstrate the universality of our message by looking at both time-multiplexed and spatially-multiplexed reservoir computing.

Enhanced performances of photonic reservoir computing using a semiconductor laser with random distributed optical feedback

We propose and experimentally demonstrate a photonic time-delay reservoir computing (TDRC) system with random distributed optical feedback under optical injection. To evaluate the performance, we calculate the memory ability and perform two benchmark tasks, i.e., chaotic time series prediction and nonlinear channel equalization task. Our numerical results show that the proposed TDRC has a superior performance compared with the case with conventional single optical feedback. This is attributed to the fact that the random distributed optical feedback offers multiple external cavity modes, which enhance the nonlinearity of the reservoir laser. Additionally, the experimental result also shows that our proposed TDRC scheme outperforms the computer with single optical feedback in the chaotic time series prediction task. To the best of our knowledge, our work offers a novel path to improve the performance of TDRC by introducing random distributed optical feedback.