Chaotic Time Series Research Articles

Characterising the variability in ocean data using fractal and correlation dimension analysis

Understanding the variability of ocean data is essential for predicting its behaviour in time domain. This study aims to examine the usability of several nonlinear time series analysis techniques for diagnosing the complexity of the underlying dynamics of various ocean data. For such purpose, one-year data from an offshore platform and two-year data from a weather buoy were collected and analysed. The results indicate that all the ocean data considered here are chaotic in nature, reflected by the unfolded structure in the reconstructed phase space. In general, the surface currents speed possess the largest fractal dimension (or lowest Hurst exponent) and correlation dimension, while the significant wave height has the lowest values. This implies that the dynamics associated with surface currents speed tend to be the most complex, which can lead to more random and unpredictable temporal behaviour. The fractal analysis shows that the fluctuation of these ocean data are mostly persistent, and the dependence of fractal properties on window length appears to vary amongst different ocean data. Overall, the results derived from various analysis techniques are consistent, which confirms the potential of using chaotic time series analysis technique for characterizing ocean data variability.

Evolutionary Echo State Network: A neuroevolutionary framework for time series prediction

From one side, Evolutionary Algorithms have enabled enormous progress over the last years in the optimization field. They have been applied to a variety of problems, including optimization of Neural Networks’ architectures. On the other side, the Echo State Network (ESN) model has become increasingly popular in time series prediction, for instance when modeling chaotic sequences. The network has numerous hidden neurons forming a recurrent topology, so-called reservoir, which is fixed during the learning process. Initial reservoir design has mostly been made by human experts; as a consequence, it is prone to errors and bias, and it is a time consuming task.In this paper, we introduce an automatic general neuroevolutionary framework for ESNs, on which we develop a computational tool for evolving reservoirs, called EVOlutionary Echo State Network (EvoESN). To increase efficiency, we represent the large matrix of reservoir weights in the Fourier space, where we perform the evolutionary search strategy. This frequency space has major advantages compared with the original weight space. After updating the Fourier coefficients, we go back to the weight space and perform a conventional training phase for full setting the reservoir architecture. We analyze the evolutionary search employing genetic algorithms and particle swarm optimization, obtaining promising results with the latter over three well-known chaotic time series. The proposed framework leads fast to very good results compared with modern ESN models. Hence, this contribution positions an important family of recurrent systems in the promising neuroevolutionary domain.

A multi-scale transition matrix approach to chaotic time series

There exist rich patterns in nonlinear dynamical processes, but they merge into averages in traditional statistics-based time series analysis. Herein the multi-scale transition matrix is adopted to display the patterns and their evolutions in several typical chaotic systems, including the Logistic Map, the Tent Map, and the Lorentz System. Compared with Markovian processes, there appear rich non-trivial patterns. The unpredictability of transitions matches almost exactly with the Lyapunov exponent. The eigenvalues decay exponentially with respect to the time scale, whose decaying exponents give us the details in the curves of Lyapunov exponent versus dynamical parameters. The evolutionary behaviors differ with each other and do not saturate to the ones for the corresponding shuffled series.

Urban Mobility and Knowledge Extraction from Chaotic Time Series Data: A Comparative Analysis for Uncovering COVID-19 Effects

The COVID-19 pandemic exerted devastating effects on the global economy, public health, and urban life. The geographical consequences lie in spatial time series dimensions, but the chaotic characteristics hinder intuitive understanding. Aiming to discover the variance of urban mobility in the pandemic context, this article presents a hybrid clustering technique called whole time sequence mixture (WTSM). The combination of whole time and subsequence techniques ensures robustness to data volume, dimensionality, sampling, distortion tasks, and a prior constraint. For yellow taxi trips in New York City, the case study spanned the pre-, mid-, and postpandemic periods and compared the performance of the established methods (symbolic aggregate approximation and dynamic time warping) and WTSM techniques. Findings revealed COVID-19 trends and social restriction-induced mobility variations and determined that vaccine supply did not lead to immediate mobility restoration. Meanwhile, concepts of validation indexes and computational complexity corroborate the superiority of WTSM. The transient cluster arising from temporal dissimilarity is a unique finding of WTSM, which led to high cluster cohesion and separation. WTSM can obtain new knowledge and rationale for urgent government intervention and epidemic management, not sacrificing computational efficiency and clustering quality. With the improved storage capability and desire for multidimensional pattern extraction, the increased accessibility to the chaotic time series data sets could promote further studies and help administrative schemes.

Noise resistance of next-generation reservoir computing: a comparative study with high-order correlation computation

Reservoir computing (RC) methods have received more and more attention and applications in chaotic time series prediction with their simple structure and training method. Recently, the next-generation reservoir computing (NG-RC) method has been proposed by Gauthier et al. (Nat Commun 12:5564, 2021) with less training cost and better time series predictions. Nevertheless, in practice, available data on dynamic systems are contaminated with noise. Though NG-RC is shown highly efficient in learning and predicting, its noise resistance captivity is not clear yet, limiting its use in practical problems. In this paper, we study the noise resistance of the NG-RC method, taking the well-known denoising method, the high-order correlation computation (HOCC) method, as a reference. Both methods have similar procedures in respect of function bases and regression processes. With the simple ridge regression method, the NG-RC method has a strong noise resistance for white noise, even better than the HOCC method. Besides, the NG-RC method also shows a good prediction ability for small colored noise, while it does not provide correct reconstruct dynamics. In this paper, other than reconstruction parameters, four numerical indicators are used to check the noise resistance comprehensively, such as the training error, prediction error, prediction time, and auto-correlation prediction error, for both the short-time series and long climate predictions. Our results provide a systematic estimation of NG-RC’s noise resistance capacity, which is helpful for its applications in practical problems.

An entropy-based measure of correlation for time series

In this paper, an information-based measure of association between time series, called information-based correlation coefficient (ICC), is introduced to potentially overcome some of the problems related to Pearson's correlation coefficient and other commonly used coefficients. The ICC, besides its simple definition, seems to have other interesting advantages over traditional correlation coefficients, such as a more natural interpretation in terms of the reciprocal informativeness between two series, higher reliability for independent white noise, Student's t, and AR(1) processes, higher reliability in the presence of spurious correlation and outliers, and a well-developed mathematical theory based on entropy. The ICC can also detect a number of nonlinear relationships, although it may not be equitable and general. Moreover, the ICC can be computed also for ordinal data, and it offers higher reliability for independent ordinal-valued time series. Monte Carlo simulations, applications to real data, properties of the estimator and asymptotics under independence are also discussed in this paper. In particular, the paper includes applications to signal processing, chaotic time series, macroeconomic and weather data.

Effect of temporal resolution on the reproduction of chaotic dynamics via reservoir computing.

Reservoir computing is a machine learning paradigm that uses a structure called a reservoir, which has nonlinearities and short-term memory. In recent years, reservoir computing has expanded to new functions such as the autonomous generation of chaotic time series, as well as time series prediction and classification. Furthermore, novel possibilities have been demonstrated, such as inferring the existence of previously unseen attractors. Sampling, in contrast, has a strong influence on such functions. Sampling is indispensable in a physical reservoir computer that uses an existing physical system as a reservoir because the use of an external digital system for the data input is usually inevitable. This study analyzes the effect of sampling on the ability of reservoir computing to autonomously regenerate chaotic time series. We found, as expected, that excessively coarse sampling degrades the system performance, but also that excessively dense sampling is unsuitable. Based on quantitative indicators that capture the local and global characteristics of attractors, we identify a suitable window of the sampling frequency and discuss its underlying mechanisms.

Multistep-Ahead Chaotic Time Series Prediction Based on Hierarchical Echo State Network With Augmented Random Features

Multistep-ahead chaotic time series prediction is a kind of highly nonlinear problem, which puts forward higher requirements both for the dynamical memory and nonlinearity of the model. Echo state network (ESN) is frequently employed in the realm of chaotic time series modeling and prediction, but the basic ESN has been proved to have an antagonistic trade-off between nonlinear transformation and memory capacity. To overcome this trade-off, a new architecture named hierarchical echo state network with augmented random features (HESN-ARF) is proposed. On the basis of traditional linear random projection, the proposed HESN-ARF further leverages nonlinear kernel transformation to construct augmented random features, which can enable the linear and nonlinear properties to be fully represented. Moreover, the HESN-ARF utilizes low-rank kernel approximation to further reduce the computational cost, preserving the advantage of efficient modeling as much as possible while ensuring the capacities of nonlinear transformation and dynamical memory simultaneously. The proposed HESN-ARF can mine and learn the latent evolution patterns hidden in the dynamic system layer by layer through the hierarchical strategy, and achieves excellent performance in multi-step-ahead chaotic time series prediction, as demonstrated by experimental findings on two synthetic chaotic systems and a real-world meteorological dataset.

Type-2 evolving fuzzy Kalman filter design based on unobservable spectral components space for interval tracking of non-stationary experimental data

In this paper, an interval type-2 evolving fuzzy Kalman filter is designed for processing of unobservable spectral components of uncertain experimental data. The adopted methodology consider the following steps: an initial model of the interval type-2 fuzzy Kalman filter, which is off-line identified from an initial window of the experimental data; the updating of antecedent proposition of interval type-2 fuzzy Kalman filter by using an interval type-2 formulation of evolving Takagi-Sugeno (eTS) clustering algorithm and the updating of consequent proposition by using a type-2 fuzzy formulation of Observer/Kalman Filter Identification (OKID) algorithm, taking into account the multivariable recursive Singular Spectral Analysis of the experimental data. The computational results for tracking the Mackey-Glass chaotic time series illustrate the efficiency of proposed methodology as compared to relevant approaches from literature, and the experimental results for tracking a 2DoF helicopter demonstrate its applicability.

Chaotic Time Series Prediction of Multi‐Dimensional Nonlinear System Based on Bidirectional LSTM Model

AbstractThe current work proposes a hybrid data‐driven model—Convolutional bidirectional long–short term memory (CNN‐BLSTM) for predicting chaotic behavior of three‐coupled Duffing oscillator nonlinear system, in which the CNN is for efficiently extracting the more robust and informative representations of chaotic sequences while the BLSTM is for holding the long‐term dependencies combining the past and future contexts. Different from traditional analytical and numerical approaches, the proposed prediction model features the benefit of focusing on the measured data solely without extensive professional domain knowledge. Additionally, three more recurrent neural network (RNN) models, including simple RNNs, stack LSTMs, and BLSTM, are built and comparisons of generalization performances to the CNN‐BLSTM are conducted. From the findings so far, the CNN‐BLSTM is able to learn the pattern of chaotic time sequence data with less training time and apply the acquired knowledge to the unseen dataset with lower errors. Moreover, the current work decently demonstrates that the proposed model outperforms other three models in terms of stability at different noise levels from two evaluation criteria. The CNN‐BLSTM provides useful guidance for the consideration of predicting multi‐dimensional nonlinear chaotic behavior.

Neural Network Entropy (NNetEn): Entropy-Based EEG Signal and Chaotic Time Series Classification, Python Package for NNetEn Calculation

Entropy measures are effective features for time series classification problems. Traditional entropy measures, such as Shannon entropy, use probability distribution function. However, for the effective separation of time series, new entropy estimation methods are required to characterize the chaotic dynamic of the system. Our concept of Neural Network Entropy (NNetEn) is based on the classification of special datasets in relation to the entropy of the time series recorded in the reservoir of the neural network. NNetEn estimates the chaotic dynamics of time series in an original way and does not take into account probability distribution functions. We propose two new classification metrics: R2 Efficiency and Pearson Efficiency. The efficiency of NNetEn is verified on separation of two chaotic time series of sine mapping using dispersion analysis. For two close dynamic time series (r = 1.1918 and r = 1.2243), the F-ratio has reached the value of 124 and reflects high efficiency of the introduced method in classification problems. The electroencephalography signal classification for healthy persons and patients with Alzheimer disease illustrates the practical application of the NNetEn features. Our computations demonstrate the synergistic effect of increasing classification accuracy when applying traditional entropy measures and the NNetEn concept conjointly. An implementation of the algorithms in Python is presented.

Fingerprint construction of optical transmitters based on the characteristic of electro-optic chaos for secure authentication.

In this paper, an optical transmitter authentication method using hardware fingerprints based on the characteristic of electro-optic chaos is proposed. By means of phase space reconstruction of chaotic time series generated by an electro-optic feedback loop, the largest Lyapunov exponent spectrum (LLES) is defined and used as the hardware fingerprint for secure authentication. The time division multiplexing (TDM) module and the optical temporal encryption (OTE) module are introduced to combine chaotic signal and the message to ensure the security of the fingerprint. Support vector machine (SVM) models are trained to recognize legal and illegal optical transmitters at the receiver. Simulation results show that LLES of chaos has the fingerprint characteristic and is highly sensitive to the time delay of the electro-optic feedback loop. The trained SVM models can distinguish electro-optic chaos generated by different feedback loops with a time delay difference of only 0.03ns and have a good anti-noise ability. Experimental results show that the recognition accuracy of the authentication module based on LLES can reach 98.20% for both legal and illegal transmitters. Our strategy can improve the defense ability of optical networks against active injection attacks and has high flexibility.

Enhanced FPGA implementation of Echo State Networks for chaotic time series prediction

The Echo State Network (ESN) is one of the most used machine learning methods for predicting chaotic time series. The topology of an ESN comprises three layers, namely: input, hidden, and output layer. In this work, different ESN topologies are analyzed by modifying the structure and number of internal connections in the hidden layer, which neurons are connected randomly. The ESN topologies are analyzed to have 2%, 6%, 20%, 40%,…, and up to 100% of internal connections. The prediction error is associated to the Mean Squared Error (MSE), which results have similar values for all ESN topologies, as long as the Echo state property is guaranteed. It is highlighted that the prediction horizon obtained with 2% of internal connections, is more significant than that obtained with 100%, providing a prediction of 80 and 60 steps-ahead, respectively. The prediction results significantly impact the implementation in a field-programmable gate array (FPGA), so that an enhanced hardware realization is introduced herein by reducing the use of multipliers by more than 90% compared with a fully connected topology, without significantly affecting the measure of the prediction horizon and MSE. Another contribution is the FPGA implementation of the activation function that is an hyperbolic tangent function for which different alternatives are shown to minimize the error of the approximated function without significantly increasing the implementation cost or latency.

Online quantum time series processing with random oscillator networks

Reservoir computing is a powerful machine learning paradigm for online time series processing. It has reached state-of-the-art performance in tasks such as chaotic time series prediction and continuous speech recognition thanks to its unique combination of high computational power and low training cost which sets it aside from alternatives such as traditionally trained recurrent neural networks, and furthermore is amenable to implementations in dedicated hardware, potentially leading to extremely compact and efficient reservoir computers. Recently the use of random quantum systems has been proposed, leveraging the complexity of quantum dynamics for classical time series processing. Extracting the output from a quantum system without disturbing its state too much is problematic however, and can be expected to become a bottleneck in such approaches. Here we propose a reservoir computing inspired approach to online processing of time series consisting of quantum information, sidestepping the measurement problem. We illustrate its power by generalizing two paradigmatic benchmark tasks from classical reservoir computing to quantum information and introducing a task without a classical analogue where a random system is trained to both create and distribute entanglement between systems that never directly interact. Finally, we discuss partial generalizations where only the input or only the output time series is quantum.

Chaotic time series wind power prediction method based on OVMD-PE and improved multi-objective state transition algorithm

As the global wind power generation capacity is constantly increasing, the problems of safe operation and utilization after grid connection are becoming more and more prominent. Aiming at the problem of low accuracy and stability of wind power time series with chaotic characteristics, a time series prediction method combining chaotic characteristic processing and neural networks is proposed. First, optimal variational mode decomposition with permutation entropy (OVMD-PE) is used to decompose the original wind power time series and overcome the disadvantage of insufficient mode aliasing encountered by empirical and integrated empirical modes. Second, an improved multi-objective state transition algorithm is proposed to determine the weight coefficients among the neural networks and improve the accuracy of the reconstructed predictive neural networks. Finally, the combined prediction method is used to study and analyze the wind power data from a wind farm in Xinjiang, China, from the perspectives of multiple scenarios and multiple time scales. The experimental results show that OVMD-PE can successfully deal with chaotic characteristics and the improved algorithm has improved the prediction accuracy. Compared with other traditional prediction models, the combined prediction model has higher robustness and stability.

Ordinal pattern-based complexity analysis of high-dimensional chaotic time series.

The ordinal pattern-based complexity-entropy plane is a popular tool in nonlinear dynamics for distinguishing stochastic signals (noise) from deterministic chaos. Its performance, however, has mainly been demonstrated for time series from low-dimensional discrete or continuous dynamical systems. In order to evaluate the usefulness and power of the complexity-entropy (CE) plane approach for data representing high-dimensional chaotic dynamics, we applied this method to time series generated by the Lorenz-96 system, the generalized Hénon map, the Mackey-Glass equation, the Kuramoto-Sivashinsky equation, and to phase-randomized surrogates of these data. We find that both the high-dimensional deterministic time series and the stochastic surrogate data may be located in the same region of the complexity-entropy plane, and their representations show very similar behavior with varying lag and pattern lengths. Therefore, the classification of these data by means of their position in the CE plane can be challenging or even misleading, while surrogate data tests based on (entropy, complexity) yield significant results in most cases.

Reservoir computing based on solitary-like waves dynamics of liquid film flows: A proof of concept

Several theoretical works have shown that solitons —waves that self-maintain constant shape and velocity as they propagate— can be used as a physical computational reservoir, a concept where machine learning algorithms designed for digital computers are replaced by analog physical systems that exhibit nonlinear dynamical behaviour. Here we propose and experimentally validate a novel reservoir computing (RC) system that for the first time employs solitary-like (SL) waves propagating on the surface of a liquid film flowing over an inclined surface. We demonstrate the ability of the SL wave RC system (SLRC) to forecast chaotic time series, also producing experimental evidence of the possibility to combine RC with nonlinear vector autoregression techniques not only in a computer program but also in a physical system.

Optimal Choices of Reconstruction Parameters United Improved C-C Algorithm and Improved G-P Algorithm

For the phase space reconstruction of chaotic time series, the precise selection of delay and embedding dimension is indispensable. Therefore, this paper makes an in-depth study on the selection of reconstruction parameters, and puts forward some suggestions for improving the theoretical defects of the original C-C algorithm and G-P algorithm. The modified C-C algorithm optimizes the intermediate determination parameters of delay time and embedded window of the original algorithm, and makes the selection of delay time and embedded window more accurate. The improved G-P algorithm optimizes the range of neighborhood radius and the selection of scale-free interval, and improves the calculation accuracy of embedding dimension. On the theoretical basis, Lorenz sequence and Rossler sequence are used to verify, and the simulation results can be obtained. The improved algorithm can select parameters more accurately. Finally, the parameters obtained by the two improved algorithms are compared, and it is considered that the phase space reconstruction can be better by using the improved C-C algorithm to solve the delay and the improved G-P algorithm to solve the embedding dimension.

A novel time series prediction method based on pooling compressed sensing echo state network and its application in stock market

In the prediction of time series, the echo state network (ESN) exhibits exclusive strengths and a unique training structure. Based on ESN model, a pooling activation algorithm consisting noise value and adjusted pooling algorithm is proposed to enrich the update strategy of the reservoir layer in ESN. The algorithm optimizes the distribution of reservoir layer nodes. And the nodes set will be more matched to the characteristics of the data. In addition, we introduce a more efficient and accurate compressed sensing technique based on the existing research. The novel compressed sensing technique reduces the amount of spatial computation of methods. The ESN model based on the above two techniques overcomes the limitations in traditional prediction. In the experimental part, the model is validated with different chaotic time series as well as multiple stocks, and the method shows its efficiency and accuracy in prediction.

Intraday Seasonality and Volatility Pattern: An Explanation with Recurrence Quantification Analysis

The Recurrence Quantification Analysis (RQA), a pattern recognition-based time series analysis method, can be successfully utilized for short, nonstationary, nonlinear, and chaotic time series. These RQA measures quantify several properties of time series, including predictability, regularity, stability, randomness, and complexity. In this regard, first, we analyzed the intraday seasonality with RQA and demonstrated how RQA measures change among the intraday periods by using 160 million row matched orders of 100 shares from Borsa Istanbul Equity Market between 2019M10 and 2020M02. We selected 50 stocks from the BIST50 Index group and 50 stocks from outside of the BIST100 Index group. Since these two share groups exhibit similar intraday RQA seasonality, our results are robust. Second, we explained intraday volatility with RQA measures and found a relationship between RQA measures and intraday volatility using a regression model.