Multivariate Chaotic Time Series Research Articles

Dynamic adaptive graph convolutional transformer with broad learning system for multi-dimensional chaotic time series prediction

Chaotic time series data is extensively applied in financial stocks, climate monitoring, and sea clutter, in which data fusion from various sources and multi-sensor information make accurate predictions of chaotic time series challenging under complex nonlinear conditions. Previous works focus on designing different model frameworks to capture the temporal dependence and extract richer nonlinear features to improve the accuracy of univariate chaotic time series prediction, which ignores the spatial dependence of multivariable. However, in this paper, we argue that spatial correlation among multiple variables is essential to improve the prediction accuracy of chaotic time series. To fill the gap, we innovatively propose a Dynamic Adaptive Graph Convolutional Transformer with a Broad Learning System (DAGCT-BLS), a GCN and Transformer-based model utilizing multivariate spatial dependence for multi-dimensional chaotic time series forecasting. In DAGCT-BLS, the multivariate chaotic time series are reconstructed into the phase space, and the reconstructed data are rapidly feature-extracted using a cascade network BLS with frozen weights to maximize the retention of chaotic properties and nonlinear relationships. Then, the Dynamic Adaptive Graph Convolutional Network (DAGCN) is proposed to capture the spatial correlation among the multiple variables. Finally, improved multi-head attention of the Transformer Encoder is used to capture the temporal dependence of the phase point sequence. Experiments of our proposed model on three datasets (Lorenz, Rossler, and Sea clutter) show that DAGCT-BLS can achieve the best prediction performance and have strong interpretability, and multivariate-based joint modeling of chaotic time series helps to improve the prediction performance.

CTF-former: A novel simplified multi-task learning strategy for simultaneous multivariate chaotic time series prediction

Multivariate chaotic time series prediction is a challenging task, especially when multiple variables are predicted simultaneously. For multiple related prediction tasks typically require multiple models, however, multiple models are difficult to keep synchronization, making immediate communication between predicted values challenging. Although multi-task learning can be applied to this problem, the principles of allocation and layout options between shared and specific representations are ambiguous. To address this issue, a novel simplified multi-task learning method was proposed for the precise implementation of simultaneous multiple chaotic time series prediction tasks. The scheme proposed consists of a cross-convolution operator designed to capture variable correlations and sequence correlations, and an attention module proposed to capture the information embedded in the sequence structure. In the attention module, a non-linear transformation was implemented with convolution, and its local receptive field and the global dependency of the attention mechanism achieve complementarity. In addition, an attention weight calculation was devised that takes into account not only the synergy of time and frequency domain features, but also the fusion of series and channel information. Notably the scheme proposed a purely simplified design principle of multi-task learning by reducing the specific network to single neuron. The precision of the proposed solution and its potential for engineering applications were verified with the Lorenz system and power consumption. The mean absolute error of the proposed method was reduced by an average of 82.9% in the Lorenz system and 19.83% in power consumption compared to the Gated Recurrent Unit.

Mixformer: An improved self-attention architecture applied to multivariate chaotic time series prediction

Multivariate chaotic time series prediction has been a challenging problem, especially the coupling relationship between multiple variables need to be carefully considered. We propose a self-attention architecture with an information interaction module, called mixformer, applied to the multivariate chaotic time series prediction task. First, Based on rethinking the multivariate data structure, we compensate the deficiency of phase space reconstruction with a group of cross-convolution operator that can automatically update parameters, and propose an explicitly designed feature reconstruction module. Then, to address the problem of interactive fusion of series information and channel information, we propose an information interaction module that enables the feature communication by expanding and contracting dimensions. By breaking the communication barrier between series information and channel information, the feature representation capability is enhanced. Finally, we construct mixformer that combines locally sparse features and global context features. And notably, mixformer integrates the information interaction module and the feature reconstruction module to form a continuous solution. By comparing the results with existing models using multiple simulated data (Lorenz, Chen, and Rossler systems) and real-world application (power consumption), it is verified that our proposed model achieves surprising performance in practice.

A hybrid model of stacked autoencoder and modified particle swarm optimization for multivariate chaotic time series forecasting

Phase space reconstruction (PSR) is an effective method for chaotic system modeling, which can reveal the implicit evolution information in a complex system. However, the reconstructed time series tend to have a high dimension and contain some redundant information. It is difficult for a traditional simple model to directly forecast the reconstructed time series. In this paper, we propose a hybrid model of stacked autoencoder (SAE) and modified particle swarm optimization (MPSO) for multivariate chaotic time series forecasting. We utilize SAE to extract the reconstructed time series and adopt feedforward neural network (FNN) to forecast time series. In the proposed hybrid model, the SAE is followed by FNN, and we make the MPSO to train the output weights of the model, which is a large-scale optimization problem. To enhance the generalization ability and prevent over-fitting, we add a regularization item to the objective function when MPSO trains the weights of the model. Experimental results show that MPSO algorithm has advantages in the exploration and exploitation in large-scale optimization problems. Then, experiments on Lorenz time series and two real-world time series datasets verify the effectiveness of the hybrid model in multivariate chaotic time series forecasting.

Learning Both Dynamic-Shared and Dynamic-Specific Patterns for Chaotic Time-Series Prediction.

In the real world, multivariate time series from the dynamical system are correlated with deterministic relationships. Analyzing them dividedly instead of utilizing the shared-pattern of the dynamical system is time consuming and cumbersome. Multitask learning (MTL) is an effective inductive bias method to utilize latent shared features and discover the structural relationships from related tasks. Base on this concept, we propose a novel MTL model for multivariate chaotic time-series prediction, which could learn both dynamic-shared and dynamic-specific patterns. We implement the dynamic analysis of multiple time series through a special network structure design. The model could disentangle the complex relationships among multivariate chaotic time series and derive the common evolutionary trend of the multivariate chaotic dynamical system by inductive bias. We also develop an efficient Crank-Nicolson-like curvilinear update algorithm based on the alternating direction method of multipliers (ADMM) for the nonconvex nonsmooth Stiefel optimization problem. Simulation results and analysis demonstrate the effectiveness on dynamic-shared pattern discovery and prediction performance.

Prediction of heavy metal content in multivariate chaotic time series based on LSTM

Prediction of heavy metal content in multivariate chaotic time series based on LSTM

Multivariate Chaotic Time Series Prediction Based on Improved Grey Relational Analysis

In multivariate chaotic time series prediction, correlation analysis is important for reducing input dimensions and improving prediction performance. Grey relational analysis (GRA) has proved to be an effective method for data correlation analysis, especially for inexact data and incomplete data. In GRA, points are usually regarded as objects, and the distance between points or the concave and convex degree are mostly used to measure the correlations. However, with discrete variables, correlation analysis results always tend to have some deviations when using prior GRA methods. Furthermore, GRA methods cannot directly use vector datasets. Therefore, in this paper, an improved GRA method is proposed based on vector projections. The input and output variables are expressed as vectors by linking two adjacent points. The vectors, instants of the points, are regarded as the objects, and the projection length of input variables to output variables is used to measure the correlations. The smaller the difference between the projection length and the input variables, the higher the correlation. Then, a hybrid variable selection and prediction model is proposed based on the improved GRA method for multivariate chaotic time series predictions, in order to overcome the negative effects of irrelevant and redundant variables caused by phase-space reconstruction. The experimental results based on the gas furnace dataset and San Francisco river runoff dataset demonstrate that the improved GRA method is effective for data correlation analysis, and the prediction accuracy is better than prior GRA-based methods.

Nonlinear dynamics model for social popularity prediction based on multivariate chaotic time series

In online social network, it is of great significance in the popularity perception of social topics. Aiming to deeply analyze the nonlinear dynamics mechanism in the propagation process of social topics, the chaotic characteristics of information dissemination are thoroughly explored in this paper, and Bayesian estimation theory is employed to integrate the cross-platform factors affecting the topic popularity. Firstly, the time series of topic popularity from different platforms are redefined. And Principal Component Analysis (PCA) method is introduced to quantize the redefined time series and receive the main components affecting popularity. Secondly, this paper focuses on specific topics and performs small-data method to calculate the maximum Lyapunov exponent, which can be found that there are chaotic characteristics in time series of topic popularity. Then, the main components affecting popularity are reconstructed based on the phase space reconstruction theory, and the chaotic attractor is recovered in the high-dimensional reconstructed phase space. Smoothly, the evolution regularities and properties of complex system are restored to perceive the propagating situation of topic popularity. At the same time, the novel and fused phase space is obtained by using the Bayesian estimation theory to optimally fuse multiple variables in the same high-dimensional space. Finally, RBF (Radial Basis Function) neural network is employed to predict the fused popularity due to its strong ability to approximate nonlinear functions. Experiments show that the prediction approach can not only merge topic popularity from multiple social platforms, but also can improve the prediction accuracy to some degree. In the meanwhile, the developing trend of topics can be perceived from a more fine-grained perspective.

Application research on PM2.5 concentration prediction of multivariate chaotic time series

PM2.5 is affected by complex factors such as meteorological elements in the air system and other pollutants in the air. So PM2.5 has chaotic property, which makes the prediction of PM2.5 concentration extremely difficult. In order to improve the prediction accuracy of PM2.5 concentration, this paper introduces the chaotic time series prediction method to establish multivariate time for PM2.5 concentration. The sequence prediction model achieves short-term predictions based on the hour concentration of PM2.5 in Beijing. Firstly, the chaotic time series phase space of the relevant unit is expanded into the multi-time sequence phase space, and the multi-time sequence phase space matrix of PM2.5 concentration is constructed. Then the RBF neural network is used to predict the state point in the multi-phase space system. The phase space points of the PM2.5 concentration sequence is separated for prediction. Finally, the comparison between the prediction model and the traditional prediction model is carried out. The results show that the root mean square error of the predicted PM2.5 concentration in the multivariate chaotic time series prediction model based on the phase space reconstruction is 4.92% in the next 5 hours. The average absolute error is 2.40%, which is more effective than the commonly used statistical prediction method.

Hybrid Regularized Echo State Network for Multivariate Chaotic Time Series Prediction.

Multivariate chaotic time series prediction is a hot research topic, the goal of which is to predict the future of the time series based on past observations. Echo state networks (ESNs) have recently been widely used in time series prediction, but there may be an ill-posed problem for a large number of unknown output weights. To solve this problem, we propose a hybrid regularized ESN, which employs a sparse regression with the L1/2 regularization and the L2 regularization to compute the output weights. The L1/2 penalty shows many attractive properties, such as unbiasedness and sparsity. The L2 penalty presents appealing ability on shrinking the amplitude of the output weights. After the output weights are calculated, the input weights, internal weights, and output weights are fine-tuning by a Hessian-free optimization method-conjugate gradient backpropagation algorithm. The fine-tuning helps to bubble up the input information toward the output layer. Besides, the largest Lyapunov exponent is used to calculate the predictable horizon of a chaotic time series. Experimental results on benchmark and real-world datasets show that our proposed method is superior to other ESN-based models, as sparser, smaller-absolute-value, and more informative output weights are obtained. All of the predictions within the predictable horizon of the proposed model are accurate.

Nonuniform State Space Reconstruction for Multivariate Chaotic Time Series.

State space reconstruction is the foundation of chaotic system modeling. Selection of reconstructed variables is essential to the analysis and prediction of multivariate chaotic time series. As most existing state space reconstruction theorems deal with univariate time series, we have presented a novel nonuniform state space reconstruction method using information criterion for multivariate chaotic time series. We derived a new criterion based on low dimensional approximation of joint mutual information for time delay selection, which can be solved efficiently through the use of an intelligent optimization algorithm with low computation complexity. The embedding dimension is determined by conditional entropy, after which the reconstructed variables have relatively strong independence and low redundancy. The scheme, which integrates nonuniform embedding and feature selection, results in better reconstructions for multivariate chaotic systems. Moreover, the proposed nonuniform state space reconstruction method shows good performance in forecasting benchmark and actual multivariate chaotic time series.

Multivariate Chaotic Time Series Online Prediction Based on Improved Kernel Recursive Least Squares Algorithm.

Kernel recursive least squares (KRLS) is a kind of kernel methods, which has attracted wide attention in the research of time series online prediction. It has low computational complexity and updates in a recursive form. However, as data size increases, computational complexity of calculating kernel inverse matrix will raise. And it has some difficulties in accommodating time-varying environments. Therefore, we have presented an improved KRLS algorithm for multivariate chaotic time series online prediction. Approximate linear dependency, dynamic adjustment, and coherence criterion are combined with quantization to form our improved KRLS algorithm. In the process of online prediction, it can bring computational efficiency up and adjust weights adaptively in time-varying environments. Moreover, Lorenz chaotic time series, El Nino-Southern Oscillation indexes chaotic time series, yearly sunspots and runoff of the Yellow River chaotic time series online prediction are presented to prove the effectiveness of our proposed algorithm.

Multivariate Chaotic Time Series Prediction Based on ELM–PLSR and Hybrid Variable Selection Algorithm

In this paper, a novel method (Hybrid–ELM–PLSR) is proposed based on hybrid variable selection algorithm and improved extreme learning machine (ELM) for multivariate chaotic time series prediction. The hybrid variable selection algorithm combines the advantages of filter and wrapper, effectively balancing the calculation speed and prediction accuracy. Moreover, for ELM, multicollinearity, which can result in ill-condition, is always existent among the hidden layer output matrix. And the optimal number of hidden nodes is also difficult to be determined. Therefore,in order to overcome these problems, an improved ELM (ELM–PLSR) is proposed based on partial least square regression (PLSR). It can effectively enhance the stability performance and prediction performance of ELM. Hybrid–ELM–PLSR can be divided into three stages. At first, filter is used to rearrange the input variables through the correlations with desired variables. Then wrapper is used to select the optimal variable subset through evaluating the prediction performance of different subsets. Finally, ELM–PLSR is used to build the prediction model. The simulation experiment results based on San Francisco river runoff dataset demonstrate that the proposed method is effective for multivariate chaotic time series. And the prediction accuracy and reliability are higher than other methods.

Soft Measurement Modeling Based on Chaos Theory for Biochemical Oxygen Demand (BOD)

The precision of soft measurement for biochemical oxygen demand (BOD) is always restricted due to various factors in the wastewater treatment plant (WWTP). To solve this problem, a new soft measurement modeling method based on chaos theory is proposed and is applied to BOD measurement in this paper. Phase space reconstruction (PSR) based on Takens embedding theorem is used to extract more information from the limited datasets of the chaotic system. The WWTP is first testified as a chaotic system by the correlation dimension (D), the largest Lyapunov exponents (λ1), the Kolmogorov entropy (K) of the BOD and other water quality parameters time series. Multivariate chaotic time series modeling method with principal component analysis (PCA) and artificial neural network (ANN) is then adopted to estimate the value of the effluent BOD. Simulation results show that the proposed approach has higher accuracy and better prediction ability than the corresponding modeling approaches not based on chaos theory.

Multivariate chaotic time series prediction using multiple kernel extreme learning machine

Multivariate chaotic time series is widely present in nature, such as in economy, society, industry and other fields. Modeling and predicting multivariate time series will help human to better manage, control, and make decision. A prediction method based on multiple kernel extreme learning machine is proposed in this paper to model the complex dynamics of multivariate chaotic time series. First, the multivariate chaotic time series is reconstructed in phase space, transforming the temporal correlation into spatial correlation. Then, a prediction model-multiple kernel extreme learning machine, which combines the multiple kernel learning and extreme learning machine with kernels, is proposed to approximate the nonlinear function of the input - output data in phase space. The proposed multiple kernel extreme learning machine could effectively combine the simple training of extreme learning machine with kernels and the data fusion capabilities of multiple kernel learning. Simulation results based on Lorenz chaotic time series prediction and San Francisco monthly runoff prediction demonstrate that, compared with other state-of-art chaotic time series prediction methods, the proposed multiple kernel extreme learning machine could get a better prediction accuracy.

Prediction of multivariate chaotic time series via radial basis function neural network

AbstractIn this article, a new multivariate radial basis functions neural network model is proposed to predict the complex chaotic time series. To realize the reconstruction of phase space, we apply the mutual information method and false nearest‐neighbor method to obtain the crucial parameters time delay and embedding dimension, respectively, and then expand into the multivariate situation. We also proposed two the objective evaluations, mean absolute error and prediction mean square error, to evaluate the prediction accuracy. To illustrate the prediction model, we use two coupled Rossler systems as examples to do simultaneously single‐step prediction and multistep prediction, and find that the evaluation performances and prediction accuracy can achieve an excellent magnitude. © 2013 Wiley Periodicals, Inc. Complexity, 2013.

A hybrid prediction model of multivariate chaotic time series based on error correction

Considering the problem that simply modifying the reservoir algorithm cannot significantly improve the prediction accuracy of chaotic multivariate time series, in this paper we propose a hybrid prediction model based on error correction. The observed data includes both linear and nonlinear features. First, we use autoregressive and moving average model to capture the linear features, then build a regularized echo state network to portray the dynamic nonlinear features. Finally, we add the predicted nonlinear value to the predicted linear value, in order to improve forecasting accuracy achieved by either of the models used separately. The experimental results of Lorenz and Sunspot-Runoff in the Yellow River time series demonstrate the effectiveness and characteristics of the proposed model herein.

Multivariate chaotic time series prediction based on extreme learning machine

For multivariate chaotic time series prediction problem, a prediction based on input variable selection and extreme learning machine is proposed in this paper. The multivariate chaotic time series is reconstructed in phase space, and a mutual information based method is used to select the input variables, which have high statistics information with the output variables. The extreme learning machine is conducted to model the multivariate chaotic time series in the phase space by utilizing its approximation capability. In order to improve the prediction accuracy, a model selection algorithm is conducted for extreme learning machine to choose an expected minimum risk prediction model. Simulation results based on Lorenz, Rssler multivariate chaotic time series and Rssler hyperchaotic time series show the effectiveness of the proposed method.

Multivariate chaotic time series phase space reconstruction based on extending dimension by conditional entropy

For multivariate chaotic time series, a method of conditional entropy extending dimension(CEED) in the reconstructed phase space is proposed. First, the delay time of any variable time series is selected by mutual information method, and then the embedding dimension of phase space is extended by the conditional entropy. This method can ensure the independence of reconstructed coordinates from low space to high space and eliminate the redundancy of phase space, because the largest condition entropy is choosen. The effective input vector for the prediction of multivariate time series is given. Simulations of the Lorenz system and Henon system show that the neural network predictions of multivariate time series are much better than the prediction of univariate and existing multivariate. Therefore, CEED is effective for multivariate chaotic systems.

Forecasting Model of Multivariate Chaotic Financial Time Series Based on DFNN

Forecasting Model of Multivariate Chaotic Financial Time Series Based on DFNN