Multivariate Chaotic Time Series Prediction Research Articles

CTF-former: A novel simplified multi-task learning strategy for simultaneous multivariate chaotic time series prediction

Multivariate chaotic time series prediction is a challenging task, especially when multiple variables are predicted simultaneously. For multiple related prediction tasks typically require multiple models, however, multiple models are difficult to keep synchronization, making immediate communication between predicted values challenging. Although multi-task learning can be applied to this problem, the principles of allocation and layout options between shared and specific representations are ambiguous. To address this issue, a novel simplified multi-task learning method was proposed for the precise implementation of simultaneous multiple chaotic time series prediction tasks. The scheme proposed consists of a cross-convolution operator designed to capture variable correlations and sequence correlations, and an attention module proposed to capture the information embedded in the sequence structure. In the attention module, a non-linear transformation was implemented with convolution, and its local receptive field and the global dependency of the attention mechanism achieve complementarity. In addition, an attention weight calculation was devised that takes into account not only the synergy of time and frequency domain features, but also the fusion of series and channel information. Notably the scheme proposed a purely simplified design principle of multi-task learning by reducing the specific network to single neuron. The precision of the proposed solution and its potential for engineering applications were verified with the Lorenz system and power consumption. The mean absolute error of the proposed method was reduced by an average of 82.9% in the Lorenz system and 19.83% in power consumption compared to the Gated Recurrent Unit.

Mixformer: An improved self-attention architecture applied to multivariate chaotic time series prediction

Multivariate chaotic time series prediction has been a challenging problem, especially the coupling relationship between multiple variables need to be carefully considered. We propose a self-attention architecture with an information interaction module, called mixformer, applied to the multivariate chaotic time series prediction task. First, Based on rethinking the multivariate data structure, we compensate the deficiency of phase space reconstruction with a group of cross-convolution operator that can automatically update parameters, and propose an explicitly designed feature reconstruction module. Then, to address the problem of interactive fusion of series information and channel information, we propose an information interaction module that enables the feature communication by expanding and contracting dimensions. By breaking the communication barrier between series information and channel information, the feature representation capability is enhanced. Finally, we construct mixformer that combines locally sparse features and global context features. And notably, mixformer integrates the information interaction module and the feature reconstruction module to form a continuous solution. By comparing the results with existing models using multiple simulated data (Lorenz, Chen, and Rossler systems) and real-world application (power consumption), it is verified that our proposed model achieves surprising performance in practice.

Learning Both Dynamic-Shared and Dynamic-Specific Patterns for Chaotic Time-Series Prediction.

In the real world, multivariate time series from the dynamical system are correlated with deterministic relationships. Analyzing them dividedly instead of utilizing the shared-pattern of the dynamical system is time consuming and cumbersome. Multitask learning (MTL) is an effective inductive bias method to utilize latent shared features and discover the structural relationships from related tasks. Base on this concept, we propose a novel MTL model for multivariate chaotic time-series prediction, which could learn both dynamic-shared and dynamic-specific patterns. We implement the dynamic analysis of multiple time series through a special network structure design. The model could disentangle the complex relationships among multivariate chaotic time series and derive the common evolutionary trend of the multivariate chaotic dynamical system by inductive bias. We also develop an efficient Crank-Nicolson-like curvilinear update algorithm based on the alternating direction method of multipliers (ADMM) for the nonconvex nonsmooth Stiefel optimization problem. Simulation results and analysis demonstrate the effectiveness on dynamic-shared pattern discovery and prediction performance.

Multivariate Chaotic Time Series Prediction Based on Improved Grey Relational Analysis

In multivariate chaotic time series prediction, correlation analysis is important for reducing input dimensions and improving prediction performance. Grey relational analysis (GRA) has proved to be an effective method for data correlation analysis, especially for inexact data and incomplete data. In GRA, points are usually regarded as objects, and the distance between points or the concave and convex degree are mostly used to measure the correlations. However, with discrete variables, correlation analysis results always tend to have some deviations when using prior GRA methods. Furthermore, GRA methods cannot directly use vector datasets. Therefore, in this paper, an improved GRA method is proposed based on vector projections. The input and output variables are expressed as vectors by linking two adjacent points. The vectors, instants of the points, are regarded as the objects, and the projection length of input variables to output variables is used to measure the correlations. The smaller the difference between the projection length and the input variables, the higher the correlation. Then, a hybrid variable selection and prediction model is proposed based on the improved GRA method for multivariate chaotic time series predictions, in order to overcome the negative effects of irrelevant and redundant variables caused by phase-space reconstruction. The experimental results based on the gas furnace dataset and San Francisco river runoff dataset demonstrate that the improved GRA method is effective for data correlation analysis, and the prediction accuracy is better than prior GRA-based methods.

Application research on PM2.5 concentration prediction of multivariate chaotic time series

PM2.5 is affected by complex factors such as meteorological elements in the air system and other pollutants in the air. So PM2.5 has chaotic property, which makes the prediction of PM2.5 concentration extremely difficult. In order to improve the prediction accuracy of PM2.5 concentration, this paper introduces the chaotic time series prediction method to establish multivariate time for PM2.5 concentration. The sequence prediction model achieves short-term predictions based on the hour concentration of PM2.5 in Beijing. Firstly, the chaotic time series phase space of the relevant unit is expanded into the multi-time sequence phase space, and the multi-time sequence phase space matrix of PM2.5 concentration is constructed. Then the RBF neural network is used to predict the state point in the multi-phase space system. The phase space points of the PM2.5 concentration sequence is separated for prediction. Finally, the comparison between the prediction model and the traditional prediction model is carried out. The results show that the root mean square error of the predicted PM2.5 concentration in the multivariate chaotic time series prediction model based on the phase space reconstruction is 4.92% in the next 5 hours. The average absolute error is 2.40%, which is more effective than the commonly used statistical prediction method.

Hybrid Regularized Echo State Network for Multivariate Chaotic Time Series Prediction.

Multivariate chaotic time series prediction is a hot research topic, the goal of which is to predict the future of the time series based on past observations. Echo state networks (ESNs) have recently been widely used in time series prediction, but there may be an ill-posed problem for a large number of unknown output weights. To solve this problem, we propose a hybrid regularized ESN, which employs a sparse regression with the L1/2 regularization and the L2 regularization to compute the output weights. The L1/2 penalty shows many attractive properties, such as unbiasedness and sparsity. The L2 penalty presents appealing ability on shrinking the amplitude of the output weights. After the output weights are calculated, the input weights, internal weights, and output weights are fine-tuning by a Hessian-free optimization method-conjugate gradient backpropagation algorithm. The fine-tuning helps to bubble up the input information toward the output layer. Besides, the largest Lyapunov exponent is used to calculate the predictable horizon of a chaotic time series. Experimental results on benchmark and real-world datasets show that our proposed method is superior to other ESN-based models, as sparser, smaller-absolute-value, and more informative output weights are obtained. All of the predictions within the predictable horizon of the proposed model are accurate.

Multivariate Chaotic Time Series Prediction Based on ELM–PLSR and Hybrid Variable Selection Algorithm

In this paper, a novel method (Hybrid–ELM–PLSR) is proposed based on hybrid variable selection algorithm and improved extreme learning machine (ELM) for multivariate chaotic time series prediction. The hybrid variable selection algorithm combines the advantages of filter and wrapper, effectively balancing the calculation speed and prediction accuracy. Moreover, for ELM, multicollinearity, which can result in ill-condition, is always existent among the hidden layer output matrix. And the optimal number of hidden nodes is also difficult to be determined. Therefore,in order to overcome these problems, an improved ELM (ELM–PLSR) is proposed based on partial least square regression (PLSR). It can effectively enhance the stability performance and prediction performance of ELM. Hybrid–ELM–PLSR can be divided into three stages. At first, filter is used to rearrange the input variables through the correlations with desired variables. Then wrapper is used to select the optimal variable subset through evaluating the prediction performance of different subsets. Finally, ELM–PLSR is used to build the prediction model. The simulation experiment results based on San Francisco river runoff dataset demonstrate that the proposed method is effective for multivariate chaotic time series. And the prediction accuracy and reliability are higher than other methods.

Multivariate chaotic time series prediction using multiple kernel extreme learning machine

Multivariate chaotic time series is widely present in nature, such as in economy, society, industry and other fields. Modeling and predicting multivariate time series will help human to better manage, control, and make decision. A prediction method based on multiple kernel extreme learning machine is proposed in this paper to model the complex dynamics of multivariate chaotic time series. First, the multivariate chaotic time series is reconstructed in phase space, transforming the temporal correlation into spatial correlation. Then, a prediction model-multiple kernel extreme learning machine, which combines the multiple kernel learning and extreme learning machine with kernels, is proposed to approximate the nonlinear function of the input - output data in phase space. The proposed multiple kernel extreme learning machine could effectively combine the simple training of extreme learning machine with kernels and the data fusion capabilities of multiple kernel learning. Simulation results based on Lorenz chaotic time series prediction and San Francisco monthly runoff prediction demonstrate that, compared with other state-of-art chaotic time series prediction methods, the proposed multiple kernel extreme learning machine could get a better prediction accuracy.

Multivariate chaotic time series prediction based on extreme learning machine

For multivariate chaotic time series prediction problem, a prediction based on input variable selection and extreme learning machine is proposed in this paper. The multivariate chaotic time series is reconstructed in phase space, and a mutual information based method is used to select the input variables, which have high statistics information with the output variables. The extreme learning machine is conducted to model the multivariate chaotic time series in the phase space by utilizing its approximation capability. In order to improve the prediction accuracy, a model selection algorithm is conducted for extreme learning machine to choose an expected minimum risk prediction model. Simulation results based on Lorenz, Rssler multivariate chaotic time series and Rssler hyperchaotic time series show the effectiveness of the proposed method.