Chaotic Time Series Prediction Research Articles

Prediction of Sea Clutter Based on Recurrent Neural Network

As radar sea clutter has chaotic characteristics, a prediction model can be established to predict sea clutter according to short-term predictability of chaotic time series. Therefore, this paper proposes a sea clutter amplitude prediction model based on LSTM, Stacked LSTM and Nested LSTM. This paper first obtains the best state space for sea clutter prediction according to the phase space reconstruction theory and establishes the prediction equation, then uses three networks to predict the sea clutter amplitude, and finally compares the effects of the three neural networks and analyzes the factors that affect the prediction precision. The experimental results on the IPIX radar data and P-band radar data show that, the model used in this paper has a smaller prediction error than RBF, and the prediction performance is better. The model can achieve high-precision and high-efficiency prediction of sea clutter and has verified the effectiveness of the method.

Study on fault identification of mechanical dynamic nonlinear transmission system

Abstract To solve the problems of large mechanical powertrain such as complex structure, serious accident, strong nonlinear characteristics of running state, bad operating environment, non-Gaussian noise, and various uncertain factors, it is difficult to make an accurate fault diagnosis. This paper proposes a method for dealing with nonlinear characteristics using nuclear waves, as well as a system, deeply conducted nuclear base fault feature extraction, classification, and decision making, such as nuclear base state trend prediction technology research, focusing on exploring and improving the accuracy of fault diagnosis under nonlinear conditions, technical method, and way to state prediction accuracy. It offers effective technical assistance for the advancement and use of mechanical power train monitoring and diagnosis technology. A fault detection method based on kernel method is proposed. Based on the characteristics of this method in dealing with nonlinear problems, the research on kernel feature extraction, kernel fault classification and decision making, and kernel state trend prediction are carried out systematically. The experimental results show that the simulation analysis of typical chaotic time series prediction and the application of the operation state prediction of a certain ship main steam turbine unit have achieved good results, among which the average relative error of the single-step prediction of the unit state is 1.7881%, and the average relative error of the 30-step prediction is 3.3983%. Proved that the nuclear methods systematically applied to mechanical power transmission system fault diagnosis and state prediction, effectively enhancing some traditional methods and techniques dealing with nonlinear feature extraction, the nonlinear prediction capability for fault identification, and nonlinear state, to deal with nonlinear fault diagnosis problems of engineering practice, a large number of explored effective solution.

Pipeline FPGA-Based Implementations of ANNs for the Prediction of up to 600-Steps-Ahead of Chaotic Time Series

Chaotic time series prediction can be performed by applying different architectures of artificial neural networks (ANNs) that can be implemented on field-programmable gate arrays (FPGAs). However, the main challenges are the reduction of hardware resources to develop faster ANNs and the prediction capabilities for large horizons. In this manner, the contribution is devoted to introduce pipeline architectures in which some registers are placed between combinational blocks to divide the logic into shorter stages that can run with a faster clock. The cases of study are the multilayer perceptron (MLP), nonlinear autoregressive with exogenous input (NARX), and echo state network (ESN). In addition, another contribution is devoted to introduce the application of the decimation technique to extend the prediction horizon of the ANNs from 12 to 600-steps-ahead. The prediction capabilities of the MLP, NARX and ESN are compared by using eight chaotic time series with different maximum Lyapunov exponents. The pipeline FPGA-based implementations show that the ESN with a reservoir of at least 30 neurons guarantees a large prediction horizon of 600-steps-ahead. Another important advantage of the ESN is that its FPGA-based implementation can be performed by reusing one neuron, thus requiring the lowest quantity of hardware resources.

Prediction of chaotic time series using hybrid neural network and attention mechanism

Chaotic time series forecasting has been widely used in various domains, and the accurate predicting of the chaotic time series plays a critical role in many public events. Recently, various deep learning algorithms have been used to forecast chaotic time series and achieved good prediction performance. In order to improve the prediction accuracy of chaotic time series, a prediction model (Att-CNN-LSTM) is proposed based on hybrid neural network and attention mechanism. In this paper, the convolutional neural network (CNN) and long short-term memory (LSTM) are used to form a hybrid neural network. In addition, a attention model with <i>softmax</i> activation function is designed to extract the key features. Firstly, phase space reconstruction and data normalization are performed on a chaotic time series, then convolutional neural network (CNN) is used to extract the spatial features of the reconstructed phase space, then the features extracted by CNN are combined with the original chaotic time series, and in the long short-term memory network (LSTM) the combined vector is used to extract the temporal features. And then attention mechanism captures the key spatial-temporal features of chaotic time series. Finally, the prediction results are computed by using spatial-temporal features. To verify the prediction performance of the proposed hybrid model, it is used to predict the Logistic, Lorenz and sunspot chaotic time series. Four kinds of error criteria and model running times are used to evaluate the performance of predictive model. The proposed model is compared with hybrid CNN-LSTM model, the single CNN and LSTM network model and least squares support vector machine(LSSVM), and the experimental results show that the proposed hybrid model has a higher prediction accuracy.

Chaotic time series prediction with deep belief networks: an empirical evaluation

Chaotic time series are widespread in several real world areas such as finance, environment, meteorology, traffic flow, weather. A chaotic time series is considered as generated from the deterministic dynamics of a nonlinear system. The chaotic system is sensitive to initial conditions; points that are arbitrarily close initially become exponentially further apart with progressing time. Therefore, it is challenging to make accurate prediction in chaotic time series. The prediction using conventional statistical techniques, k-nearest-nearest neighbors algorithm, Multi-Layer-Perceptron (MPL) neural networks, Recurrent Neural Networks, Radial-Basis-Function (RBF) Networks and Support Vector Machines, do not give reliable prediction results for chaotic time series. In this paper, we investigate the use of a deep learning method, Deep Belief Network (DBN), combined with chaos theory to forecast chaotic time series. DBN should be used to forecast chaotic time series. First, the chaotic time series are analyzed by calculating the largest Lyapunov exponent, reconstructing the time series by phase-space reconstruction and determining the best embedding dimension and the best delay time. When the forecasting model is constructed, the deep belief network is used to feature learning and the neural network is used for prediction. We also compare the DBN –based method to RBF network-based method, which is the state-of-the-art method for forecasting chaotic time series. The predictive performance of the two models is examined using mean absolute error (MAE), mean squared error (MSE) and mean absolute percentage error (MAPE). Experimental results on several synthetic and real world chaotic datasets revealed that the DBN model is applicable to the prediction of chaotic time series since it achieves better performance than RBF network.

A novel pipelined neural FIR architecture for nonlinear adaptive filter

This paper presents a novel adaptive pipelined neural finite impulse response (PNFIR) filter for nonlinear signal processing. Unlike traditional pipelined recurrent neural network (PRNN), each module of the PNFIR filter is a simple architecture that includes a standard FIR filter followed by a nonlinear activation function. The complete design of proposed filter includes two subsections: The nonlinear part consists of neural FIR (NFIR) modules which is interconnected in a chained form and simultaneously executed in a parallel fashion; the linear subsection is a tapped-delay-line (TDL) linear combiner. Based on convex combination architecture, the adaptive algorithm derived from the gradient descent approach is utilized to update weights of the nonlinear and linear parts. Moreover, the analysis of stability conditions and computational complexity is also presented. Numerous simulation experimental results on nonlinear dynamic systems identification, speech signal and chaotic time series prediction show that the proposed PNFIR filter has simpler architecture, faster convergence rate, and lower computation complexity than the PRNN and joint process filter using pipelined feedforward second-order Volterra architecture (JPPSOV).

Recursive Minimum Kernel Mixture Mean p-Power Error Algorithm Based on the Nyström Method

In this brief, a novel kernel mixture mean $p$ -power error (KMP) criterion is proposed by combining the mixture of two Gaussian functions into the kernel function of kernel mean $p$ -power error. The mixture correntropy (MC) measure can be viewed as a special case of the KMP with $p = 2$ . And the KMP with an appropriate $p$ can provide better accuracy than MC for robust learning. Some properties of KMP are presented for discussion. The Nystrom method is an efficient method for curbing the growth of network size of kernel adaptive filters (KAFs), and the recursive update form can improve the tracking ability of KAFs. To this end, we apply the Nystrom method and recursive update form to the KMP criterion, generating a novel recursive minimum kernel mixture mean $p$ -power error algorithm based on the Nystrom method (NysRMKMP). Monte Carlo simulations on chaotic time-series prediction illustrate the desirable accuracy and robustness of NysRMKMP.

Learning Both Dynamic-Shared and Dynamic-Specific Patterns for Chaotic Time-Series Prediction.

In the real world, multivariate time series from the dynamical system are correlated with deterministic relationships. Analyzing them dividedly instead of utilizing the shared-pattern of the dynamical system is time consuming and cumbersome. Multitask learning (MTL) is an effective inductive bias method to utilize latent shared features and discover the structural relationships from related tasks. Base on this concept, we propose a novel MTL model for multivariate chaotic time-series prediction, which could learn both dynamic-shared and dynamic-specific patterns. We implement the dynamic analysis of multiple time series through a special network structure design. The model could disentangle the complex relationships among multivariate chaotic time series and derive the common evolutionary trend of the multivariate chaotic dynamical system by inductive bias. We also develop an efficient Crank-Nicolson-like curvilinear update algorithm based on the alternating direction method of multipliers (ADMM) for the nonconvex nonsmooth Stiefel optimization problem. Simulation results and analysis demonstrate the effectiveness on dynamic-shared pattern discovery and prediction performance.

A hybrid prediction method based on empirical mode decomposition and multiple model fusion for chaotic time series

Chaotic time series exist in nature, such as in the field of meteorology or physics, with unpredictable features caused by their inherent high complexity and nonstationary motion. To improve the prediction effect of chaotic time series, a hybrid prediction method on the basis of the empirical mode decomposition (EMD) and neural networks (NN) is proposed. First, the original chaotic time series is decomposed into several intrinsic mode functions (IMFs) and one residual by EMD, whose components are divided into high, medium and low components by using the runs test. The high IMFs change dramatically, and the correlation among data is not high; therefore, we use the fuzzy first-order transition rules trained neural network (NN-FFOTR) model to predict the high frequency components. Autoregressive integrated moving average (ARIMA) models are applied according to the random characteristic exhibited by the medium frequency IMFs and residual. Finally, the prediction results of the two models are adaptively superimposed together to obtain the final prediction results. The proposed hybrid method is named EMD-ARIMA-NN-FFOTR for simplicity. The simulation results are verified through two typical chaotic time series including the Lorenz-63 system and Mackey-Glass time series. For the x-axis of the Lorenz-63 system, the experimental results indicates that the symmetric mean absolute percentage error (SMAPE) of the ARIMA, NN-FFOTR, ARIMA-NN-FFOTR and EMD-ARIMA-NN-FFOTR are 1.0657, 1.5281, 1.0657 and 0.0041;the root mean square error (RMSE) are 2.2591,0.6830,0.4231 and 0.0012; the mean absolute error (MAE) are 50.5149,15.2130,9.4610 and 0.0019; the sum of error squares (SSE) are 101.4,96.32,65.8300 and 21.6351; and the Theil inequality coefficient are 0.0674, 0.0741, 0.0213 and 0.0162, the proposed model's TIC are much lower than those of the comparison models. The values of the R square of the developed prediction approach utilizing ARIMA, NN-FFPTR, ARIMA-NN-FFOTR and EMD-ARIMA-NN-FFOTR method are 0.9674,0.9741, 0.9799 and 0.9963, and the index of agreement are 0.8992,0.9921,0.9845 and 0.9991, the value of the R-square and the index of agreement of the proposed model's are much higher than those of the comparison models. The Pearson's test results show that the association strength between the actual value and the predicted values of the proposed model is stronger. These results show the following: (a) compared with other related, recent studies, the prediction accuracy of the hybrid system EMD-ARIMA-NN-FFOTR proposed in this article is higher; (b) the proposed hybrid system attains superior performance compared with single models; and (c) the proposed hybrid system balances the forecast accuracy and convergence speed simultaneously during prediction. Therefore, it is feasible to apply the hybrid model to the prediction of chaotic time series.

Projected Kernel Least Mean $p$ -Power Algorithm: Convergence Analyses and Modifications

Sparsified kernel adaptive filters (SKAFs) is an attractive filtering solution with low memory and computational complexity. Most of existing SKAFs are based on the mean square error (MSE) criterion under Gaussian noise assumption for its simplicity and convenience. When the assumption deviates largely from the underlying truth, the performance of these methods could degrade significantly. In this paper, we propose a novel SKAF, named as projected kernel least mean $p$ -power algorithm (PKLMP), based on the mean $p$ -power error (MPE) criterion and vector projection (VP) method. We provide convergence analyses in terms of the stead-state MSE, based on a Taylor expansion method, and derive the lower and upper bounds for the steady-state excess MSE. We also conduct mean convergence analysis for PKLMP, and derive convergence conditions. To exploit the information in the desired outputs, we further derive a modified PKLMP by smoothing the desired signal. Finally, a simple and effective online variable kernel centers strategy is proposed to improve the filtering performance of the proposed KAFs. Simulation results under a static function estimation, a chaotic time-series prediction, and two real-world time-series predictions are conducted and validate the effectiveness of the proposed PKLMP algorithms.

Using multidimensional speckle dynamics for high-speed, large-scale, parallel photonic computing.

The recent rapid increase in demand for data processing has resulted in the need for novel machine learning concepts and hardware. Physical reservoir computing and an extreme learning machine are novel computing paradigms based on physical systems themselves, where the high dimensionality and nonlinearity play a crucial role in the information processing. Herein, we propose the use of multidimensional speckle dynamics in multimode fibers for information processing, where input information is mapped into the space, frequency, and time domains by an optical phase modulation technique. The speckle-based mapping of the input information is high-dimensional and nonlinear and can be realized at the speed of light; thus, nonlinear time-dependent information processing can successfully be achieved at fast rates when applying a reservoir-computing-like-approach. As a proof-of-concept, we experimentally demonstrate chaotic time-series prediction at input rates of 12.5 Gigasamples per second. Moreover, we show that owing to the passivity of multimode fibers, multiple tasks can be simultaneously processed within a single system, i.e., multitasking. These results offer a novel approach toward realizing parallel, high-speed, and large-scale photonic computing.

Enhanced Prediction Performance of a Neuromorphic Reservoir Computing System Using a Semiconductor Nanolaser With Double Phase Conjugate Feedbacks

A neuromorphic reservoir computing (RC) system using a semiconductor nanolaser (SNL) with double phase conjugate feedbacks (PCF) is proposed for the first time and demonstrated numerically. The prediction performance of such RC system is investigated via Santa Fe chaotic time series prediction task. The Purcell cavity-enhanced spontaneous emission factor F and the spontaneous emission coupling factor β are included in the rate equations, and the influences of F and β on the prediction performance of such RC system are analyzed extensively. For the purpose of comparison, the prediction performance of SNL-based RC system with single PCF is also considered. The simulation results indicate that, compared with the SNL-based RC system with single PCF, enhanced prediction performance can be obtained for the SNL-based RC system with double PCF. Moreover, the influences of bias current, the modulation depth of input signal, feedback strength, as well as feedback delay, are also taken into account. The proposed SNL-based RC system subject to double PCF in this paper has the potential to develop the RC-based neuromorphic photonic integrated circuit.

Patch Learning

There have been different strategies to improve the performance of a machine learning model, e.g., increasing the depth, width, and/or nonlinearity of the model, and using ensemble learning to aggregate multiple base/weak learners in parallel or in series. This article proposes a novel strategy called patch learning (PL) for this problem. It consists of three steps: first, train an initial global model using all training data; second, identify from the initial global model the patches that contribute the most to the learning error, and train a (local) patch model for each such patch; and, third, update the global model using training data that do not fall into any patch. To use a PL model, we first determine if the input falls into any patch. If yes, then the corresponding patch model is used to compute the output. Otherwise, the global model is used. We explain in detail how PL can be implemented using fuzzy systems. Five regression problems on one-dimensional (1-D)/2-D/3-D curve fitting, nonlinear system identification, and chaotic time-series prediction, verified its effectiveness. To our knowledge, the PL idea has not appeared in the literature before, and it opens up a promising new line of research in machine learning.

Chaotic Time Series Prediction Using LSTM with CEEMDAN

Chaotic systems are complex dynamical systems that play a very important role in the study of the atmosphere, aerospace engineering, finance, etc. To improve the accuracy of chaotic time series prediction, this study proposes a hybrid model CEEMDAN-LSTM which combines Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) and long short-term memory (LSTM). In the model, the original time series is decomposed into several intrinsic mode functions (IMFs) and a residual component. To reduce the difficulty of predicting chaotic time series and provide a high level of predictive accuracy, the LSTM prediction model is built for all each characteristic series from CEEMDAN deposition. Finally, the final prediction results are obtained by combining all the prediction sequences. To test the effectiveness of this model we proposed, we examined the CEEMDAN-LSTM model using the Lorenz-63 system. Further compared to Autoregressive Integrated Moving Average (ARIMA ), Support Vector Regression (SVR), multilayer perceptron (MLP), and the single LSTM model, the results of the experiment show that the proposed model performs better in the prediction of chaotic time series. Besides, the hybrid model proposed in this paper has better results than the LSTM model alone. Therefore, hybrid models based on deep learning methods and signal decomposition methods have great potential in the field of chaotic time series prediction.

Chaotic time series prediction using phase space reconstruction based conceptor network.

The Conceptor network is a new framework of reservoir computing (RC), in addition to the features of easy training, global convergence, it can online learn new classes of input patterns without complete re-learning from all the training data. The conventional connection topology and weights of the hidden layer (reservoir) of RC are initialized randomly, and are fixed to be no longer fine-tuned after initialization. However, it has been demonstrated that the reservoir connection of RC plays an important role in the computational performance of RC. Therefore, in this paper, we optimize the Conceptor's reservoir connection and propose a phase space reconstruction (PSR) -based reservoir generation method. We tested the generation method on time series prediction task, and the experiment results showed that the proposed PSR-based method can improve the prediction accuracy of Conceptor networks. Further, we compared the PSR-based Conceptor with two Conceptor networks of other typical reservoir topologies (random connected, cortex-like connected), and found that all of their prediction accuracy showed a nonlinear decline trend with increasing storage load, but in comparison, our proposed PSR-based method has the best accuracy under different storage loads.

An asynchronously deep reservoir computing for predicting chaotic time series

Chaotic time series prediction is a research topic in both theoretical and real-life area. Its aim is to predict the future of the time series based on past observations. Reservoir computing (RC) is a promising tool widely used in time series prediction. Short-term memory (STM) is very important to model time-dependent time series by the RC approach. However, traditional RC hardly achieves sufficient STM capacity required by a complicated time series prediction task. For this reason, this paper proposes an asynchronous deep RC (ADRC), which is composed of a number of sub-reservoirs that are connected one by one in sequence. Moreover, delayed modules are inserted between every two adjacent sub-reservoirs. The sub-reservoirs in the proposed ADRC preserve the input characteristics by a relay mode and deal with them asynchronously. This makes the reservoir achieve large STM capacity and rich dynamics. The experimental results demonstrate that the proposed ADRC is prominent in modeling chaotic time series signals with high performance.

Design of a self-organizing reciprocal modular neural network for nonlinear system modeling

Aiming to improve the model’s generalization performance for nonlinear system modeling, a self-organizing reciprocal modular neural network (SORMNN) is proposed in the present study, which imitates the modular structure with inter-module connections observed in human brains. The inter-module connections in SORMNN are built by inputting the output of each subnetwork to other subnetworks. All subnetworks work in parallel to process the allocated features, and the structure of each subnetwork is designed to be self-organized by using a growing and pruning algorithm based on the contribution of hidden neurons. An improved Levenberg-Marquardt (LM) algorithm using a sliding window is conducted to update the parameters of SORMNN, which makes SORMNN available for solving online problems. To validate the effectiveness of the proposed model, SORMNN is tested on chaotic benchmark time series prediction, four UCI benchmark problems and a practical problem for biochemical oxygen demand prediction in wastewater treatment process. Experimental results demonstrate that SORMNN exhibits both a higher training accuracy and a better generalization ability for nonlinear system modeling than other modular neural networks, and the inter-module connections have a positive effect on the superior performance of the proposed model and can make the network structure compact.

Evolutionary optimization of artificial neural network using an interactive phase-based optimization algorithm for chaotic time series prediction

The prediction of chaotic time series is an important issue in nonlinear information procession. Due to the multi-modal, high-dimensional and non-differentiable or discontinuous characteristics of chaotic systems, global optimization techniques are required to avoid from falling into local optima for the prediction of chaotic time series. Phase-based optimization is recently proposed as a global search algorithm inspired by natural phenomena. In this paper, an improved phase-based optimization algorithm integrating stochastic interaction strategy and global optimal interaction strategy, termed interactive phase-based optimization (IPBO), is proposed to train feed-forward neural networks (FNNs) for chaotic time series prediction. The combination of stochastic interaction strategy and global optimal interaction strategy can balance the capability of exploration and exploitation in the global optimization process. To demonstrate the searching capability, sixteen widely used benchmark functions are firstly used to investigate its optimization performance. Then, the prediction effectiveness of FNNs trained by IPBO has been illustrated using classical chaotic time series of Lorenz, Box–Jenkins and Mackey–Glass. The training and testing performances of IPBO and other state-of-the-art optimization algorithms have been compared for predicting these time series. Conducted numerical experiments indicate that IPBO is not only competitive in functions optimization and has also a better learning ability in training FNNs among other state-of-the-art optimization algorithms.

Underwater Acoustic Signal Prediction Based on MVMD and Optimized Kernel Extreme Learning Machine

Aiming at the chaotic characteristics of underwater acoustic signal, a prediction model of grey wolf-optimized kernel extreme learning machine (OKELM) based on MVMD is proposed in this paper for short-term prediction of underwater acoustic signals. To solve the problem of K value selection in variational mode decomposition, a new K value selection method MVMD is proposed from the perspective of mutual information, which avoids the blindness of variational mode decomposition (VMD) in the preset modal number. Based on the prediction model of kernel extreme learning machine (KELM), this paper uses grey wolf optimization (GWO) algorithm to optimize and select its regularization parameters and kernel parameters and proposes an optimized kernel extreme learning machine OKELM. To further improve the prediction performance of the model, combined with MVMD, an underwater acoustic signal prediction model based on MVMD-OKELM is established. MVMD-OKELM prediction model is applied to Mackey–Glass chaotic time series prediction and underwater acoustic signal prediction and is compared with ARIMA, EMD-OKELM, and other prediction models. The experimental results show that the proposed MVMD-OKELM prediction model has a higher prediction accuracy and can be effectively applied to the prediction of underwater acoustic signal series.

High-Speed Neuromorphic Reservoir Computing Based on a Semiconductor Nanolaser With Optical Feedback Under Electrical Modulation

A high-speed neuromorphic reservoir computing system based on a semiconductor nanolaser with optical feedback (SNL-based RC) under electrical modulation is proposed for the first time and demonstrated numerically. A Santa-Fe chaotic time series prediction task is employed to quantify the prediction performance of the SNL-based RC system. The effects of the Purcell cavity-enhanced spontaneous emission factor F and the spontaneous emission coupling factor β on the proposed RC system are analyzed extensively. It is found that, in general, increased F and β extend the range of good prediction performance of the SNL-based RC system. Moreover, the influences of bias current and feedback phase are also considered. Due to the ultra-short photon lifetime in SNL, the information processing rate of the SNL-based RC system reaches 10Gpbs. The proposed high-speed SNL-based RC system in this paper provides theoretical guidelines for the design of RC-based integrated neuromorphic photonic systems.