Harnessing deep learning for chaotic time series forecasting: a performance comparison of different methods and models

In order to study forecasting of chaotic time series, artificial chaotic time series that are derived from difference transformations on time series generated by deterministic maps such as the tent map and the logistic map are theoretically analyzed. In order to predict future values of these model time series, we need to know past data in addition to the current data. Here, model time series in which the amount of data that is necessary for prediction is definitely determined and a model time series in which, depending on past data, the amount would be indefinitely large are proposed. Both of these models should be useful models of chaotic time series when we test the effectiveness of various forecasting methods that have been and will be proposed.

An algorithm is proposed that automatically generates the fuzzy rules from time series data and can subsequently be used for forecasting of the same time series. The effectiveness of the algorithm, measured by the performance indices such as the sum squared error (SSE), root mean squared error (RMSE/MSE) and the mean absolute error (MAE), is demonstrated on forecasting of chaotic time series, as well as on forecasting of homogeneous non-stationary time series with and without seasonality and trend components.

Accurate forecasting of chaotic and nonlinear time series has been a key area of research in last two decades. They find extensive applications in stock market prediction, forecasting weather conditions, determining the inferences of chemical reactions and many more. The manuscript deals with development of a new hybrid model based on Wavelet Neural Network (WNN) trained by Cat Swarm Optimization (CSO). The performance of the proposed model is accessed on benchmark time series like ‘Mackey-Glass’ and ‘Box Jenkins’. Comparison with WNN-PSO, Chebyshev FLANN and MLP-BP models reveal the superior performance of the proposed model in terms of response matching, Minimum MSE and lower SSE values achieved. Therefore the WNN-CSO model is a preferred candidate for accurate prediction of Chaotic and Nonlinear time series.

In recent years, quantum inspired neural networks have been applied to various practical problems since their proposal. Here we investigate whether our qubit neural network(QNN) leads to an advantage over the conventional (real-valued) neural network(NN) in the forecasting of chaotic time series. QNN is constructed from a set of qubit neuron, of which internal state is a coherent superposition of qubit states. In this paper, we evaluate the performance of QNN through a prediction of well-known Lorentz attractor, which produces chaotic time series by three dynamical systems. The experimental results show that QNN can forecast time series more precisely, compared with the conventional NN. In addition, we found that QNN outperforms the conventional NN by reconstructing the trajectories of Lorentz attractor.

Modern learning systems are often faced with the need to account for non-linear relationships between the parameters of the learning process, individual characteristics of students and final results. However, the use of traditional statistical methods (e.g., linear regression) is limited by their ability to deal with uncertainty and complex data. An additional difficulty is the lack of systematized approaches to selecting the optimal neural network architecture, which makes it difficult to implement them in educational practice. The aim of the study is comparison of neural network models and traditional machine learning methods (multinomial logistic regression, random forests) for predicting learning outcomes, and determination of the optimal parameters of neural network architecture using the Python programming language. The paper presents the performance results of neural networks, multinomial regression and ensemble methods (random forest, XGBoost) on educational process data. The influence of the number of neurons in the hidden layer on the accuracy of the model has been analyzed. The universality of neural network models as approximators for predicting learning outcomes taking into account individual characteristics of students has been confirmed. The results of the study can be used to develop adaptive educational platforms that provide personalized selection of assignments and dynamic adjustment of curricula, to create the training data analysis systems integrating neural network and fuzzy models for working with uncertain pedagogical parameters, to optimize the knowledge assessment processes through the choice of optimal neural network architecture taking into account the available resources.

An intelligent approach is proposed for processing of time series based on a neuro-fuzzy network and an adaptive genetic algorithm (AGA). A chaotic time series data is used for network training because the trained network should be applied for forecasting of chaotic time series. A simple technique is used to measure the convergence speed of the GA, which in turn determines the probability values of genetic operators in each generation. Using the adaptive versions of probability values of genetic operators the modified GA version has improved its convergence towards the desired fitness function. As the accuracy measure of the forecast the performance indices such as sum square error (SSE), mean square error (MSE), and mean absolute error (MAE) are used. It was shown that the proposed intelligent approach is an excellent tool for forecasting the chaotic time series.

There has been active investigation into deep learning approaches for time series analysis, including foundation models. However, most studies do not address significant scientific applications. This paper aims to identify key features in time series by examining complex hydrology data. Our work advances computer science by emphasizing critical application features and contributes to hydrology and other scientific fields by identifying modeling approaches that effectively capture these features. Scientific time series data are inherently complex, involving observations from multiple locations, each with various time-dependent data streams and exogenous factors that may be static or time-varying and either application-dependent or purely mathematical. This research analyzes hydrology time series from the CAMELS and Caravan global datasets, which encompass rainfall and runoff data across catchments, featuring up to six observed streams and 209 static parameters across approximately 8000 locations. Our investigation assesses the impact of exogenous data through eight different model configurations for key hydrology tasks. Results demonstrate that integrating exogenous information enhances data representation, reducing root mean squared error by up to 40% in the largest dataset. Additionally, we present a detailed performance comparison of over 20 state-of-the-art pattern and foundation models. The analysis is fully open-source, facilitated by Jupyter Notebook on Google Colab for LSTM-based modeling, data preprocessing, and model comparisons. Preliminary findings using alternative deep learning architectures reveal that models incorporating comprehensive observed and exogenous data outperform more limited approaches, including foundation models. Notably, natural annual periodic exogenous time series contribute the most significant improvements, though static and other periodic factors are also valuable. This research serves as both an educational tool and benchmark resource.

BayesValidRox is an open-source python package that provides methods for surrogate modeling, Bayesian inference and model comparison. Release 2.0.0 improves the modularity and uniformity of the package and introduces template classes for surrogate models and methods for generating posterior samples.(2025-02-05)

The accuracy of a model to forecast a time series diminishes as the prediction horizon increases, in particular when the prediction is carried out recursively. Such decay is faster when the model is built using data generated by highly dynamic or chaotic systems. This paper presents a topology and training scheme for a novel artificial neural network, named "Hybrid-connected Complex Neural Network" (HCNN), which is able to capture the dynamics embedded in chaotic time series and to predict long horizons of such series. HCNN is composed of small recurrent neural networks, inserted in a structure made of feed-forward and recurrent connections and trained in several stages using the algorithm back-propagation through time (BPTT). In experiments using a Mackey-Glass time series and an electrocardiogram (ECG) as training signals, HCNN was able to output stable chaotic signals, oscillating for periods as long as four times the size of the training signals. The largest local Lyapunov Exponent (LE) of predicted signals was positive (an evidence of chaos), and similar to the LE calculated over the training signals. The magnitudes of peaks in the ECG signal were not accurately predicted, but the predicted signal was similar to the ECG in the rest of its structure.

Back-propagation neural network (BPN) is sufficiently suitable for forecasting of Chaotic data time series by two approaches i.e., parametric forecasting and time-series forecasting. However, conformation of its optimum architecture for a specific case is pre-requisite. Total rainfall-runoff (R–R) from Basantpur station over Mahanadi river basin was in under study. Initially modelling of R–R by parametric forecast approach was done by optimum architecture of BPN. It is found that BPN performs excellent for the months of July, August, and October. For the September performance was drastically unfortunate. Modelling was denied by hypothesis \(MAD\,\left( {\% \,of\,LPA} \right)\,<\,{\raise0.7ex\hbox{$1$} \!\mathord{\left/ {\vphantom {1 {2 }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${2 }$}} SD (\% \,of\,LPA)\). By 20,000,000 epochs of training \(MAD\,\left( {\% \,of\,LPA} \right)\) were 23.40 and \(SD\,\left( {\% \,of\,LPA} \right)\) were 33.84. Therefore second approach of modelling i.e., time-series forecasting was applied. In which, ‘n’ years monthly past recorded R–R over the station is used to forecast of \((n + 1)\)th year monthly R–R over the station. This was significantly found appropriate and better evaluated over parametric approach. The model was highly acceptable under the hypothesis wherein \(MAD\,\left( {\% \,of\,LPA} \right)\) was 2.4516164772862348E-6 and \(SD\,\left( {\% \,of\,LPA} \right)\) was 33.45 and corelation coefficient (CC) between actual and predicted R–R is obtained 1.0 in training and 0.82 in testing independently. In this paper detail design of BPN in time-series forecasting, optimization of its architecture, training with 35 years data sets, testing with 7 years data sets, its evaluation over BPN in parametric forecast is discussed.

For time series forecasting, obtaining models is based on the use of past observations from the same sequence. In those cases, when the model is learning from data, there is not an extra information that discuss about the quantity of noise inside the data available. In practice, it is necessary to deal with finite noisy datasets, which lead to uncertainty about the propriety of the model. For this problem, the employment of the Bayesian inference tools are preferable. A modified algorithm used for training a long-short term memory recurrent neural network for time series forecasting is presented. This approach was chosen to improve the forecasting of the original series, employing an implementation based on the minimization of the associated Kullback-Leibler Information Criterion. For comparison, a nonlinear autoregressive model implemented with a feedforward neural network was also presented. A simulation study was conducted to evaluate and illustrate results, comparing this approach with Bayesian neural-networks-based algorithms for artificial chaotic time-series and showing an improvement in terms of forecasting errors.

Chaotic time series forecasting such as maximum wind speed rates is of great importance in the fields of meteorology and renewable energy to reduce and control the harmful negative effects. The problem of wind speed is that it is affected by several interrelated factors such as temperature and atmospheric pressure, which are characterized by non-linearity through the influence of time series on differences that may be a cause of the emergence of uncertainty problems, which makes it difficult to model using traditional univariate time series methods. Echo State Network (ESN) is a neural network specialized in time series forecasting after addressing the problem of synchronization with the time variable as a recurrent network to address time-dependent effects and accurate prediction of time series in addition to its ability to model nonlinearly. This study presents the use of the Autoregressive (AR) model and then its hybridization with the deep echo state network, which is called the AR-ESN hybrid method by using the optimal structure of the AR model to determine the optimal inputs to the ESN network as the main contributions to solving the prediction problems for real data forecasts. The use of ESN as a proposed forecasting method is to improve the forecasting efficiency to reduce the risks associated with extreme weather fluctuations compared with traditional forecasting results. The results indicate that the ESN model based on AR model can contribute to increasing the forecasting accuracy of maximum wind speed compared with traditional models by using mean absolute percentage error (MAPE) as one of the criteria the forecasting accuracy.

A neuro-fuzzy approach for forecasting of chaotic time series is proposed, based on neuro-implementation of a fuzzy logic system with the Gaussian membership functions. To construct the neuro-fuzzy system that will approximate and forecast the future values of a chaotic time series, the parameters of the membership functions, i.e. the mean (c) and the variance (/spl sigma/) of the selected Gaussian functions, as well as the center of fuzzy region (y/sup l/) are to be adjusted either by backpropagation or the Levenberg-Marquardt training algorithm. To examine the effectiveness of the forecasting method the performance function, like the sum squared errors, mean squared errors, and mean absolute errors, are evaluated. In this way it was shown that the proposed neuro-fuzzy approach is an excellent tool for chaotic time series prediction.

Chaotic time series forecasting has been widely used in various domains, and the accurate predicting of the chaotic time series plays a critical role in many public events. Recently, various deep learning algorithms have been used to forecast chaotic time series and achieved good prediction performance. In order to improve the prediction accuracy of chaotic time series, a prediction model (Att-CNN-LSTM) is proposed based on hybrid neural network and attention mechanism. In this paper, the convolutional neural network (CNN) and long short-term memory (LSTM) are used to form a hybrid neural network. In addition, a attention model with &lt;i&gt;softmax&lt;/i&gt; activation function is designed to extract the key features. Firstly, phase space reconstruction and data normalization are performed on a chaotic time series, then convolutional neural network (CNN) is used to extract the spatial features of the reconstructed phase space, then the features extracted by CNN are combined with the original chaotic time series, and in the long short-term memory network (LSTM) the combined vector is used to extract the temporal features. And then attention mechanism captures the key spatial-temporal features of chaotic time series. Finally, the prediction results are computed by using spatial-temporal features. To verify the prediction performance of the proposed hybrid model, it is used to predict the Logistic, Lorenz and sunspot chaotic time series. Four kinds of error criteria and model running times are used to evaluate the performance of predictive model. The proposed model is compared with hybrid CNN-LSTM model, the single CNN and LSTM network model and least squares support vector machine(LSSVM), and the experimental results show that the proposed hybrid model has a higher prediction accuracy.

Chaos is widespread in non-linear systems such as finance, energy, and weather. In the chaos system, a variable changing with time generates a chaotic time series, which contains a wealth of information about the non-linear system, and it is helpful for us to analyze and understand chaos systems. Traditional hybrid models for chaotic time series prediction based on neural networks have problems such as low prediction accuracy and difficulty in determining the network topologies. In recent years, the chaotic time series prediction has attached the attention of researchers in the area of deep learning. In this paper, we use a deep hybrid neural network (DHNN) based on convolutional neural network (CNN), gated recurrent unit (GRU) network, and attention mechanism to predict chaotic time series. Besides, we use the idea of neuroevolution to optimize the topologies of the DHNN. In the DHNN, we use CNN to capture spatial features from phase space reconstruction of chaotic time series. Then, we combine spatial features with the original chaotic time series. GRU extracts the spatio-temporal features from the combined sequence, and an attention mechanism with a non-linear activation function is designed to capture critical spatio-temporal features. Besides, we propose an improved differential evolution (IDE) algorithm to optimize the topologies of the DHNN, including the filter sizes of CNN and the number of hidden neurons of GRU. We develop the IDE with an adaptive mutation operator and dynamic chaos crossover operator, which can improve convergence speed and reduce optimization time. In this paper, we use the theoretical Lorenz datasets, monthly mean total sunspot datasets, and the actual coal-mine gas concentration datasets to verify the prediction accuracy of the proposed prediction model. Experimental results have shown that the proposed prediction model performs well in chaotic time series forecasting.