Chaotic time series prediction with deep belief networks: an empirical evaluation

Abstract

Chaotic time series are widespread in several real world areas such as finance, environment, meteorology, traffic flow, weather. A chaotic time series is considered as generated from the deterministic dynamics of a nonlinear system. The chaotic system is sensitive to initial conditions; points that are arbitrarily close initially become exponentially further apart with progressing time. Therefore, it is challenging to make accurate prediction in chaotic time series. The prediction using conventional statistical techniques, k-nearest-nearest neighbors algorithm, Multi-Layer-Perceptron (MPL) neural networks, Recurrent Neural Networks, Radial-Basis-Function (RBF) Networks and Support Vector Machines, do not give reliable prediction results for chaotic time series. In this paper, we investigate the use of a deep learning method, Deep Belief Network (DBN), combined with chaos theory to forecast chaotic time series. DBN should be used to forecast chaotic time series. First, the chaotic time series are analyzed by calculating the largest Lyapunov exponent, reconstructing the time series by phase-space reconstruction and determining the best embedding dimension and the best delay time. When the forecasting model is constructed, the deep belief network is used to feature learning and the neural network is used for prediction. We also compare the DBN –based method to RBF network-based method, which is the state-of-the-art method for forecasting chaotic time series. The predictive performance of the two models is examined using mean absolute error (MAE), mean squared error (MSE) and mean absolute percentage error (MAPE). Experimental results on several synthetic and real world chaotic datasets revealed that the DBN model is applicable to the prediction of chaotic time series since it achieves better performance than RBF network.