Depth asynchronous time delay reservoir for nonlinear time series forecasting task

Abstract

The forecasting of nonlinear time series has a wide range of applications both in theoretical research and industrial production, as the fitting of nonlinear systems, the early warning of sunspot activity and the prediction of network traffic peaks in advance. These tasks require models with strong nonlinear mapping capabilities and sufficient short-term memory capacity. Single-node time delay Reservoir (TDR) have been frequently applied on nonlinear time series forecasting tasks. To address the lack of memory capacity of traditional TDR and its extended version (Delay Decoupled Reservoir (DDR), Deep Time-Delay Reservoir (DTDR)), we propose a Depth Asynchronous Time-Delay Reservoir (DATDR) model. Firstly, the model retains the deep network structure of the DTDR to ensure the dynamic properties of the model, and the history states of each layer are from the previous layers instead of the current layer. Secondly, the same signal input method as the DDR is used, i.e., each layer is fed with the original signal, and this data input method ensures that the feature of signal does not decay gradually during the transmission between layers. Finally, a time delay operator is inserted into two adjacent layers, and the memory capacity of the model could be controlled effectively by adjusting the delay time of the time delay operator to accommodate some time series prediction tasks that require long memory capacity. In this paper, eight datasets from three different types are used to validate the proposed model, including NARMA10-30, Mackey-Glass Chaos time series, He ́non map series, Sunspot Sequence, and two real network traffic datasets. Experiments show that proposed model gets significant improvement in memory capacity compared with TDR, DDR, and DTDR, and performs better for some nonlinear time series datasets especially some long-term memory datasets.