Forecasting of noisy chaotic systems with deep neural networks

Abstract

Recurrent neural networks have recently proved the state-of-the-art approach in forecasting complex oscillatory time series on a multi-step horizon. Researchers in the field investigated different machine learning techniques and training approaches on dynamical systems with different degrees of complexity. Still, these analyses are usually limited to noise-free chaotic time series. This paper extends the analysis from a deterministic to a noisy environment, by considering both observation and structural noise. Observation noise is evaluated by adding different levels of artificially-generated random values on deterministic processes obtained from the simulation of four archetypal chaotic systems. A case of structural noise is implemented through a time-varying version of the logistic map, which exhibits a slow structural change of the system’s dynamic that makes the system non-stationary. Finally, a time series of ozone concentration in Northern Italy is considered to test the theoretical findings on a real-world case study in which both forms of noise play a significant role. Recurrent neural networks formed by LSTM cells are compared with two benchmark feed-forward architectures. LSTM trained without the standard teacher forcing approach, i.e., with training that replicates the setting used in inference mode, proved to have the best performance in compensating the stochasticity generated by the observation noise and reproducing the structural non-stationarity of the process.