Abstract
Abstract. Neural networks are able to approximate chaotic dynamical systems when provided with training data that cover all relevant regions of the system's phase space. However, many practical applications diverge from this idealized scenario. Here, we investigate the ability of feed-forward neural networks to (1) learn the behavior of dynamical systems from incomplete training data and (2) learn the influence of an external forcing on the dynamics. Climate science is a real-world example where these questions may be relevant: it is concerned with a non-stationary chaotic system subject to external forcing and whose behavior is known only through comparatively short data series. Our analysis is performed on the Lorenz63 and Lorenz95 models. We show that for the Lorenz63 system, neural networks trained on data covering only part of the system's phase space struggle to make skillful short-term forecasts in the regions excluded from the training. Additionally, when making long series of consecutive forecasts, the networks struggle to reproduce trajectories exploring regions beyond those seen in the training data, except for cases where only small parts are left out during training. We find this is due to the neural network learning a localized mapping for each region of phase space in the training data rather than a global mapping. This manifests itself in that parts of the networks learn only particular parts of the phase space. In contrast, for the Lorenz95 system the networks succeed in generalizing to new parts of the phase space not seen in the training data. We also find that the networks are able to learn the influence of an external forcing, but only when given relatively large ranges of the forcing in the training. These results point to potential limitations of feed-forward neural networks in generalizing a system's behavior given limited initial information. Much attention must therefore be given to designing appropriate train-test splits for real-world applications.
Highlights
1.1 Neural networks for weather and climate applicationsNeural networks are a series of interconnected – potentially nonlinear – functions whose mutual relations are “learned” by the network by training on data
We focus on the widely used feed-forward neural networks and address two open questions related to their use for approximating the dynamics of chaotic systems
We explored how well feed-forward neural networks can (1) generalize the behavior of a chaotic dynamical system to its full phase space when trained only on part of said phase space and (2) learn the influence of a slow external forcing on a chaotic dynamical system
Summary
Neural networks are a series of interconnected – potentially nonlinear – functions whose mutual relations are “learned” by the network by training on data One of their many applications is forecasting the time evolution of dynamical systems. These (and other variants of the Lorenz system) are widely used as toy models for studying atmosphere-like systems, in the context of machine learning (e.g., Vlachas et al, 2018; Watson, 2019; Lu et al, 2018; Chattopadhyay et al, 2019) and parameter optimization (e.g., Schevenhoven and Selten, 2017) Both the questions we raise are of direct relevance to climate applications. Could they learn the influence of unprecedented greenhouse-gas concentrations on the dynamics of the climate system, given a past record of the system subjected to varying greenhouse-gas levels?
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
