Abstract
A new explanation of the geometric nature of the reservoir computing (RC) phenomenon is presented. RC is understood in the literature as the possibility of approximating input-output systems with randomly chosen recurrent neural systems and a trained linear readout layer. Light is shed on this phenomenon by constructing what is called strongly universal reservoir systems as random projections of a family of state-space systems that generate Volterra series expansions. This procedure yields a state-affine reservoir system with randomly generated coefficients in a dimension that is logarithmically reduced with respect to the original system. This reservoir system is able to approximate any element in the fading memory filters class just by training a different linear readout for each different filter. Explicit expressions for the probability distributions needed in the generation of the projected reservoir system are stated, and bounds for the committed approximation error are provided.
Highlights
M ANY dynamical problems in engineering, signal processing, forecasting, time-series analysis, recurrent neural networks (RNNs), or control theory can be described using input–output (IO) systems
We focus on a promising strategy for empirical system identification known as reservoir computing (RC)
We show that the truncated Volterra series representation admits a state-space representation with linear readouts in a high-dimensional adequately constructed tensor space. We refer to this system as the signature state-affine system (SAS) (SigSAS): on the one hand, it belongs to the SAS family, and on the other hand, it shares fundamental properties with the so-called signature process from the theory of rough paths, which inspired the title of the article
Summary
M ANY dynamical problems in engineering, signal processing, forecasting, time-series analysis, recurrent neural networks (RNNs), or control theory can be described using input–output (IO) systems. In which access to all the variables that determine the behavior of the systems is difficult or impossible, or when a precise mathematical relationship between input and output is not known, it has proved more efficient to carry out the system identification using generic families of models with strong approximation abilities that are estimated using observed data This approach, which we refer to as empirical system identification, has been developed using different techniques coming simultaneously from engineering, statistics, and computer science. We show that the truncated Volterra series representation (whose associated truncation error can be quantified) admits a state-space representation with linear readouts in a (potentially) high-dimensional adequately constructed tensor space We refer to this system as the signature SAS (SigSAS): on the one hand, it belongs to the SAS family, and on the other hand, it shares fundamental properties with the so-called signature process from the (continuous-time) theory of rough paths, which inspired the title of the article. The results in the following show that randomly generated SAS reservoir systems approximate well any sufficiently regular IO system just by tuning a linear readout because they coincide with an errorcontrolled random projection of a higher dimensional Volterra series expansion of that system
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
