Tackling the Trade-Off Between Information Processing Capacity and Rate in Delay-Based Reservoir Computers

Silvia Ortín,Luis Pesquera

doi:10.3389/fphy.2019.00210

Silvia Ortín, Luis Pesquera

Open Access

https://doi.org/10.3389/fphy.2019.00210

Copy DOI

Abstract

We study the role of the system response time in the computational capacity of delay-based reservoir computers. Photonic hardware implementation of these systems offers high processing speed. However, delay-based reservoir computers have a trade-off between computational capacity and processing speed %an intrinsic speed limitation due to the non-zero response time of the nonlinear node. The reservoir state is obtained from the sampled output of the nonlinear node. We show that the computational capacity is degraded when the sampling output rate is higher than the inverse of the system response time. We find that the computational capacity depends not only on the sampling output rate but also on the misalignment between the delay time of the nonlinear node and the data injection time. We show that the capacity degradation due to the high sampling output rate can be reduced when the delay time is greater than the data injection time. We find that this mismatch gives an improvement of the performance of delay-based reservoir computers for several benchmarking tasks. Our results show that the processing speed of delay-based reservoir computers can be increased while keeping a good computational capacity by using a mismatch between delay and data injection times. It is also shown that computational capacity for high sampling output rates can be further increased by using an extra feedback line and delay times greater than the data injection time.

Highlights

Reservoir computing (RC) is a successful brain-inspired concept to process information with temporal dependencies [1, 2]
We show that memory capacity can be further increased with this architecture for small values of θ/T when the delay time is greater than the information processing time
We have shown that the computational capacity is degraded when the sampling output rate is higher than the inverse of the system response time

Summary

Introduction

Reservoir computing (RC) is a successful brain-inspired concept to process information with temporal dependencies [1, 2]. RC conceptually belongs to the field of recurrent neural networks (RNN) [3]. In these systems, the input signal is non-linearly projected onto a high-dimensional state space where the task can be solved much more than in the original input space. The high-dimensional space is typically a network of interconnected non-linear nodes (called neurons). The ensemble of neurons is called the reservoir. RC implementations are generally composed of three layers: input, reservoir, and output (see Figure 1). The input layer feeds the input signal to the reservoir via fixed weighted connections. The input weights are often chosen randomly. These weights determine how strongly each of the inputs couples to each of the neurons.

Methods

Results

Conclusion