An associative memory readout for ESNs with applications to dynamical pattern recognition

Abstract

The use of echo state networks (ESN) to find patterns in time (dynamical pattern recognition) has been limited. This paper argues that ESNs are particularly well suited for dynamical pattern recognition and proposes a linear associative memory (LAM) as a novel readout for ESNs. From the class of LAMs, the minimum average correlation energy (MACE) filter is adopted because of its high rejection characteristics that allow its use as a detector in the automatic pattern recognition literature. In the ESN application, the MACE interprets the states of the ESN as a two-dimensional "image", one dimension being time and the other the processing element index (space). An optimal template image for each class, which associates ESN states with the class label, can be analytically computed using training data. During testing, ESN states are correlated with each template image and the class label of the template with the highest correlation is assigned to the input pattern. The ESN-MACE combination leads to a nonlinear template matcher with robust noise performance as needed in non-Gaussian, nonlinear digital communication channels. A real-world data experiment for chemical sensing with an electronic nose is included to demonstrate the validity of this approach. Moreover, the proposed readout can also be used with liquid state machines eliminating the need to convert spike trains into continuous signals by binning or low-pass filtering.