Functional echo state network for time series classification

Abstract

Echo state networks (ESNs) are a new approach to recurrent neural networks (RNNs) that have been successfully applied in many domains. Nevertheless, an ESN is a predictive model rather than a classifier, and methods to employ ESNs in time series classification (TSC) tasks have not yet been fully explored. In this paper, we propose a novel ESN approach named functional echo state network (FESN) for time series classification. The basic idea behind FESN is to replace the numeric variable output weights of an ESN with time-varying output-weight functions and introduce a temporal aggregation operator to the output layer that can project temporal signals into discrete class labels, thereby transforming the ESN from a predictive model into a true classifier. Subsequently, to learn the output-weight functions, a spatio-temporal aggregation learning algorithm is proposed based on orthogonal function basis expansion. By leveraging the nonlinear mapping capacity of a reservoir and the accumulation of temporal information in the time domain, FESN can not only enhance the separability of different classes in a high-dimensional functional space but can also consider the relative importance of temporal data at different time steps according to dynamic output-weight functions. Theoretical analyses and experiments on an extensive set of UCR data were conducted on FESN. The results show that FESN yields better performance than single-algorithm methods, has comparable accuracy with ensemble-based methods and exhibits acceptable computational complexity. Interestingly, for some time series datasets, we visualized some interpretable features extracted by FESN via specific patterns within the output-weight functions.