Discriminative and regularized echo state network for time series classification

Abstract

An echo State Network (ESN) is a special structure of a recurrent neural network (RNN) in which the recurrent neurons are randomly connected. ESN models which have achieved a high accuracy on time series prediction tasks can be used as time series prediction models in many domains. Nevertheless, in most ESN models, the input weights are randomly generated and the output weights calculated by the least square method are susceptible to outliers, which cannot guarantee that the ESN models will always be optimal for a given task. In this paper, a novel discriminative and regularized ESN (DR-ESN) combines discriminative feature aggregation (DFA) and outlier-robust weights (ORW) algorithms are proposed for time series classification. DFA is firstly proposed to replace the random input weights of ESN with the constrained weights generated from sample information. In DFA, weight vectors are selected from the vector space spanned by initial input sequence vectors, then the new generated input weights can adequately represent the data features. Secondly, ORW is employed to enhance the robustness of output weights by constraining the weights assigned to samples with large training errors. The weights evaluation and experiments on a massive set of the synthetic time series data, real-world bearing fault data and UCR benchmarks indicate that the proposed DR-ESN can not only considerably improve the original ESN classifier but also effectively suppress the effect of outliers on classification performance.