Adaptive forgetting factor echo state networks for time series prediction

Abstract

Echo state networks ESN are an emerging learning technique proposed for generalised single-hidden layer feed forward networks SLFNs. However, the conventional ESN ignores training data timeliness, which may reduce prediction accuracy for time varying data. To solve this problem, a novel algorithm based on ESN with adaptive forgetting factor AF-ESN is proposed. The adaptive forgetting factor is introduced to ESN sequential learning phase, which automatically tunes the valid training data window size according to prediction error magnitude. A comparison of the proposed AF-ESN with other algorithms is evaluated on three chaotic time series and an actual time series. Compared with conventional ESN and FOS-ELM online sequential extreme learning machine with forgetting mechanism, though AF-ESN consumes much computation time, AF-ESN provides the highest prediction accuracy with high stability.