Optimizing echo state network with backtracking search optimization algorithm for time series forecasting

Abstract

The echo state network (ESN) is a state-of-the art reservoir computing approach, which is particularly effective for time series forecasting problems because it is coupled with a time parameter. However, the linear regression algorithm commonly used to compute the output weights of ESN could usually cause the trained network over-fitted and thus obtain unsatisfactory results. To overcome the problem, we present four optimized ESNs that are based on the backtracking search optimization algorithm (BSA) or its variants to improve generalizability. Concretely, we utilize BSA and its variants to determine the most appropriate output weights of ESN given that the optimization problem is complex while BSA is a novel evolutionary algorithm that effectively unscrambles optimal solutions in complex spaces. The three BSA variants, namely, adaptive population selection scheme (APSS)–BSA, adaptive mutation factor strategy (AMFS)–BSA, and APSS&AMFS–BSA, were designed to further improve the performance of BSA. Time series forecasting experiments were performed using two real-life time series. The experimental results of the optimized ESNs were compared with those of the basic ESN without optimization, and the two other comparison approaches, as well as the other existing approaches. Experimental results showed that (a) the results of the optimized ESNs are more accurate than that of basic ESN and (b) APSS&AMFS–BSA–ESN nearly outperforms basic ESN, the three other optimized ESNs, the two comparison approaches, and other existing optimization approaches.