Echo State Networks: Novel reservoir selection and hyperparameter optimization model for time series forecasting

Abstract

The use of computational intelligence models for multi-step time series forecasting tasks has presented satisfactory results in such a way that they are considered models with an excellent future for this type of problem. From the point of view of computational cost, the current alternatives combined with classical models are generating hybrid models that present even better results. Within the AutoML category, the optimization of hyperparameters and the selection of network topologies has become a challenge. Reservoir Computing, which is within the area of ​​Recurrent Neural Networks (RNN), proposes a particular model called Echo State Networks. which has been tested in different applications with excellent results; however, the difficulty in specifying the hyperparameters has been the subject of continuous study given the random nature of the set of neurons called Reservoir. Based on the Separation Ratio Graph (SRG) model for performance analysis, this paper proposes a new model, called Echo State Network - Genetic Algorithm - Separation Ratio Graph (ESN-GA-SRG), which optimizes network hyperparameters and at the same time selects the best topology for the Reservoir using the SRG coefficient, to find the reservoir that offers the most suitable dynamic behavior. The performance of this new model is evaluated on forecasting two sets of time series benchmarks with different characteristics of sampling periodicity, skewness, and stationarity. The results obtained show that the ESN-GA-SRG model was superior in predicting these time series in most cases, with statistical significance, when compared to other models that have been presented for this type of problem in the literature.