A pseudo-inverse decomposition-based self-organizing modular echo state network for time series prediction

Abstract

Echo state network (ESN) refers to a popular recurrent neural network with a largely and randomly generated reservoir for its rapid learning ability. However, it is difficult to design a reservoir that matches a specific task. To solve the structure design of the reservoir, a pseudo-inverse decomposition-based self-organizing modular echo state (PDSM-ESN) is proposed. PDSM-ESN is constructed by growing–pruning method, where the error and condition number are used, respectively. Since the self-organizing process may negatively affect the learning speed, the pseudo-inverse decomposition is adopted to improve learning speed, which means the output weights are learned by an iterative incremental method. Meanwhile, to solve the ill-posed problem, the modular sub-reservoirs corresponding to the high condition number are pruned. Simulation results indicate that PDSM-ESN has better prediction performance and run-time complexity compared with the traditional ESN models.