Harnessing Non-linearity by Sigmoid-wavelet Hybrid Echo State Networks (SWHESN)

Abstract

We propose one efficient method to improve the performance of Echo State Network (ESN) when as chaotic predictor. ESN is one special Recurrent Neural Network (RNN) with great nonlinear dynamic feature and facile training process, outperforms the previously best techniques applied on chaotic prediction by a factor of 700. In order to expand internal spatial spectrum of this ESN, this method transformed the original ESN into SWHESN (sigmoid-wavelet hybrid ESN) and amplifies the memory capacity (MC) of ESN meanwhile retaining its non-linear feature via injecting some tuned wavelet neurons (wavelons). Experimental result shows SWHESNs possess more robust exploitation period and more stable running situation, with less computing consumption, compared with the original ESN. Using the same data set, SWHESN can predict 46% further than the ESN without typical deviation, but only utilizes 30% of time of what ESN uses.