Echo state networks with Tikhonov regularization: optimization using integral gain

Abstract

This study proposes feed-forward echo state networks (ESN) as an estimator, and couples it with second-order proportional-integral-derivative (PID) feedback extension to compensate for dead time in feedback systems. The system is tested for two-dimensional space motion patterns recognition and prediction using simulations, which allows control of noise input. Tikhonov regularization is employed for training readouts and second-order PID feedback minimizes prediction bias. Evaluation is done using mean squared error and the coupled system performs well compared to any of its standalone versions. The results suggest it is feasible to (1) ‘compress’ the memory capacity of the system, and (2) reduce the number optimization parameters, while maintaining the estimation performance and following the excitation property of the estimator. It is feasible to optimize the ESN using feedback gain although it plays a significant role in the proposed system because the improvement by bias correction is far greater than that of optimization; thus, simplifying the estimation to a feedback problem which is easily tuned using the Ziegler–Nichols method.