Optimizing the echo state network based on mutual information for modeling fed-batch bioprocesses

Abstract

Echo state networks (ESNs) have become one of the most effective dynamic neural networks because of its excellent fitting performance in real-valued time series modeling tasks and simple training processes. The original ESN concept used randomly fixed created reservoirs, and this concept is considered to be one of its main advantages. However, ESNs have been criticized for its randomly created connectivity and weight parameters. Determining the appropriate weight parameters for a given task is an important problem. An optimization method based on mutual information (MI) is proposed in this study to optimize the input scaling parameters and the structure of ESN to address the aforementioned problem and improve the performance of ESN. The MI optimization method mainly consists of two parts: First, the scaling parameters of multiple inputs are adjusted based on the MI between the network inputs and outputs. Second, some output weight connections are pruned for optimization based on the MI between reservoir states. Furthermore, three MI-ESN models are proposed for a fed-batch penicillin fermentation process. Our experimental outcomes reveal that the obtained MI-ESN models outperform the ESN models without optimization and other traditional neural networks.