Online adaptive dynamic programming based on echo state networks for dissolved oxygen control

Abstract

To solve the control problem of dissolved oxygen concentration in wastewater treatment process (WWTP), an online adaptive dynamic programming (ADP) scheme based on echo state networks (ESN) is proposed. The ESN-based ADP controller contains three modules composed of echo state networks, and the three modules approximate respectively the critic index, the system model and the optimal control policy. Further, an online learning algorithm on the basis of recursive least squares (RLS) is designed to train the ESN-based ADP controller, and also the convergence of the online learning algorithm is investigated. The design and operation of the ESN-based ADP controller depend mainly on the online data observed from controlled system, and minor prior knowledge is required. The proposed ESN-based ADP controller’s performance is tested and evaluated on Benchmark Simulation Model no. 1 (BSM1)—a general WWTP benchmark. Experimental results demonstrate that the proposed controller can achieve high performance on dissolved oxygen level control in WWTP.