Data-driven tracking control design with reinforcement learning involving a wastewater treatment application

Abstract

With the increase of urbanization rate, the problem of water shortage and pollution is more and more serious. It is important to improve the efficiency of wastewater treatment to protect the urban ecological environment. The wastewater treatment process involves a variety of biochemical reactions and has strong time-vary dynamics. The concentration design in the wastewater treatment process can be regarded as a tracking control problem for a class of nonlinear systems. In order to solve this problem, this paper develops an intelligent control method with tracking goal representation heuristic dynamic programming (T-GrHDP) by combining the GrHDP with a novel tracking framework. A model network is built by using a dataset consisting of real input and output data of the controlled object, which can overcome the dependence on the system dynamic. In order to improve the learning efficiency of the proposed algorithm, we introduce the goal network to provide more effective information for the critic network. The classical actor-critic scheme in reinforcement learning is used to obtain the approximate optimal control strategy. By introducing some necessary lemmas and assumptions, the convergence of the proposed algorithm is proved. Finally, the T-GrHDP method is successfully applied in two industrial simulations including the wastewater treatment system.