A Direct Reinforcement Learning Approach for Nonautonomous Thermoacoustic Generator

Abstract

For nonautonomous nonlinear systems, the optimal control design is affected by the terms of partial derivative. If a reinforcement learning (RL) strategy is developed to approximate the optimal control scheme in nonautonomous nonlinear systems, then the closed control system might be unstabilizing. Therefore, in this article, the approach of direct RL law for a nonautonomous thermoacoustic generator (TAG) is investigated. We establish the mathematical model of TAG by partial differential equations (PDEs) and then transforming them into time varying nonlinear systems. The direct RL technique with Newton–Leibniz formula is implemented to consider the partial derivative term from classical policy iteration (PI) method by modifying the computation using data collection between the two sampling times. Finally, several simulation studies with some comparisons are conducted to validate the theoretical analyses.