Self-evolution direct thrust control for turbofan engine individuals based on reinforcement learning methods

Abstract

With the development of high-performance aero engines, the need for faster and more accurate thrust control has become increasingly critical. Remarkable achievements have been made in direct thrust control (DTC) for turbofan engines to meet these demands in recent years. However, most model-based methods only focus on the control law optimization to pursue better thrust control performance and ignore the model difference between the nominal model and the real engine. When there exist individual differences, the offline-designed control laws may suffer from significant online control performance degeneration problems. This paper proposes a novel self-evolution direct thrust control (SDTC) architecture to solve this problem, in which an uncertainty-aware state prediction model (USPM) and a self-evolution thrust controller are included. The USPM is proposed to predict the engine states of the individual engine, in which bootstrapped neural network (NN) ensemble methods are applied to deal with the epistemic uncertainty during the online identification process. To solve the performance degeneration problem, a novel sample evaluation prediction control (SEPC) and online policy evolution (OPE) reinforcement learning (RL) algorithm are proposed. When the nominal control policy is first applied in the onboard controller, the SEPC algorithm will temporarily adjust the output action in order to achieve a stable dynamic performance based on the prediction results of the USPM. The OPE algorithm updates the policy periodically and makes sure the policy is adapted to the new characteristics. Numerical simulation shows that the USPM can achieve high prediction accuracy under epistemic uncertainties, which helps the SEPC achieve satisfactory control performances under individual differences. The OPE algorithm can facilitate policy adaptation to individual characteristics with relatively low computational cost. Compared with other methods, our proposed SDTC architecture takes the lead in both performance and complexity and is instructive for other DTC researches.