Intelligent Controller Based on Distributed Deep Reinforcement Learning for PEMFC Air Supply System

Abstract

Air supply system is an important subsystem in a PEMFC engine system. Research on the control strategy of air supply system is of great importance and significance in engineering. In this paper an intelligent controller based on distributed deep reinforcement learning which exerts better control over the air flux of a proton exchange membrane fuel cell (PEMFC) air supply system is proposed. In addition, a collective intelligence exploration distributed multi-delay deep deterministic policy gradient (CIED-MD3) algorithm is presented for the controller. This improved algorithm is developed on the basis of deep deterministic policy gradient (DDPG) and adopted the collective intelligence exploration policy which enables full exploration of the environment. This classification experience replay mechanism is introduced to improve training efficiency. A number of techniques are employed in an effort to address the Q-value overestimation problem of the DDPG, including clipped multiple Q-learning, delayed update of policy and smooth regularization of target policy. Finally, the application of CIED-MD3 (with its better global search ability and optimization speed) is demonstrated to the model-free PEMFC air flux intelligent controller. The simulation results show that the proposed controller exerts greater control of the PEMFC air supply system. Compared with other control methods, the proposed intelligent controller exhibits better control performance and robustness. The control algorithm proposed in this paper is of significance to future PEMFC air flux control research.