A new adaptive controller based on distributed deep reinforcement learning for PEMFC air supply system

Abstract

To better control the PEMFC air flow, this paper introduces a distributed deep reinforcement learning-based adaptive proportional integral (PI) controller for controlling air flow in a proton exchange membrane fuel cell (PEMFC). In this paper a novel algorithm for tuning the controller is proposed, termed the multi-role exploration strategy distributed deep deterministic policy gradient (MESD-DDPG), which builds upon the deep deterministic policy gradient (DDPG) by employing various trick including a multi-role exploration strategy. In this strategy, explorers and followers with different exploration principles are used for distributed exploration to enhance the exploration efficiency. Moreover, a probability experience replay mechanism is included to foster the training efficiency. In addition, hot start mechanism of experience pool is applied to deal with the reward sparsity problem. The MESD-DDPG also includes a number of innovations for solving the problem of Q-value overestimation. The proposed MESD-DDPG algorithm considers the PI coefficients naturally in the objective and enables the controller better online coefficient adjusting ability via learning, which leads to improved controller adaptability. The simulation results demonstrate the adaptability and the excellent performance of the MESD-DDPG adaptive PI controller.