Deep Reinforcement Learning Control for Pulsed Power Load Online Deployment in DC Shipboard Integrated Power System

Abstract

Online deployment of pulsed power load (PPL) is one of the most challenging issues in DC shipboard integrated power systems (SIPSs), which leads to a multi-objective optimal control problem subject to various constraints in this paper. Since traditional model-based methods face difficulties in designing the optimal control policy and are prone to model inaccuracy and parameter uncertainty, there is an urgent need for a model-free and also high-performance control approach. Thus, a deep reinforcement learning (DRL) optimal control, which employs the twin-delayed deep deterministic policy gradient (TD3) algorithm, is presented in this paper. The DRL optimal control adopts a stack-based state observation technique to enhance learning and control performance, and it uses a multi-objective reward function design to signify the overall dynamic performance. Besides achieving the safe and fast online deployment of PPL, it also fulfills the regulation of DC bus voltage and the proportional current sharing among distributed generations (DGs). Moreover, the DRL control has an advantage in handling the ramp rate constraints of SIPS. The optimal control satisfying ramp rate constraints can be obtained through a deep learning process. The performance of the proposed DRL control is validated by case studies considering different load conditions.