Evolutionary computing assisted deep reinforcement learning for multi-objective integrated energy system management

Abstract

This paper investigates the multi-objective optimal operation problem of an integrated energy system (IES) which integrates grid-connected photovoltaic (PV) generator, gas boiler, battery energy storage system, and thermal storage to satisfy energy demand in forms of electricity and heat. To handle the changes from the system uncertainty (e.g., PV generation, electrical loads, thermal loads, etc.) and unknown thermal dynamic model for temperature control, deep reinforcement learning-based model-free optimization method is proposed to solve the multi-objective optimization problem in which the multi-objective optimization problem is firstly formulated as a multi-objective Markov decision process (MDP) problem. The multi-objective MDP problem is converted to many single-objective MDP problems by the sum technique which are solved by multi-agent deep deterministic policy gradient (DDPG) algorithm. To improve the performance of multi-agent DDPG algorithm, evolutionary computing-based parameter-tuning method is further proposed to fine-tune the policy parameters in DDPG algorithm. The proposed methods are verified on real data. Experiments results illustrate that the multi-agent DDPG algorithm can efficiently solve the multi-objective optimal operation problem of the IES while the evolutionary computing-based policy parameter-tuning method can further improve the approximation of Pareto frontier.