Deep reinforcement learning-based scheduling for integrated energy system utilizing retired electric vehicle battery energy storage

Abstract

Retired electric vehicle batteries (REVBs) retain substantial energy storage capacity, holding great potential for utilization in integrated energy systems. However, the dynamics of supply and demand, alongside battery safety constraints, present challenges to the optimal dispatch of energy. This paper proposes a hybrid system including thermal and electric energy employing REVB as the energy storage component. This system relies on photovoltaics (PV) technology to harness solar energy and convert it into thermal and electrical energy for supplying users. The REVB, as the core component of the system, functions to regulate the allocation of energy by managing its charging and discharging behavior. To explore the optimal energy scheduling strategy, a two-stage deep reinforcement learning (DRL) optimization method is presented. In the first phase, a deep deterministic policy gradient (DDPG) method is introduced to determine the REVB's behavior, and in the second phase, a competitive bidding mechanism is presented to optimally distribute energy. Simulation results, validated with real scenarios and data, achieve a reduction of 35.2 % in energy dissipation and 4.6 % in users' expenditure for purchasing energy compared to standard strategy. Simultaneously, experiments have shown that REVB is nearly as efficient as new battery as energy storage component, with energy wastage only about 1.1 % higher and users' expenditure less than 1 % higher.