Reinforcement learning-based optimal scheduling model of battery energy storage system at the building level

Abstract

Installing the battery energy storage system (BESS) and optimizing its schedule to effectively address the intermittency and volatility of photovoltaic (PV) systems has emerged as a critical research challenge. Nonetheless, some existing studies still have limitations in terms of the efficiency of the BESS scheduling due to the lack of comprehensive consideration of diverse user objectives. As a response to this gap, this study aimed to develop a reinforcement learning (RL)-based optimal scheduling model to better reflect the continuous behaviors in the complex real world. To this end, focused on residential buildings connected to the grid and equipped with a BESS and PV system, its optimal scheduling models were developed using four algorithms from among the various RL techniques according to training methods. The results of the case study showed that the developed RL-based optimal scheduling model using Proximal Policy Optimization (PPO) can be applied to effectively operate the BESS with a PV system, considering possible uncertainties in the real world. The case study demonstrated the effectiveness and feasibility of the developed RL-based optimal scheduling model. Compared to other algorithms, the PPO-based RL model has better decision-making for optimal BESS scheduling strategies to maximize their self-sufficiency rate and economic profits by coping with changing variables in the real world. Therefore, the RL-based BESS scheduling model will offer an optimal solution, specifically tailored for use within a virtual power plant, where numerous buildings continuously share electricity.