A bi-level reinforcement learning model for optimal scheduling and planning of battery energy storage considering uncertainty in the energy-sharing community

Abstract

Sharing of battery energy storage systems (BESS) in the energy community by reflecting the real world can play a significant role in achieving carbon neutrality. Therefore, this study aimed to develop a bi-level reinforcement learning (RL) model of BESS considering uncertainty in the energy-sharing community for the following optimization strategies: (i) short-term scheduling model for optimal electricity flows considering operational objectives (i.e., self-sufficiency rate (SSR), peak load, and economic profit); and (ii) long-term planning model for optimal BESS plan (i.e., install, replace, and disuse) along with battery types (new or reused batteries). A case study in the South Korea Nonhyeon neighborhood was conducted to evaluate the developed bi-level RL model feasibility based on future scenarios considering the time-dependent variables. The developed model increased economic profit by up to 18,830 USD compared to the rule-based model. Compared to the case where BESS was not installed, SSR increased by up to 7.79% and peak demand decreased by up to 1.31 kWh. These results show that the developed model could maximize the economic feasibility of community-shared BESS by reflecting the uncertainty in the real world, ultimately benefiting participants in the energy-sharing community.