Design of cost-based sizing and energy management framework for standalone microgrid using reinforcement learning

Abstract

The standalone photovoltaic-battery energy storage (PV-BES) microgrid has gained substantial interest recently due to its ability to provide uninterrupted power to consumers in remote areas. In such microgrids, components must be precisely sized and energy must be supplied most cost-effectively at all times. This paper presents a cost-based framework for determining the optimal size and energy management of standalone microgrids using reinforcement learning. Fundamental to this framework is two essential phases; the first is finding the best size of PV-BES using an analytical and economic sizing (AES) model based on minimum levelized cost of energy (LCOE). The AES phase is then followed by optimizing the energy management strategy (EMS) of the microgrid using reinforcement learning to provide optimum cost savings. The novelty in this work can be outlined as optimizing both the size and EMS of a standalone PV-BES microgrid using the AES model and Q-learning in an integrated framework. This can lead to improved performance demonstrated in reducing the LCOE, decreasing diesel generator working hours, and enhancing PV utilization and system efficiency. The results show an advantageous reduction in total cost while meeting load requirements. Additionally, the proposed framework is evaluated using several metrics to measure the impact of employing Q-learning against the AES-finite automata model. For instance, a decrease of 22% in diesel generator working hours and an increase of 6% in PV utilization while a reduction of 11% in the LCOE is accomplished. On the other hand, the proposed framework is examined against two rule-based EMSs, load following strategy (LFS) and cycle charging strategy (CCS), and outperforms these two EMSs in terms of LCOE, PV utilization, and system efficiency.