Optimal operation and maintenance of energy storage systems in grid-connected microgrids by deep reinforcement learning

Abstract

The operation of microgrids, i.e., energy systems composed of distributed energy generation, local loads and energy storage capacity, is challenged by the variability of intermittent energy sources and demands, the stochastic occurrence of unexpected outages of the conventional grid and the degradation of the Energy Storage System (ESS), which is strongly influenced by its operating conditions. To effectively address these challenges, a novel method for combined operation and maintenance management of ESS has been developed. Unlike the currently available solutions, which typically address the one-day-ahead scheduling problem, the present work considers, for the first time, the realistic case of a microgrid in which the ESS degrades and unexpected outages of the conventional grid can occur along the long-time horizons of the entire microgrid lifetimes. The proposed method, which is based on deep reinforcement learning, is tested on a simulated grid-connected microgrid of a residential building equipped with photovoltaic modules and an ESS. The method outperforms other state-of-the-art approaches based on heuristics and metaheuristics by increasing the profit by 15% and reducing the average number of ESS replacements during its lifetime. Therefore, it can be concluded that the proposed DRL-based framework allows achieving prescriptive maintenance since the suggested actions are optimal from the point of view of effectively maximizing the profit and minimizing the maintenance interventions over the entire lifetime of the microgrid.