A Reinforcement Learning controller optimizing costs and battery State of Health in smart grids

Abstract

Smart Grids are the evolution of traditional electric grids and allow two-way flows of electricity and information between different actors. At the edge of this network, customers can both produce and consume energy. Due to the intermittent nature of renewable energy sources, customers are characterized by moments of energy surplus and deficit. To solve this problem, customers are connected to the power grid, and, usually, they are also provided with Lithium-Ion battery packs positioned near the energy source used to store energy in excess for later use, reducing expensive energy exchanges with the grid. On the one hand, using the battery at its full capabilities produces significant economic savings. On the other hand, massive use of the battery leads to degradation and consequently to a more frequent substitution of the battery. Therefore, depending on the cost of energy and batteries, one should carefully choose when it is favorable to use it. To avoid inefficiencies, it is common to design controllers that regulate the energy flow within the battery packs, deciding whether to exchange energy with the network or store it in the battery. In this work, a Reinforcement Learning controller optimizing energy flow is developed. The controller’s goal is to balance the costs of exchanges with the power grid and those derived from the degradation due to battery usage. A synthetic experimental campaign conducted using real-world data demonstrates that the policy learned shows an improvement in the worst-case of 3% w.r.t. state-of-the-art baselines.