Optimal battery charge with safe exploration

Abstract

Li-ion batteries have become the standard power source for electric vehicles (EVs) as the alternative of choice to reduce CO2 emissions. But before becoming a reliable technology, Li-ion batteries must deal with two significant challenges: undesirable electrochemical reactions caused by excessive charging rates and considerable time for an EV to get charged. It is necessary to employ balanced current profiles that prevent both serious battery degradation effects and the inconvenience to end users. In this work, the authors propose a safe exploration deep reinforcement learning (SDRL) approach in order to determine optimal charging profiles under variable operating conditions. One of the main advantages of RL techniques is that they can learn from interaction with the real or simulated system while incorporating the nonlinearity and uncertainty derived from fluctuating environmental conditions. However, since RL techniques must explore undesirable states before obtaining an optimal policy, no safety guarantees are provided. The proposed approach aims at maintaining zero-constraint violations throughout the learning process through the integration of a safety layer that corrects the action if a constraint is likely to be violated. The proposed method is tested in the equivalent circuit of a Li-ion battery under varying conditions. Results reveal that with the integration of this safety layer, SDRL is able to find safe optimized charging policies while considering a trade-off between the charging speed and the battery lifespan, including a 30% reduction in charging time while still maintaining temperatures within permissible limits and up to 38% of battery life conservation, compared with benchmark methods. Moreover, our approach does not experience episodes demonstrating restriction violations throughout the pre-training, training, and evaluation phases.