Optimal energy storage system control using a Markovian degradation model—Reinforcement learning approach

Abstract

The degradation property of an energy storage system (ESS) has a decisive impact on the economic benefits of ESS operation. However, existing degradation models either do not fully reflect the effects of battery usage or require a full operation history of the battery. This study proposes an accurate Markovian model for battery degradation that reflects battery usage and requires only the current state. We demonstrate the use of the model by establishing a Markov decision process based on the proposed Markovian degradation model and solving it with deep reinforcement learning algorithms. Our intelligent agent, which aims to minimize the sum of electricity cost and degradation cost, generates cost savings of 5%–29% compared to baseline strategies. This framework offers an optimal ESS operation strategy considering battery degradation in a tractable and practical way.