Developing Optimal Energy Arbitrage Strategy for Energy Storage System Using Reinforcement Learning

Abstract

This paper introduced a reinforcement learning based method for developing operational strategy for an energy storage system (ESS) to achieve energy arbitrage in a microgrid or power system. In comparison to conventional energy resources such as gas turbines units or wind plant, it is more challenging to design an optimal strategy for ESS because of their limited energy and the impact of future electricity prices. The energy arbitrage problem also presents unique challenges than the economic dispatch problem because the ESS owner has very limited information of the system compared to those available to grid operators. In this work, reinforcement learning method was applied to determine the best time for charge/discharge in order to maximize the profit. Moreover, different scenarios were designed, and the performance of proposed reinforcement learning algorithm was analyzed by comparing the results with those of optimization-based methods.