Reinforcement-Learning Based Threshold Policies for Continuous Intraday Electricity Market Trading

Abstract

Continuous intraday electricity market h as become increasingly important in recent years, due to the increasing integration of renewable resources in power systems. Trading in this market is challenging due to the multistage nature of the problem, its high uncertainty, and the fact that decisions need to be made rapidly in order to lock in profitable trades. We cast the problem of trading in continuous intraday markets as a reinforcement learning problem, and tackle the problem using policy function approximation. We specifically parametrize the trading policy using price thresholds, and optimize the choice of these thresholds using the REINFORCE algorithm. We demonstrate the effectiveness of our proposed policy by showing that it outperforms the method, classically used in the industry, rolling intrinsic of 4.2% (out of sample) on the 165 last days of 2015 in the German continuous intraday market.