Managing chance-constrained hydropower with reinforcement learning and backoffs

Abstract

The control of multi-reservoir hydropower systems is a crucial cogwheel in the current transition of power systems towards renewable energy. In particular in northern regions where inflows vary enormously through the year, the satisfaction of storage constraints cannot be ensured at all times. We thus add chance constraints to the underlying Markov Decision Process problem, and solve it with a Reinforcement Learning, policy gradient approach. “Backoffs”, common in other areas of optimal control, are introduced to better manage the numerous chance constraints as a single joint constraint. Stochastic Dynamic Programming (SDP) is used as a benchmark approach. We show numerically that our approach can deliver high quality policies, and just as importantly, that a three-reservoir system is solved in proportionally little more time than a one-reservoir system.