Fit for purpose: Modeling wholesale electricity markets realistically with multi-agent deep reinforcement learning

Abstract

Electricity markets need to continuously evolve to address the growing complexity of a predominantly renewable energy-driven, highly interconnected, and sector-integrated energy system. Simulation models allow testing market designs before implementation, which offers advantages for market robustness and efficiency. This work presents a novel approach to simulate the electricity market by using multi-agent deep reinforcement learning for representing revenue-maximizing market participants. The learning capability makes the agents highly adaptive, thereby facilitating a rigorous performance evaluation of market mechanisms under challenging yet practical conditions. Through distinct test cases that vary the number and size of learning agents in an energy-only market, we demonstrate the ability of the proposed method to diagnose market manipulation and reflect market liquidity. Our method is highly scalable, as demonstrated by a case study of the German wholesale energy market with 145 learning agents. This makes the model well-suited for analyzing large and complex electricity markets. The capability of the presented simulation approach facilitates market design analysis, thereby contributing to the establishment future-proof electricity markets to support the energy transition.