Multi-agent simulation for strategic bidding in electricity markets using reinforcement learning

Abstract

In this paper, Multi-agent Simulation (MAS) and Reinforcement Learning (RL) are studied, a theoretical framework of MAS is proposed for strategic bidding in electricity markets using reinforcement learning, which consists of two parts: one is a MAS system used to simulate the competitive bidding of the actual electricity market; the other is an adaptive learning strategy bidding system used to provide agents with more intelligent bidding strategies. An Experience-Weighted Attraction (EWA) reinforcement learning algorithm (RLA) is applied to the MAS model and a new MAS method is presented for strategic bidding in electricity markets using a new Improved EWA (IEWA). From both qualitative and quantitative perspectives it is compared with other three MAS methods using the Roth-Erev (RE), Q-learning and EWA. The result shows that the performance of the MAS method using IEWA is proved to be better than the others. The four MAS models using four RLAs are built for strategic bidding in electricity markets. Through running the four MAS models, the rationality and correctness of the four MAS methods are verified for strategic bidding in electricity markets using reinforcement learning.