Markov game approach for multi‐agent competitive bidding strategies in electricity market

Abstract

In a competitive electricity market, suppliers seek the optimal bidding strategy, maximising their individual profit. Due to the uncertainties, unknown parameters and dynamics of electricity market, the optimal bidding strategies cannot be attained through straightforward optimisation methods. In this study, the supply function equilibrium model is considered together with uniform price market clearing mechanism. The interaction among suppliers is modelled as an incomplete information game problem and multi-agent reinforcement learning (MARL) is utilised to find the optimal bidding strategy of the suppliers in a non-cooperative game. The proposed framework is an extension, from single agent Markov decision process to Markov game process, which is suitable for studying multi-agent decision making problems in stochastic environments. In this study, a Markov game model of the electricity market is proposed and a new state-action and Markov model of electricity market is proposed to built up the MARL environment. In addition, a learning-based non-cooperative MARL method is utilised for learning the optimal bidding strategies of the suppliers in a day-ahead electricity market. The proposed method is successfully applied to IEEE-30-bus power system. To examine the statistical significance of the results, the T-test method is applied to compare the competitive behaviour of the players in both multi- and single-agent frameworks.