Reinforcement Learning-Based Bi-Level strategic bidding model of Gas-fired unit in integrated electricity and natural gas markets preventing market manipulation

Abstract

Due to its efficient operation and environment-friendly characteristic, gas-fired unit (GFU) plays a more and more important role in electric power systems and natural gas systems. To investigate the performance of GFU's participation in integrated electricity and natural gas markets, a bi-level strategic bidding model considering price and quantity factors is proposed. With the increasing participation of consumers in electricity markets, demand response (DR) management is implemented in the electricity market clearing process and user comfort level (UCL) is considered in the market clearing model. Since GFU participates in both the electricity market and the natural gas market, a local marginal price penalty (LMPP) variable is defined in this paper to prevent potential market manipulation (MM) of GFU. Then a modified reinforcement learning (RL)-based method is proposed to solve the model, combining deep deterministic policy gradient (DDPG) algorithm with autocorrelated noise. Test results on an integrated electricity-gas system show that the proposed method can reflect the strategic behaviors of GFU effectively. The proposed method has better performance than traditional DDPG algorithm with Gaussian noise and the Deep Q-Network (DQN) algorithm. And electricity markets with LMPP can save about 3.03% in generation cost by preventing MM of GFU.