Price-taker Bidding and Pricing Strategy Using Deep Deterministic Policy Gradient Algorithm with Transformer Neural Networks

Abstract

The joint strategy optimization problem of a load serving entity (LSE) in both wholesale electricity market (WEM) and retail electricity market (REM) is converted into an aggregated load prediction problem and a sequential decision optimization problem, under the condition of "price-taker". By formulating the original retail price optimizing problem as a Markov decision process (MDP), a novel deep deterministic policy gradient (DDPG) algorithm combining with a transformer based representation network (DDPG-TSFR) is proposed to solve this MDP. A new network structure is designed for the proposed DDPG-TSFR by using multiple loss functions, and a method based on gradient normalization (GradNorm) is adopted to realize adaptive loss weighting factors. We conduct several numerical experiments to compare the training and computational behaviors of the proposed DDPG-TSFR with different DRL based approaches. Numerical results validate the effectiveness and superiority of the proposed approach.