Machine learning analytics for virtual bidding in the electricity market

Abstract

In order to solve the problem of the high risks and low efficiency caused by the inconsistency of the day-ahead and real-time prices in two-settlement electricity market, virtual bidding is used to arbitrage on the difference between such two market prices that are unknown to virtual bidders to promote the price convergence. The problem of optimal bidding for virtual bidders from the spatio-temporal dimensions is addressed in this paper. The model takes the budget constraints of virtual bidders into account, as well as considers decrement and increment bids of virtual bidding to maximize the cumulative payoff of virtual bidders, which is formulated as a Markov Decision Process problem. Meanwhile, the conditional value-at-risk is used to quantify and hedge the risks faced by virtual bidders. A deep reinforcement learning algorithm is used to achieve an effective solution to the optimal bidding strategy problem through continuous interaction with a simulated building environment to obtain feedback and update the parameters of the neural network without referring to any prior model knowledge. The PJM data from 2016 to 2018 is used to calculate the cumulative profits and Sharpe ratio of virtual bidders. Compared with greedy algorithm and dynamic programming, the deep reinforcement learning algorithm is verified the effectiveness and superiority in this paper.