Application of Gradient Descent Continuous Actor-Critic Algorithm for Bilateral Spot Electricity Market Modeling Considering Renewable Power Penetration

Huiru Zhao,Qingkun Tan,Mingrui Zhao,Chuyu Sun,Yuwei Wang

doi:10.3390/a10020053

Huiru Zhao, Qingkun Tan + Show 3 more

Open Access

https://doi.org/10.3390/a10020053

Copy DOI

Abstract

The bilateral spot electricity market is very complicated because all generation units and demands must strategically bid in this market. Considering renewable resource penetration, the high variability and the non-dispatchable nature of these intermittent resources make it more difficult to model and simulate the dynamic bidding process and the equilibrium in the bilateral spot electricity market, which makes developing fast and reliable market modeling approaches a matter of urgency nowadays. In this paper, a Gradient Descent Continuous Actor-Critic algorithm is proposed for hour-ahead bilateral electricity market modeling in the presence of renewable resources because this algorithm can solve electricity market modeling problems with continuous state and action spaces without causing the “curse of dimensionality” and has low time complexity. In our simulation, the proposed approach is implemented on an IEEE 30-bus test system. The adequate performance of our proposed approach—such as reaching Nash Equilibrium results after enough iterations of training are tested and verified, and some conclusions about the relationship between increasing the renewable power output and participants’ bidding strategy, locational marginal prices, and social welfare—is also evaluated. Moreover, the comparison of our proposed approach with the fuzzy Q-learning-based electricity market approach implemented in this paper confirms the superiority of our proposed approach in terms of participants’ profits, social welfare, average locational marginal prices, etc.

Highlights

In order to further enhance competitiveness, in recent years the bilateral spot electricity market (EM) has been introduced and utilized to improve restructuring in the power industry of many countries [1]
Considering renewable resource penetration, these highly random, intermittent, and non-dispatchable power resources make it more difficult to develop a proper EM modeling approach, which is a necessary tool for decision-making analysis, market simulation, relevant policy design analysis, etc. [4,5,6]
In a bilateral spot EM with renewable power penetration, non-renewable power generation companies (NRGenCOs) and distribution companies must bid in this stochastically fluctuating environment of renewable power generation in order to improve their own profits

Summary

Introduction

In order to further enhance competitiveness, in recent years the bilateral spot electricity market (EM) has been introduced and utilized to improve restructuring in the power industry of many countries [1]. Based on the actor-critic structure [40], state and action spaces can be made continuous by using a linear combination of many basis functions. The detailed mathematical principle of GDCAC algorithm can be described as follows: By using a linear function [40], we estimate and repeatedly update in an agent’s critic part a value function defined by the continuous state spaceX: n V (x) =. Θn )T ∈ Rn. By using a linear function [40], we estimate and repeatedly update in an agent’s actor part an optimal policy function Â : X → U defined by the continuous state space X: ux (optimal ) = Â(x) = φ(x) T ω x ∈ X,

Objectives

Discussion

Conclusion