Application of Q-learning with temperature variation for bidding strategies in market based power systems

Abstract

The electric power industry is confronted with restructuring in which the operation scheduling is going to be decided based on a competitive market. In this new arrangement, bidding strategy has become a major issue. Participants in this deregulated energy market place may be able to compete better by choosing a suitable bidding strategy for trading electricity. Different classical methods for decision making in the uncertain environment of the market can be applied to select a suitable strategy. Most of these methods, such as game theory, that insure reaching the best solution for all market participants, require a lot of information about the other market players and the market. However, in the real market place only a little information, such as the spot price, is available for all participants. In this paper, a modified reinforcement learning based on temperature variation has been first proposed and then applied to determine the optimal strategy for a power supplier in the electricity market. A Pool-Co model has been considered here, and the simulation results are shown to be the same as those of standard game theory. Adaptation of the method in the presence of parameter variation has been verified as well. The main advantage of the proposed method is that no information about other participants is required. Furthermore, our investigation shows that even if all participants use this method, they will stay in Nash equilibrium.