Q-adjusted annealing for Q-learning of bid selection in market-based multisource power systems

Abstract

The electric power industry is confronted with a major restructuring in which operation scheduling is decided based on a competitive market. In this new arrangement, bidding strategy becomes an important issue. Participants of the deregulated energy marketplace may be able to compete better by choosing a suitable bidding strategy for trading electricity. In such a deregulated market, coalition formation with other participants may change the diffusion of profit. Therefore the problem changes to the selection of the best participant for coalition and joint bidding strategies. Different classic methods for decision making in the uncertain environment of the market can be applied to select a suitable strategy. Most of these methods, such as game theory that insures reaching the optimal solution for all market participants, require a lot of information about other market players and the market. However, in a real marketplace only a little information such as the spot price is available for all participants. A modified reinforcement-learning approach based on Q-adjusted annealing has been applied to determine the optimal strategy for a power supplier in an electricity market with multiple sources. A modified IEEE 30-bus system has been considered and the simulation results are shown to be the same as with standard game theory. The main advantage of the proposed method is that no information about other participants is required. Investigation shows that if all participants use this method they will stay in Nash equilibrium.