Agent-Based Modeling of Retail Electrical Energy Markets With Demand Response

Abstract

In this paper, we study the behavior of a day-ahead (DA) retail electrical energy market with price-based demand response from air conditioning (AC) loads through a hierarchical multiagent framework, employing a machine learning approach. At the top level of the hierarchy, a retailer agent buys energy from the DA wholesale market and sells it to the consumers. The goal of the retailer agent is to maximize its profit by setting the optimal retail prices, considering the response of the price-sensitive loads. Upon receiving the retail prices, at the lower level of the hierarchy, the AC agents employ a ${Q}$ -learning algorithm to optimize their consumption patterns through modifying the temperature set-points of the devices, considering both consumption costs and users’ comfort preferences. Since the retailer agent does not have direct access to the AC loads’ underlying dynamics and decision process (i.e., incomplete information) the data privacy of the consumers becomes a source of uncertainty in the retailer’s decision model. The retailer relies on techniques from the field of machine learning to develop a reliable model of the aggregate behavior of the price-sensitive loads to reduce the uncertainty of the decision-making process. Hence, a multiagent framework based on machine learning enables us to address issues such as interoperability and decision-making under incomplete information in a system that maintains the data privacy of the consumers. We will show that using the proposed model, all the agents are able to optimize their behavior simultaneously. Simulation results show that the proposed approach leads to a reduction in overall power consumption cost as the system converges to its equilibrium. This also coincides with maximization in the retailer’s profit. We will also show that the same decision architecture can be used to reduce peak load to defer/avoid distribution system upgrades under high penetration of photo-voltaic power in the distribution feeder.