Reinforcement learning in local energy markets

Samrat Bose,Christof Weinhardt,Jan Eberbach,Enrique Kremers,Esther Marie Mengelkamp

doi:10.1186/s42162-021-00141-z

Abstract

Local energy markets (LEMs) are well suited to address the challenges of the European energy transition movement. They incite investments in renewable energy sources (RES), can improve the integration of RES into the energy system, and empower local communities. However, as electricity is a low involvement good, residential households have neither the expertise nor do they want to put in the time and effort to trade themselves on their own on short-term LEMs. Thus, machine learning algorithms are proposed to take over the bidding for households under realistic market information. We simulate a LEM on a 15 min merit-order market mechanism and deploy reinforcement learning as strategic learning for the agents. In a multi-agent simulation of 100 households including PV, micro-cogeneration, and demand shifting appliances, we show how participants in a LEM can achieve a self-sufficiency of up to 30% with trading and 41,4% with trading and demand response (DR) through an installation of only 5kWp PV panels in 45% of the households under affordable energy prices. A sensitivity analysis shows how the results differ according to the share of renewable generation and degree of demand flexibility.

Highlights

The recent development of emerging technologies in the power industry has led to a paradigm shift in the frameworks and business models of the electricity retail market of the future (Chen et al 2018)
Conclusion and further research The agent-based simulation model represented in this work demonstrates the application of reinforcement learning for intelligent agent strategies for peer-to-peer trading in a Local energy markets (LEMs)
We have demonstrated the convergence of various strategies with changing parameters of the modified Erev-Roth algorithm, giving the participants flexibility to choose between different strategies with different gains and penalties

Summary

Introduction

The recent development of emerging technologies in the power industry has led to a paradigm shift in the frameworks and business models of the electricity retail market of the future (Chen et al 2018). The clean energy package contains the adoption of two directives with relevance to LEMs, including the Internal Electricity Market Directive (EU) 2019/944 which introduced the “Citizen Energy Community” and the Renewable Energy Directive (EU) 2018/2001 which introduced the “Renewable Energy Community” (Caramizaru and Uihlein 2020). These regulations describe the role of consumer participation in achieving (2021) 4:7 the flexibility which is essential to accommodate the variable and distributed renewable electricity generation in the electricity system. LEMs are targeted towards establishing a balance between the local generation and consumption which may facilitate a reduction in energy transmission, network congestion and expedite proper inclusion of decentralised RES (Mengelkamp et al 2018a)

Methods

Results

Discussion

Conclusion