Smart Grid for Industry Using Multi-Agent Reinforcement Learning

Martin Roesch,Christian Linder,Andreas Rudolf,Andrea Hohmann,Roland Zimmermann,Gunther Reinhart

doi:10.3390/app10196900

Abstract

The growing share of renewable power generation leads to increasingly fluctuating and generally rising electricity prices. This is a challenge for industrial companies. However, electricity expenses can be reduced by adapting the energy demand of production processes to the volatile prices on the markets. This approach depicts the new paradigm of energy flexibility to reduce electricity costs. At the same time, using electricity self-generation further offers possibilities for decreasing energy costs. In addition, energy flexibility can be gradually increased by on-site power storage, e.g., stationary batteries. As a consequence, both the electricity demand of the manufacturing system and the supply side, including battery storage, self-generation, and the energy market, need to be controlled in a holistic manner, thus resulting in a smart grid solution for industrial sites. This coordination represents a complex optimization problem, which additionally is highly stochastic due to unforeseen events like machine breakdowns, changing prices, or changing energy availability. This paper presents an approach to controlling a complex system of production resources, battery storage, electricity self-supply, and short-term market trading using multi-agent reinforcement learning (MARL). The results of a case study demonstrate that the developed system can outperform the rule-based reactive control strategy (RCS) frequently used. Although the metaheuristic benchmark based on simulated annealing performs better, MARL enables faster reactions because of the significantly lower computation costs for its own execution.

Highlights

In order to mitigate the effects of anthropological climate change, efforts are being made worldwide to reduce greenhouse gas emissions (GHG) and increase the share of renewable energies
It became clear that the presented multi-agent reinforcement learning (MARL) system outperformed the reactive control approach (PCS) on production and energy costs
MARL thereby achieved lower average energy costs than SA, the production costs were up by 40%. This indicated that MARL is stuck in a local optimal, which focused on the energy cost reduction, there is still great potential on the production cost side

Summary

Introduction

In order to mitigate the effects of anthropological climate change, efforts are being made worldwide to reduce greenhouse gas emissions (GHG) and increase the share of renewable energies. One of the first industrial countries to do so, Germany announced in 2010 the plan to reduce GHG by 80% by 2050 along with plans to generate 80% of total electricity using renewable sources [1]. In 2019, 42% of the total consumed electricity was generated by renewable sources. This trend has entailed some challenging side effects. Electricity prices have been increasingly fluctuating, especially in the short term (see Figure 1a) since power generation using renewable sources, e.g., wind or solar, is subject to sudden changes in weather and is, neither controllable nor predictable

Objectives

Methods

Results

Conclusion