Data Center Microgrid Energy Management Based on Model-Based Reinforcement Learning

Abstract

With the increasing of energy consumption and price, energy management is becoming increasingly important for data center with larger and larger scale. Microgrid can provide reliable, stable and continuous power services for data centers, and improve energy efficiency and reduce the operating cost. In this paper, we study the energy management problem for data center microgrid (DCMG) integrated with renewable energy. We introduce the energy storage device to mediate the intermittent nature of renewable energy. The energy management problem is formulated as a Markov Decision Process (MDP). Traditional methods greedy choice actions does not consider the impact of the actions on future cumulative operating cost. Model-free reinforcement learning is a candidate solution to this problem. However, model-free reinforcement learning methods converge slowly. Therefore, in this paper, we propose a model- based deep reinforcement learning algorithms to solve the energy management problem (Model-Based Reinforcement Learning Energy Management, MBRL-EM) of DCMG. We use long short-term memory (LSTM) model to design a system transition model, which predicts the future renewable energy production and energy demand. Then we obtain the optimal policy based on model predictive control to minimize the long-term operation cost. The simulation results show that MBRL-EM outperforms the baseline algorithms of model-based PPO and model-free PPO and reduces the average cost by up to 23% and 38% respectively.