Distributed Data Center Cooling control based on Multi-Agent Reinforcement Learning

Abstract

The data center cooling system is a complex-structured system integrated with a large number of components. Traditional centralized control methods are hard to apply in production data centers due to scalability issues. This paper studies the distributed cooling control problem where the data center is cooled by multiple Computer Room Air Conditioners (CRACs) to minimize the energy consumption. To characterize the interactions of CRACs, we formulate the distributed data center cooling control problem as a stochastic game, where each CRAC acts as an agent to learn the optimal local cooling policy. We propose a Multi-Agent Reinforcement Learning (MARL) framework based on the Counterfactual Multi-Agent (COMA). Specifically, we policies based on the agents' observations of the global state. After the implementation of these policies, they will receive their own rewards, then evaluate the quality of the policies. Simulation results show that compared with the Independent Q-Learning (IQL) method, this algorithm reduces the power consumption of CRACs by 6.2%.