Global optimization strategy of prosumer data center system operation based on multi-agent deep reinforcement learning

Abstract

The escalating issues of high energy consumption and carbon emissions in data centers (DCs) necessitate the optimization of system operations. However, early optimization strategies were overly simplistic and lacked automated updating and iterative capabilities. With the evolution of artificial intelligence (AI), researchers have applied deep reinforcement learning (DRL) algorithms to system operations. However, the optimization focus has been limited to the internal systems, lacking global optimization. In this paper, a global optimization control strategy based on the Dueling double-deep Q network (D3QN) and value decomposition network (VDN) algorithms is proposed to make the DCs system operate more closely with the upstream, midstream, and downstream. By adjusting battery charging/discharging capacity, computational workload, and waste heat utilization heating temperature global synergistic optimization is achieved. Compared with without optimization, renewable energy waste, operation cost, total electricity consumption, and grid electricity consumption are reduced by 18.37%, 9.78%, 4.01%, and 29.74%, respectively. Additionally, a detailed comparison between non-algorithmic optimization and algorithmic optimization is provided, offering valuable insights for substantial energy savings and emissions reduction in DCs. The results demonstrate the importance of fully exploring the interactive potential between upstream energy supply, midstream computational workload, and downstream waste heat recovery to achieve synergistic global optimization of "computing power", "thermal energy" and "electrical energy" for the sustainable and green development of DCs or other prosumer buildings.