Dynamie management of data center resources using reinforcement learning

Abstract

For effective management of cloud data center resources there is a need to develop and implement simple and robust methods that take into account the data center power consumption, user Service Level Agreement compliance, and different conditions of resource utilization. In this paper, the authors analyze the possibility of application of the reinforcement learning method to cloud data center resource management. Due to the intensive changes of the workloads and different conditions of resource utilization the data center resource management problem should be solved dynamically in an online manner. To address such problem, the authors propose the method of dynamic virtual machine placement based on the reinforcement learning. The proposed method takes into account the power consumption and the number of SLA violations while producing control impacts. Besides, the method considers the changes in the resource utilizations to make a decision on whether to switch underloaded physical servers to the sleep mode in order to reduce the power consumption. The proposed reinforcement learning agent is based on the Q-learning approach which allows to determine the optimal policy for managing the physical servers without creating an environment model and preliminary information about the workload.