Machine Learning for Load Balancing in Cloud Datacenters

Abstract

In the cloud datacenter, the resource utilization of different virtual machine (VM) and physical machine (PM) varies with time and it may lead to SLO violation and then degrade the application performance. In order to minimize the probability of SLO violation, load balancing is used to dynamically migrate VMs from overloaded PMs to underloaded PMs. Previous load balancing methods fail to achieve long term load balance. To address this problem, in this paper, we propose different load balancing methods and evaluate their performance on several metrics. We use the Fast Fourier Transform (FFT) method, an improved FFT method considering more frequencies in FFT and the long short term memory (LSTM) machine learning model to predict the resource utilization of VM and PM in the future. LSTM can always achieve the best prediction performance in the prediction. Taking advantage of the ML technique, we then propose a heuristic based method and a reinforcement learning (RL) based method relying on ML workload prediction to generate the VM migration plan in the datacenter. We conduct experiments in both trace-driven simulation (based on Google cluster trace, PlanetLab trace, Worldcup trace) and real implementation in terms of the SLO violation rate, the number of migrations and overhead. The experimental results show that the workload prediction helps reduce the SLO violation rate and/or the number of migrations, which improves the load balance performance in a datacenter. Also, the RL based VM migration method outperforms the heuristic based method in a heavily loaded system but does not show obvious advantages in a lightly loaded system.