A Q-learning-based approach for virtual network embedding in data center

Abstract

Virtual network embedding (VNE) refers to allocating reasonable substrate network resources for virtual network (VN) requests that include computing resources and network resources, so as to obtain optimal income from leasing virtual resources. Such a way of providing virtual resources is the key technology of cloud computing and can greatly save the operating cost of enterprises and provide flexibility of application deployment. However, the existing VNE algorithms are mostly oriented to traditional stochastic network topologies. Due to the high connectivity and server density of data centers and the complexity of the user’s resource requirements, the traditional VNE algorithms suffer from low resource utilization rate and revenues in the VNE on the data centers. Different from the existing algorithms which are often based on heuristic algorithms, this paper proposes a VNE algorithm for data center topology based on the Q-learning algorithm which is a typical reinforcement learning method. The algorithm an agent for each VN designs a reward function related to the effect of virtual link embedding, which is used to update the Q-matrix through unsupervised learning process. Then, the agent can find the optimal embedding strategy based on the Q-table from each learning. Simulation results demonstrate that the proposed algorithm can improve the resource utilization ratio and obtain a better revenue/cost ratio of the substrate network compared with the traditional heuristic algorithms.