DQN-based energy-efficient routing algorithm in software-defined data centers

Zan Yao,Ying Wang,Xuesong Qiu

doi:10.1177/1550147720935775

Abstract

With the rapid development of data centers in smart cities, how to reduce energy consumption and how to raise economic benefits and network performance are becoming an important research subject. In particular, data center networks do not always run at full load, which leads to significant energy consumption. In this article, we focus on the energy-efficient routing problem in software-defined network–based data center networks. For the scenario of in-band control mode of software-defined data centers, we formulate the dual optimal objective of energy-saving and the load balancing between controllers. In order to cope with a large solution space, we design the deep Q-network-based energy-efficient routing algorithm to find the energy-efficient data paths for traffic flow and control paths for switches. The simulation result reveals that the deep Q-network-based energy-efficient routing algorithm only trains part of the states and gets a good energy-saving effect and load balancing in control plane. Compared with the solver and the CERA heuristic algorithm, energy-saving effect of the deep Q-network-based energy-efficient routing algorithm is almost the same as the heuristic algorithm; however, its calculation time is reduced a lot, especially in a large number of flow scenarios; and it is more flexible to design and resolve the multi-objective optimization problem.

Highlights

With the rapid development of modern information technologies such as cloud computing, big data, Internet of things, and edge computing, more and more studies about smart city have emerged
It is needed to design an algorithm to choose the appropriate data path for every flow and control path for every switch to obtain the dual goals of energy savings and load balancing between controllers
The load balancing performance of the deep Q-network (DQN)-EER algorithm considering load balancing between controllers is better

Summary

Introduction

With the rapid development of modern information technologies such as cloud computing, big data, Internet of things, and edge computing, more and more studies about smart city have emerged. It is needed to design an algorithm to choose the appropriate data path for every flow and control path for every switch to obtain the dual goals of energy savings and load balancing between controllers. Wu17 proposes an energy-efficient routing strategy, which uses a heuristic algorithm to coordinative calculate the paths for the switches and data traffic. For the software-defined data center network in the in-band control mode, we mainly study the problem of energy-saving routing. 2. We model the optimization problem as a Markov decision process (MDP) and propose a deep Q-network-based energy-efficient routing (DQN-EER) algorithm. We model the optimization problem as a Markov decision process (MDP) and propose a deep Q-network-based energy-efficient routing (DQN-EER) algorithm It only trains part of the states to collaboratively obtain the optimal data path and control path, and simultaneously processes the traffic in batches instead of sorting sequentially. The dualobjective optimization problem is presented in section ‘‘Model of network system,’’ and we propose the DQN-EER algorithm to solve the problem in section ‘‘DQN-based energy-efficient routing algorithm.’’ The simulation results are presented to verify the feasibility and effectiveness of the proposed approach in section ‘‘Simulation and results.’’ the conclusion is given in section ‘‘Conclusion and future work.’’

Related work

Conclusion and future work