Abstract
In datacenter networks, bandwidth-demanding elephant flows without deadline and delay-sensitive mice flows with strict deadline coexist. They compete with each other for limited network resources, and the effective scheduling of such mix-flows is extremely challenging. We propose a deep reinforcement learning with private link approach (DRL-PLink), which combines the software-defined network and deep reinforcement learning (DRL) to schedule mix-flows. DRL-PLink divides the link bandwidth and establishes some corresponding private-links for different types of flows to isolate them such that the competition among different types of flows can decrease accordingly. DRL is used to adaptively and intelligently allocate bandwidth resources for these private-links. Furthermore, to improve the scheduling policy, DRL-PLink introduces the novel clipped double Q-learning, exploration with noise, and prioritized experience replay technology for DDPG to address function approximation error, to induce lager and more randomness for exploration, as well as more effective and efficient experience replay in DRL respectively. The experiment results under actual datacenter network workloads (including Web search and data mining workload) indicate that DRL-PLink can effectively schedule mix-flows at a small system overhead. Compared with ECMP, pFabric, and Karuna, the average flow completion time of DRL-PLink decreased by 77.79%, 65.61%, and 23.34% respectively, when the deadline meet rate is increased by 16.27%, 0.02%, and 0.836% respectively. Additionally, DRL-PLink can also well achieve load balance between paths.
Highlights
A data center network (DCN) comprises various applications
Some fine flow scheduling schemes aim to purely meet DMR or flow completion time (FCT). (1) For flows with deadline, EDF-based (Earliest Deadline First) schemes aim to maximize the DMR
We evaluate the performance of deep reinforcement learning (DRL)-PLink in a large-scale software-defined data-center networks (SD-DCN) built by Ryu+Mininet where Open vSwitch (OVS) is used as a switch
Summary
A data center network (DCN) comprises various applications. Studies [1] have shown that the traffic distribution of a DCN is extremely uneven and can be classified into elephant flows (EFs, which are large and long-lived) and mice flows (MFs, which are small and short-lived). Dynamic heuristic flow scheduling schemes can control the network more flexibly; some of them (e.g., PIAS [7], [18], etc.) may cause traffic imbalance or congestion. Their performances depend on some parameters that are manually set based on the collection of long-term traffic statistics. Combining SDN and DRL, we propose the DRL with private link approach (DRL-PLink) — an automatic traffic optimization scheme to solve the mix-flow scheduling problem. The main contributions of this paper are as follows: (a) We propose a mix-flow scheduling scheme based on DRL, by establishing a private-link for different types of flows in DCN.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have