Predictive Analytics Based Knowledge-Defined Orchestration in a Hybrid Optical/Electrical Datacenter Network Testbed

Abstract

For datacenter networks (DCNs), it is always important to have an effective network orchestration scheme that can coordinate the usages of IT and bandwidth resources timely. In this paper, we consider the hybrid optical/electrical DCNs (HOE-DCNs) and propose a knowledge-defined network orchestration (KD-NO) system for them. The KD-NO system follows the predictive analytics in human behaviors, which includes forecasting based on memory and decision making based on knowledge. To explain the design of our KD-NO system, we first discuss how to fetch low-level knowledge from the telemetry data about the resource utilization in an HOE-DCN. Then, we describe how to optimize the HOE-DCN's configuration for network orchestration. Specifically, we design an online scheme based on deep reinforcement learning, and make sure that it can extract high-level knowledge from the low-level input and come up with optimal HOE-DCN configurations on-the-fly. We prototype the proposed KD-NO system and demonstrate it in an HOE-DCN testbed. The experiments run Hadoop applications in the testbed and show that our KD-NO system can make timely and correct decisions in different experimental schemes by leveraging the two-level knowledge, maintain a high matching degree between the HOE-DCN's configuration and the applications running in it, and thus, effectively reduce the job completion time.