Optimal VNF Placement via Deep Reinforcement Learning in SDN/NFV-Enabled Networks

Abstract

The emerging paradigm - Software-Defined Networking (SDN) and Network Function Virtualization (NFV) - makes it feasible and scalable to run Virtual Network Functions (VNFs) in commercial-off-the-shelf devices, which provides a variety of network services with reduced cost. Benefitting from centralized network management, lots of information about network devices, traffic and resources can be collected in SDN/NFV-enabled networks. Using powerful machine learning tools, algorithms can be designed in a customized way according to the collected information to efficiently optimize network performance. In this paper, we study the VNF placement problem in SDN/NFV-enabled networks, which is naturally formulated as a Binary Integer Programming (BIP) problem. Using deep reinforcement learning, we propose a Double Deep Q Network-based VNF Placement Algorithm (DDQN-VNFPA). Specifically, DDQN determines the optimal solution from a prohibitively large solution space and DDQN-VNFPA then places/releases VNF Instances (VNFIs) following a threshold-based policy. We evaluate DDQN-VNFPA with trace-driven simulations on a real-world network topology. Evaluation results show that DDQN-VNFPA can get improved network performance in terms of the reject number and reject ratio of Service Function Chain Requests (SFCRs), throughput, end-to-end delay, VNFI running time and load balancing compared with the algorithms in existing literatures.