Adaptive service function chaining mappings in 5G using deep Q-learning

Abstract

With introduction of Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) technologies, mobile network operators are able to provide on-demand Service Function Chaining (SFC) to meet various needs from users. However, it is challenging to map multiple SFCs to substrate networks efficiently, particularly in a number of key scenarios of forthcoming 5G, where user requests have different priorities and various resource demands. To this end, we first formulate the mapping of multiple SFCs with priorities as a multi-step Linear Integer Programming (ILP) problem, of which the mapping strategy (i.e., the objective function) in each step is configurable to improve overall CPU and bandwidth resource utilization rates. Secondly, to solve the strategy selection problem in each step and alleviate the complexity of ILP, we propose an adaptive deep Q-learning based SFC mapping approach (ADAP), where an agent is learned to make decisions from two low-complexity heuristic SFC mapping algorithms. Finally, we conduct extensive simulations using multiple SFC requests with randomly generated CPU and bandwidth demands in a real-world substrate network topology. Related results demonstrate that compared with a single strategy or random selections of strategies under the ILP-based approach or the proposed heuristic algorithms, our ADAP approach can improve whole-system resource efficiency by scheduling this two simply designed heuristic algorithms properly after limited training episodes.