Towards Intelligent Provisioning of Virtualized Network Functions in Cloud of Things: A Deep Reinforcement Learning Based Approach

Abstract

Cloud of Things (CoT) is an integration of Internet of Things (IoT) and cloud computing, where Network Function Virtualization (NFV) can dynamically provide Virtualized Network Functions (VNFs) for IoT devices based on service-specific requirements. The provisioning of VNFs in CoT is formulated as an online decision-making problem, but widely-used methods mostly focus on modeling the environment simply to obtain the optimal solution. Valuable historical experience on provisioning is ignored and Quality of Service (QoS) requirements for different types of CoT services are also not considered, which leads to inefficient and coarse-gained provisioning. In this paper, an intelligent provisioning framework of VNFs is proposed for adaptive CoT resource scheduling according to traffic identification of heterogeneous network services. The framework leverages a Deep Reinforcement Learning (DRL) based model to make decisions based on the complexity of network environments and traffic variances. In this model, a policy gradient DRL algorithm, namely, Policy Optimization using Kronecker-Factored Trust Region (POKTR) is adopted to obtain the stable performance by a novel surrogate objective function. Experimental results verify that our framework improves the QoS in CoT by real-time VNFs provisioning. The DRL-based model with POKTR algorithm reduces network congestion and achieves higher throughput than other DRL algorithms.