Multiagent Reinforcement-Learning-Aided Service Function Chain Deployment for Internet of Things

Abstract

Nowadays, the compelling applications of the Internet of Things (IoT) bring unexpected economic benefits to our daily lives. But at the same time, it also poses huge challenges to service providers. Diverse proprietary hardware (i.e., firewall and code conversion) have to be deployed in networks for meeting different applications’ requirements. Recently, network functions virtualization (NFV) is considered a promising technique. In the NFV-enabled architecture, network services can be implemented via a set of orderly virtual network functions (VNFs) on standardized compute nodes, which is termed service function chains (SFCs). However, with the explosion of IoT applications, embedding multiple SFCs in a shared NFV-enabled infrastructure becomes a challenging problem. Centralized schemes suffer from the scalability and private issue, while distributed schemes suffer from the nonconvergence problem. In this article, we propose a hybrid intelligent control architecture, which adopts the centralized training and distributed execution paradigm. A centralized critic is introduced to ease the training process of the distributed network nodes. Besides, considering the competitive behavior of users, we formulate the resource allocation problem as a multiuser competition game model. Based on this, we proposed a multiagent reinforcement learning-based SFCs deployment algorithm.