DQN-based intelligent controller for multiple edge domains

Abstract

Advanced technologies like network function virtualization (NFV) and multi-access edge computing (MEC) have been used to build flexible, highly programmable, and autonomously manageable infrastructures close to the end-users, at the edge of the network. In this vein, the use of single-board computers (SBCs) in commodity clusters has gained attention to deploy virtual network functions (VNFs) due to their low cost, low energy consumption, and easy programmability. This paper deals with the problem of deploying VNFs in a multi-cluster system formed by this kind of node which is characterized by limited computational and battery capacities. Additionally, existing platforms to orchestrate and manage VNFs do not consider energy levels during their placement decisions, and therefore, they are not optimized for energy-constrained environments. In this regard, this study proposes an intelligent controller as a global allocation mechanism based on deep reinforcement learning (DRL), specifically on deep Q-network (DQN). The conceived mechanism optimizes energy consumption in SBCs by selecting the most suitable nodes across several clusters to deploy event requests in terms of nodes’ resources and events’ demands. A comparison with available allocation algorithms revealed that our solution required 28% fewer resource costs and reduced 35% the energy consumption in the clusters’ computing nodes while maintaining high levels of acceptance ratio.