UrbanEnQoSPlace: A Deep Reinforcement Learning Model for Service Placement of Real-Time Smart City IoT Applications

Abstract

Listen

Translate article

Multi-access Edge Computing (MEC) enables IoT applications to place their services in the edge servers of mobile networks, balancing Quality-of-Service (QoS) and energy-efficiency. Previous works consider compute requirements, while the IoT and latency/bandwidth per-flow communicate requirements are largely ignored. Moreover, the Smart City domain presents unique challenges - modeling the Urban Smart Things (USTs - urban IoT clients), their connectivity with MEC network, diverse resource requirements (compute, communicate, and IoT) of application services, modeling the federation of multiple MEC providers in a city, which we consider in this paper. To address these research gaps, we propose: i) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">UrbanEnQoSMDP</i> - formulation for energy and QoS (latency) optimized service placement for a set of applications in the ‘Urban IoT-Federated MEC-Cloud’ architecture to satisfy applications’ compute, per-flow communicate, and IoT requirements; ii) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">‘’<inline-formula><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula>-greedy with mask”</i> policy for apriori satisfaction of IoT requirements by shortlisting suitable USTs; iii) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">UrbanEnQoSPlace</i> - proposed multi-action Deep Reinforcement Learning (DRL) model, designed from Dueling Deep-Q Network, that uses the proposed policy to solve the UrbanEnQoSMDP for simultaneously placing all services of an application. Extensive simulation results illustrate efficacy and scalability of proposed model against state-of-the-art DRL algorithms (better convergence, higher rewards, lesser runtime; proposed policy w.r.t fewer violations).