DQN based user association control in hierarchical mobile edge computing systems for mobile IoT services

Abstract

This paper proposes a new user association control method to maximize system utility in terms of system throughput, edge throughput, and handover cost in mmWave-based HMEC (mm-HMEC) systems. We first formulated a non-convex mathematical programming model and then proposed centralized and distributed deep Q-network (DQN)-based range expansion bias control algorithms. We discovered that the proposed schemes provided polynomial communication overhead and computation complexity. Through simulations, we evaluated the proposed centralized and distributed schemes under various user equipment (UE) mobility models: random waypoint mobility model (RWM), random direction mobility model (RDM), and Manhattan mobility model (MAN). We first confirmed that the proposed distributed control achieves almost the same performance as the proposed centralized control. Second, with 10 slave-MEC servers, the proposed distributed control provides 212.17%, 62.07%, 46.36%, and 48.63% enhanced utility; 33.68%, 18.19%, 9.28%, and 12.66% enhanced average cell throughput; 14.32%, 38.32%, 63.87%, and 24.57% enhanced edge throughput; and 39.09%, 25.42%, 29.32%, and 25.95% reduced handover cost compared to random cell range expansion (CRE), standard CRE, dynamic CRE, and Q-learning-based CRE, respectively.