DDQN-based online computation offloading and application caching for dynamic edge computing service management

Abstract

Multi-access Edge Computing (MEC) reduces task service latency and energy consumption by offloading computing tasks to MEC servers. However, constrained by the limited bandwidth and computing resources, MEC servers often cannot parallelly process all computing tasks. Simultaneously, the high dynamism of service popularity necessitates MEC servers to dynamically update cached applications, under ensuring compliance with storage resource constraints and the system cache updating cost budget for service providers. In response to the above two issues, this paper firstly formulates computation offloading and application caching as a dual-timescale decision optimization problem, aiming to minimize the average service latency for users by obtaining optimal offloading decision, cache decision, transmission bandwidth, and computing resource. Then, we propose a Deep Reinforcement Learning (DRL)-based two-stage online computation offloading and application caching (DTSO2C) algorithm, effectively stabilizing application cache update costs and enhancing Quality of Service (QoS) for users. Furthermore, we utilize convex optimization algorithms to derive the optimal communication bandwidth and computing resource allocation strategy, further reducing the average service latency for users. Simulation results demonstrate that the DTSO2C algorithm outperforms the compared algorithms, achieving an average reduction in service latency of 66.2%, with an average cache update cost of only 0.15 USD per time frame.