Online two-timescale service placement for time-sensitive applications in MEC-assisted network: A TMAGRL approach

Abstract

Mobile edge computing (MEC) integrated with the Network Functions Virtualization (NFV) technique has been regarded as a promising solution for flexible services provision and user service experience improvement. However, existing service placement in such systems still faces the challenge of satisfying computing tasks with strict latency requirements, especially when massive mobile users roam around different coverage areas of edge servers. For this purpose, we first adopt a novel service placement framework that combines proactive replicas pre-deployment and reactive service migration. Based on this, we investigate the dynamic placement problem of multiple types of services achieved by the various virtualized network functions (VNFs) to minimize long-term redeployment costs in MEC-assisted systems, subject to the completion deadline of tasks and limited computing resources of edge servers. Considering that the update timescale of VNF replicas pre-deployment is different, we design a novel two-timescale multi-agent graph convolutional network-based reinforcement learning algorithm (TMAGRL) by invoking a long-timescale training layer for proactive VNF replicas placement and a short-timescale training layer for reactive VNF migration. Extensive numerical results reveal that TMAGRL, based on the designed hybrid framework, can learn a VNF placement strategy to adapt to the dynamics of the system without any prior information. Moreover, we verify its superior performance in terms of average service response latency and overall redeployment cost by comparing it with baselines.