A self-stabilizing and auto-provisioning orchestration for microservices in edge-cloud continuum

Abstract

Modern user-facing services are progressively evolving from large monolithic applications to complex graphs of loosely-coupled microservices. While this shift provides opportunities to offload some microservices of a user-facing service to edge devices that are close to the end users, it also complicates the application deployment and resource provisioning in the edge-cloud continuum, due to complex relationships across microservices and unstable public networks. To reduce resource wastage and improve user experience, this paper presents SMO, a self-managed orchestration system for microservices. SMO leverages a self-stabilizing placement mechanism to optimally deploy microservices through perceiving both interferences and communication overheads. During runtime, it further tailors a multi-agent deep deterministic policy gradient (MADDPG)-based model in combination with attention and prioritized replay, which automatically provisions appropriate resources for each microservice subject to the differentiated tail latency Service Level Objectives (SLOs) of multiple user workloads. Experimental results demonstrate that SMO saves up to 39% of CPU cores and 47% of memory footprint while providing guarantees for heterogeneous tail latency SLOs.