Performance optimization of serverless edge computing function offloading based on deep reinforcement learning

Abstract

It is difficult for resource-constrained edge servers to simultaneously meet the performance requirements of all the latency-sensitive Internet of Things (IoT) applications in edge computing. Therefore, it is a significant challenge to efficiently generate a task offloading strategy. Recently, deep reinforcement learning (DRL)-based task offloading methods have been studied to ensure long-term performance optimization. However, there are challenges in existing DRL-based task offloading methods, such as insufficient sample diversity and high exploration cost. To optimize the performance of edge computing and facilitate the development and deployment of event-driven IoT applications, the serverless edge computing model has emerged. It combines serverless computing, also known as Function as a Service (FaaS), with edge computing and has been adopted in edge AI inference and prediction, stream processing, face recognition, etc. In this paper, an experience-sharing deep reinforcement learning-based distributed function offloading method called ES-DRL is proposed in the setting of a combined stateful and stateless execution model for serverless edge computing. ES-DRL adopts a distributed learning architecture, where each edge FaaS (EFaaS) obtains the current state of the local environment and inputs them to the local DRL agent, which outputs the function offloading strategy. Then, each EFaaS uploads the experience data interacting with the environment to a global shared replay buffer located in the cloud and randomly draws a batch of data from it to optimize the parameters of the local network. A population-guided policy search method is introduced to speed up the convergence of the DRL agent and avoid falling into the local optimum. The experimental results demonstrate that ES-DRL can reduce the average latency by up to approximately 17 percent compared to the existing DRL-based task offloading method.