Satellite-assisted edge computing management based on deep reinforcement learning in industrial internet of things

Abstract

The insufficient edge computing equipment in remote areas cannot meet explosively growing computing needs of industrial Internet of things devices, which undoubtedly leads to unaffordable overheads of the device-side energy and computation timeout. In response to this problem, we propose a collaborative computation offloading architecture based on the satellite-assisted edge computing (SAEC) deployed in low earth orbit ultra-dense satellite networks which hold great promise in the 6G communications benefiting from the low latency, high bandwidth, global coverage, etc. To make the non-differentiable computation offloading problem tractable, we propose an asynchronous advantage actor-critic based SAEC offloading (ASO) deep reinforcement learning (DRL) algorithm to optimize the integrated reward reflecting in latency and energy, and optimally train the action set determining the scale of participating SAEC servers and the distribution of their computational tasks. Numerous simulation results verify that our ASO algorithm can greatly accelerate the convergence and improve the integrated reward compared with contrast methods.