Learning-Based Computation Offloading for IoRT Through Ka/Q-Band Satellite–Terrestrial Integrated Networks

Abstract

In this article, we propose a multilayer Ka/Q-band satellite–terrestrial integrated network for the Internet of Remote Things (IoRT) to achieve a high transmission rate with communication robustness in dynamic network environments. Under this architecture, we investigate how to jointly manage the offloading path selection and resource allocation to offload computation-intensive and delay-sensitive tasks in the IoRT. Considering continuous low earth orbit (LEO) satellite movements and Markovian rainfall changes, the computation offloading problem is described as a Markov decision process (MDP) formulation with the objective of maximizing the number of offloaded tasks with satisfied delay requirements and minimizing the power consumption of the LEO satellites. A deep reinforcement learning (DRL) approach is leveraged to make optimal decisions by taking account of dynamic queues of IoRT devices, channel conditions that vary with rainfall intensities and satellite positions, and computing capabilities of ground stations. Extensive simulations are conducted to validate the effectiveness and superiority of our proposed scheme.