Age-Oriented Access Control in GEO/LEO Heterogeneous Network for Marine IoRT: A Deep Reinforcement Learning Approach

Abstract

With the growing interest in the smart ocean, the satellite-based marine Internet of Remote Things (IoRT) network has been regarded as a promising architecture for sensory data collection and transmission in infrastructure-limited offshore areas. In this article, we investigate the access control problem in the context of GEO/LEO heterogeneous IoRT networks, where multiple gateways are deployed to collect data generated by IoRT devices and then forward them to the terrestrial data center via satellite links. However, most existing access control strategies shed light on the traditional network performance (i.e., transmission delay and communication throughput) in single-layer satellite networks (i.e., low-Earth orbit (LEO) layer or geosynchronous orbit (GEO) layer), whereas the interplay between LEO and GEO layers and the freshness of information are rarely considered. To this end, we first formulate an age-oriented access control problem to minimize the long-term peak Age of Information (AoI) and transform it into a model-free Markov decision process (MDP). Then, a Deep-Double-Dueling- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -Learning (D3QN) policy is trained offline and can be deployed online to make decisions according to dynamic data arrivals and time-varying channels. Simulation results show that the proposed strategy significantly outperforms the state-of-the-art ones in terms of the long-term AoI performance. Furthermore, our strategy could make cooperative decisions for gateways and obtain a proper tradeoff between satellites on different layers.