Markov decision process‐based routing algorithm in hybrid Satellites/UAVs disruption‐tolerant sensing networks

Abstract

Recently, a hybrid remote sensing network constituted by satellites in constellation and Unmanned Aerial Vehicles (UAVs) in formation attracts a lot of interests, benefiting from the flexible architecture and excellent rapid responsiveness. Considering frequently intermittent connectivity and limited resource onboard, Disruption-Tolerant Networking (DTN) develops a feasible solution for the remote sensing scenarios. However, the intrinsic motion models of multifarious nodes lead to deterministic or semi-deterministic contacts, which makes finding a reliable end-to-end routing path for timely data delivery difficult, with typical routing strategies such as Contact Graph Routing (CGR). To cope with such routing challenge in the hybrid network, a Probabilistic Contact Graph (PCG) is designed, taking the diverse node properties into consideration. In particular, a probability prediction model for semi-deterministic contacts between the UAV nodes is proposed, with a semi-Markov motion model for the UAV nodes. Besides, a Markov Decision Process based Routing (MDPR) algorithm is designed to search for a feasible data transmission path with a series of hybrid deterministic and semi-deterministic contacts. Through the numerical and experimental simulations with Interplanetary Overlay Network (ION), the proposed MDPR algorithm shows excellent routing performance concerning delivery delay and delivery ratio, compared with the typical CGR strategy.