Heterogeneous Satellite Network Routing Algorithm Based on Reinforcement Learning and Mobile Agent

Abstract

In recent years, satellite communication systems have flourished. With the rapid increase of the number of users and the increase in QoS requirements, the single-layer satellite communication system has been far from meeting the needs. Heterogeneous satellite networks have become an inevitable development trend because they can achieve complementary advantages between constellations. The existing heterogeneous satellite network generally focus on heterogeneous networks between different orbit types, but the research on pure low-orbit satellite heterogeneous networks is rare. Therefore, this paper focuses on the research of low-orbit satellite heterogeneous networks. Among them, the routing algorithm is the basis of satellite interconnection, and its performance directly affects the performance of satellite network applications. In order to cope with common link congestion and satellite failure in heterogeneous satellite networks, this paper proposes a heterogeneous satellite network routing algorithm based on reinforcement learning and mobile agent. On the one hand, we use the reinforcement learning algorithm to adaptively reward the calculated optimal path, which is similar to ant pheromone accumulation. At the same time, the path causing congestion is appropriately punished. On the other hand, we use the inheritance of the mobile agent in orbit to solve the problem of inter-layer satellite failure. Theoretical analysis and simulation results show that reinforcement learning plays a key role in alleviating link congestion, and the greater the load, the better the effect. The mobile agent also effectively solves the problem of satellite failure. Compared with no mobile agent, the packet drop performance has been greatly improved.