Reinforcement Learning Routing in Space-Air-Ground Integrated Networks

Abstract

6G-Wisdom Connected Everything has been put into research and development. It breaks through the limitations of topography and surface and extends to natural spaces such as space, air, land, and ocean. Whether it is national defense missions, emergency communications, or large-scale deterministic services, the integration of space, air and ground has significant advantages. This paper mainly focuses on the research of the space-air-ground integrated network. The new network architecture is bound to face new problems, among which routing is the basis of network interconnection. Currently, existing routing schemes are generally aimed at single-layer network routing, and cooperative routing between multiple layers has not made much progress. Due to the diversity of node types in the network, the limitations of different layers are also different. For example, satellite bandwidth resources are limited, time delay is large, hot air balloon energy is limited, and the ground is easily affected by the geographical environment. Aiming at the above problems, this paper proposes an integrated space-air-ground routing scheme based on reinforcement learning. Through the design of reinforcement learning and its reward function, we take the lowest delay as the goal, and the remaining energy and bandwidth utilization as the constraint conditions for routing. Theoretical analysis and simulation results show that reinforcement learning effectively solves the problem of limited satellite bandwidth resources and hot air balloons energy in the space-air-ground integrated network, and can successfully implement the routing function. Compared with Floyd routing, latency, packet loss rate and bandwidth utilization performance have all significantly improved.