DQN-ALrM-Based Intelligent Handover Method for Satellite-Ground Integrated Network

Abstract

The satellite-ground integrated network (SGIN), consisting of satellites and ground base stations, is regarded as the trend to solve the problem of global coverage. However, highly dynamic scenarios pose great challenges to users’ mobility management, especially handover decisions. With the rapid development of machine learning, reinforcement learning (RL), such as Q-learning and deep Q-network (DQN), has been applied to handover decision making owing to being good at solving dynamic multi-attribute optimization problems. However, Q-learning can be a waste of storage for large state spaces. Besides, the existing algorithms for training a neural network in a deep RL framework have problems of slow convergence or unstable convergence performance. Therefore, we propose a handover method exploiting the DQN framework of adaptive learning rate with momentum (DQN-ALrM), which can not only improve decision accuracy, but also improve learning efficiency. The customized DQN framework can solve problems with a large-size state space, and the proposed ALrM method can adjust the learning rate at any time according to the training error situation. Comprehensive simulations are conducted to validate the superiority of the proposed method in convergence speed, handover rate, call failure rate and multi-metric quality of service (QoS) over the alternative solutions in the literature.