Two-Phase Neural Combinatorial Optimization with Reinforcement Learning for Agile Satellite Scheduling

Abstract

The agile Earth observing satellite (AEOS) scheduling problem consists of scheduling a set of image acquisitions and selecting the start time of each acquisition, satisfying the constraints and maximizing the gain. With the development of satellite technologies and higher requirements of users, the AEOS scheduling problem has to consider constraints and criteria that vary with the run-time on-orbit state, such as the transition time constraint and image quality criteria. These requirements increase the difficulty to model the problem. In this paper, a two-phase neural combinatorial optimization method with reinforcement learning is proposed for the AEOS scheduling problem. First, a neural combinatorial optimization with the reinforcement learning method is proposed to select a set of possible acquisitions and provide a permutation of them. Second, with the selected acquisition sequence, a reinforcement learning algorithm based on deep deterministic policy gradient is proposed to select the start time of each acquisition under the time constraints. The experimental results with artificial instances show that the proposed method is a viable candidate for the highly combinatorial AEOS scheduling problem; furthermore, it has the ability to cope with the run-time assessment, like the transition time calculation and the image quality evaluation.