Nonzero-Sum Pursuit-Evasion Game Control for Spacecraft Systems: A Q-Learning Method

Abstract

In this paper, the nonzero-sum pursuit-evasion (PE) game control problem is studied for a class of linear spacecraft control systems subject to the complete information case and incomplete information case. The incomplete information includes the cost functions and the control inputs. In practical confrontation situations, due to the incomplete information constraints, it is impossible for the pursuer and the evader to build up the exact opposite cost function. Hence, a nonzero-sum game framework is utilized to describe the PE game problem of the double-spacecraft system. First of all, under the complete information case, the nonzero-sum PE game control strategy is designed by solving the coupled Riccati recursions. Then, aiming at the incomplete information case, a control gain estimator is established, which lays the foundation for the control strategy design of the pursuit spacecraft. On the basis of the estimated control gain, the pursuit control strategy is solved by using the standard discrete-time Riccati recursion. In order to further get rid of the system information, a Q-learning-based control gain is designed for the pursuit spacecraft. Finally, a numerical example on the PE spacecraft system is provided to verify the effectiveness of the proposed PE game control strategies.