Deep Q-Learning for Decentralized Multi-Agent Inspection of a Tumbling Target

Abstract

As the number of on-orbit satellites increases, the ability to repair or de-orbit them is becoming increasingly important. The implicitly required task of on-orbit inspection is challenging due to coordination of multiple observer satellites, a highly nonlinear environment, a potentially unknown or unpredictable target, and time delays associated with ground-based control. There is a critical need for autonomous, robust, decentralized solutions. To achieve this, we consider a hierarchical, learned approach for the decentralized planning of multi-agent inspection of a tumbling target. Our solution consists of two components: a viewpoint or high-level planner trained using deep reinforcement learning, and a low-level planner that will handle the point-to-point maneuvering of the spacecraft. Operating under limited information, our trained multi-agent high-level policies successfully contextualize information within the global hierarchical environment and are correspondingly able to inspect over 90% of nonconvex tumbling targets, even in the absence of additional agent attitude control.