Multi-agent reinforcement learning satellite guidance for triangulation of a moving object in a relative orbit frame

Abstract

Multi-agent systems are of ever-increasing importance in a contested space environment—use of multiple, cooperative satellites potentially increases positive mission outcomes on orbit, while autonomy becomes an ever-increasing requirement to increase reaction time to dynamic situations and lower the burden on space operators. This research explores multi-agent satellite swarm Guidance, Navigation, and Control (GNC) using deep reinforcement learning (DRL). DRL policies are trained to provide guidance inputs to agents in multi-agent swarm environments for completing complex, teamwork-focused objectives in geosynchronous orbit. An example scenario is explored for a group of satellite agents maneuvering to triangulate an object that is non-stationary in the relative orbit frame. Reward shaping is used to encourage learning guidance that positions swarm members to maximize triangulation accuracy, using angles-only observations for navigation relative to the target. Results show the policies successfully learn guidance through reward shaping to improve triangulation accuracy by a significant factor.