Autonomous Small Body Science Operations Using Reinforcement Learning

Abstract

This work studies the feasibility of using reinforcement learning for small body science operations subject to resource constraints. Two mission scenarios are considered. In the first scenario, a spacecraft autonomously maneuvers between waypoints about a small body while performing science activities, such as mapping and imaging, and periodically downlinking data and managing on-board resources like battery charge, data buffer storage, and fuel usage. In the second scenario, the spacecraft periodically performs navigation updates to improve its state estimate, ensuring that the collected science is within the specified requirements. A Markov decision process formulation of the mission scenarios is formulated, and reinforcement learning is applied to solve the problem. A range of noisy observation types are tested, demonstrating that a fully observable formulation of the problem trained on direct observations of the state is robust to noisy measurements or a filtered state estimate. A decision-making agent is then trained to manage the state estimate by choosing when to take measurements, demonstrating that near-equivalent policies, in comparison to nominal problem formulation, can be trained with an optional navigation update. Finally, a demonstration is performed in which a ground station outage is simulated. The decision-making agent is shown to be robust to this outage, rapidly adjusting its plan to continue nominal operations.