Adaptive fault-tolerant control for spacecraft: A dynamic Stackelberg game approach with advantage actor-critic reinforcement learning

Abstract

This study introduces an innovative approach that harnesses the Stackelberg differential game framework in combination with advantage actor-critic (A2C) reinforcement learning (RL) to attain adaptive and dynamic control, in the presence of complex faults within spacecraft systems. By reconfiguring the problem of fault tolerance into a strategic interaction involving two pivotal players, specifically, the leader (fault-tolerant control, FTC) and the follower (fault detection observer, FDO), our approach facilitates the dynamic adaptation of their control strategies in response to the system's state, effectively providing a solution for addressing faults within spacecraft systems. This approach overcomes the challenges of cross-optimal indexes commonly found in traditional FTC, resulting in improved efficiency and stability of RL. It harbors substantial potential for applications requiring robust and adaptive FTC strategies. Simulation validation, both the leader and follower can successfully control the spacecraft's velocity before and after a fault occurs, ultimately achieving optimal fault estimation (FE) and FTC objectives.