Intelligent Thermal Control Algorithm Based on Deep Deterministic Policy Gradient for Spacecraft

Abstract

In practical applications, it is difficult for the control parameters of the proportional integral derivative (PID) thermal controller to be self-tuning online. As the control object or environment changes, the control parameters are required to change accordingly. An intelligent thermal controller based on the deep deterministic policy gradient, called DRLTC, is proposed. Two types of reinforcement learning agents were designed in DRLTC, which can automatically adjust the control parameters of the thermal controllers and self-optimize online after training. Both theoretical and experimental results revealed that, when the control object was the main mirror support, the DRLTC achieved a control precision of 0.01°C. Additionally, the steady-state error was reduced by 40.2, 62.5, and 33.3% in the simulation and by 5.6, 80.6, and 85.7% in the experiment, compared with the reinforcement learning PID, neural network PID, and adaptive PID control based on fuzzy control, respectively. When the control object was changed to the main mirror installation, the DRLTC achieved a control precision of 0.02°C, and the steady-state error was reduced by 87.5, 91.7, and 90.9% in the simulation and by 80.2, 90.6, and 85.7% in the experiment, compared with the above-mentioned thermal control strategies, respectively. Therefore, the DRLTC has better universality, has stronger robustness, and saves more energy.