Experimental and Simulative Evaluation of a Reinforcement Learning Based Cold Gas Thrust Chamber Pressure Controller

Abstract

At DLR neural networks, as potential future controller for rocket engines, are studied. A neural network-based chamber pressure controller for a simplified cold gas thruster is presented and analyzed in simulation and experiment. The goal of the controller is twofold: It can track a trajectory with different changes of setpoints and it allows to set and control a wide variety of steady state chamber pressures. The neural network gets feeding line pressure measurement data as input and calculates valve positions as output values. The training phase of the controller is done with a reinforcement learning algorithm in an EcosimPro/ESPSS simulation, that is validated with data from the corresponding experimental set up. To increase the robustness and to allow a transfer from the simulation directly to the test facility domain randomization is applied. The controller is evaluated in simulations and experiment. It was found that – in the range of physically possible operation points – the controller achieves a constantly high reward which corresponds to a low error and a good control performance. In the simulation the controller was able to adjust all required set points with a steady state error of less than 0.1bar while retaining a small overshoot and an optimal settling time. It is found that the controller is also able to regulate all desired set points in the real experiment. A reference trajectory with different steps, linear and sinus changes in target pressure is tested in simulation and experiment. The controller was in both cases able to successfully follow the given trajectory.