Control of quasi-equilibrium state of annular flow through reinforcement learning

Abstract

Stability control of the convection flow field has always been a focal issue. The annular flow discussed in this work is a typical research model of microgravity fluid physics, which is extracted from the industrial crystal growth by the Czochralski method. It is believed that the instability of thermal convection is the key factor affecting the quality of crystal growth. Combining the reinforcement learning algorithm with the neural network, this paper proposes a control policy that makes forced convection compete with thermocapillary convection by changing the dynamic boundary conditions of the system. This control policy is successfully applied to the control of the quasi-equilibrium state of annular flow, and the global stability of the flow field is well maintained. It first experimentally makes the annular flow field under low and medium Ma numbers achieve a quasi-equilibrium state, which is different from that before the onset of flow oscillations. Then, a simulation environment is created to imitate the experimental conditions. After training in the simulation environment, with the self-optimized algorithm, the machine learning approach can successfully maintain the simulation environment in a quasi-equilibrium state for a long period of time. Finally, the learning method is validated in the experimental environment, and a quasi-equilibrium state control policy is completely optimized by using the same optimization policy and similar neural network structure. This work demonstrates that the model can understand the physical environment and the author's control objectives through reinforcement learning. It is an important application of reinforcement learning in the real world and a clear demonstration of the research value of microgravity fluid physics.