Online beam orbit correction of MEBT in CiADS based on multi-agent reinforcement learning algorithm

Abstract

The proton linac accelerator has been the mostly appropriate equipment applied to radioactive waste disposal and cancer treatment. However, beam offset is a particularly significant problem during proton acceleration, which not only produces a halo and causes orbital distortion of the beam to make the beam injection inefficient. In exceptionally complex accelerator systems, accurate correction of beam offset is very difficult. The traditional method is based on manual experience and manual adjustment to achieve beam trajectory correction, the automation level is low and the adjustment speed is slow. Therefore, this paper proposes a new method of beam orbit correction based on Multi-Agent Reinforcement Learning. In order to verify the feasibility of the algorithm in beam orbit correction, this method takes the MEBT section of the virtual accelerator as the control object, and uses three agents to control the beam orbit of Medium Energy Beam Transport (MEBT) in sections. Each agent is designed based on the Reinforcement Learning of the Deep Deterministic Policy Gradient (DDPG) algorithm, the deterministic strategy is used to explore the large state and action space to realize discretization, and the training model convergence is successfully realized. The experimental results show that the final calibrated average beam offset of this method is about 0.39 mm, which achieves the expected experimental goal. It can control the beam orbit of MEBT, and has application value and competitiveness in the beam orbit correction of linac.