Geological adaptive intelligent control of earth pressure balance shield machine based on deep reinforcement learning

Abstract

Scientific and precise control of tunnelling parameters is of utmost importance during the construction of shield machines. Given the complexity of the working environment, manual operation is highly prone to causing safety accidents. Therefore, achieving intelligent control of the shield machine is crucial. Based on this, this paper proposes a geological adaptive intelligent control method of earth pressure balance shield machine using the Deep Deterministic Policy Gradient (DDPG) algorithm as the framework, with Actor-Critic as the basis. Firstly, DDPG agent is constructed to replace the screw conveyor control system as the main body of strategy implementation. Secondly, an environmental model is established by utilizing the mechanism model between the sealed cabin pressure and the screw conveyor speed. The real-time sealed cabin pressure, target pressure, and pressure error serve as the state space, while the screw conveyor speed is used as the action space. A combined reward function is set based on safety and accuracy. Finally, the Actor network interacts with the environment under the supervision of the reward function and Critic network. Successful training is achieved when the cumulative reward value is maximized, resulting in the output of optimal control strategy. In this paper, the method dynamically regulates the screw conveyor speed by interacting with the geological environment, to realize the precise control of the sealed cabin pressure and ensure the dynamic balance between sealed cabin pressure and excavation face pressure. The test results show that this method has a good control effect on the sealed cabin pressure under various geological conditions, and can complete 72 kinds of soil transition tasks. It has strong soil adaptability and can respond well to the dynamic changes of soil conditions. This approach enhances the intelligence of the shield machine, mitigating inaccuracies attributed to human operation, which provides a guarantee of safe shield machine operation, whilst exhibiting valuable engineering applications.