Deep reinforcement learning approach to optimize the driving performance of shield tunnelling machines

Abstract

This paper proposes a deep reinforcement learning (DRL)-based model as a valuable tool to improve the performance of the driving system (i.e. thrust force and cutterhead torque) of a shield tunnelling machine. The proposed model integrates deep-Q learning algorithm (DQL) and particle swarm optimization (PSO) based on an extreme learning machine (ELM). Specifically, the DQL–PSO model initialized the biases and weights in the ELM to achieve the optimal convergence rate and avoid instability. The DQL–PSO model evaluates the reward of action at each step and thus guides the particles to perform the appropriate action in real time. The DRL process data included shield operational parameters, geometry, and geological conditions. Field data collected from the Shenzhen railway tunnelling case study were used to validate the superiority and effectiveness of the presented DQL–PSO model. The algorithm was also evaluated using four numerical benchmark problems and compared with a theoretical model. The results revealed that the promising potential of DRL as a decision tool efficiently supports the formulation of target strategy and demonstrated its potential for engineering applications.