Physics-informed deep reinforcement learning for enhancement on tunnel boring machine's advance speed and stability

Abstract

The traditional mode of Tunnel Boring Machine (TBM) operation is limited in their applicability and efficiency to meet the growing demand for underground spaces. Current methods to address this problem often lack physics interpretability and automatic adaptivity. In response to this issue, this paper describes an approach that integrates a physics-informed reinforcement learning (PIRL) algorithm into a TBM operation. The method combines a physics-informed machine learning (PIML) model and physics reward functions considering the working mechanism of earth pressure balance (EPB) TBMs, forming into a physics-informed Twin Delayed Deep Deterministic (pTD3) algorithm. The study reveals that the TBM performance can be improved by 69.3% using the pTD3 algorithm compared to manual operation. Integrating physics knowledge into reinforcement learning proves significantly effective in enhancing the TBM operations. The proposed method has the potential to revolutionize TBM operation and pave the way for more efficient, reliable, and automatic tunnel construction.