A Target-Aware Well Path Control Method Based on Transfer Reinforcement Learning

Abstract

Summary In the actual construction process, well path control is a challenging task mainly due to the inevitable well deflection caused by geological factors, drilling tools as well as borehole enlargement. Most conventional well path control methods focus on elaborate mechanism model construction. The methods are typically constructed on the basis of certain constraints or assumptions, which reflect their limited ability to accurately capture the actual drilling process, low level of intelligence, poor anti-interference performance, and weak adaptive capacity. To address these challenges, this paper proposes a target-aware well path control method that integrates reinforcement learning and transfer learning. The proposed method employs a deep deterministic policy gradient model based on the prioritized experience replay mechanism and leverages transfer learning to accelerate model learning. This enables the construction of a target-aware well path adaptive control system with strong anti-interference capability. The proposed target-aware control method of well path based on reinforcement learning and transfer learning can accurately track the preset trajectory in diverse geological environments, reach the target area with high precision, and make reasonable trajectory optimization decisions with measurement while drilling (MWD) even when the target trajectory does not match the actual distribution of the reservoir. This approach exhibits excellent anti-interference and adaptive abilities.