Active Gamma-Ray Log Pattern Localization With Distributionally Robust Reinforcement Learning

Abstract

Accurately localizing 1D signal patterns, such as Gamma-ray well-log depth matching, is crucial in the oilfield service industry as it directly affects the quality of oil and gas exploration. However, traditional methods such as well-log curve analysis and pattern hand-picking matching are labor-intensive and heavily rely on human expertise, leading to inconsistent results. Although attempts have been made to automate this process, challenges such as low computational performance, non-robustness, and non-generalization remain unsolved. To address these challenges, we have developed a data-driven AI system that learns an active signal pattern localization strategy inspired by human attention. Our artificial intelligence system uses an offline reinforcement learning (RL) framework as its central component, which solves a highly abstracted Markov decision process problem via offline training on human-labeled historical data. The RL agent uses top-down reasoning to determine the location of target signal fragments by deforming a bounding window using simple transformation actions. To overcome distribution shifts between logged data and real and ensure generalization, we propose a discrete distributionally robust soft actor-critic RL framework (DRSAC-Discrete) to solve the Markov decision process problem under uncertainty. By exploring unfamiliar environments in a restrictive manner, the DRSAC-Discrete algorithm provides a safe solution that can be used when data is limited during the early stage of this industrial application. We evaluated the reinforcement learning-based localization system on augmented field Gamma-ray well-log datasets, and the results showed promising localization capability. Furthermore, the DRSAC-Discrete algorithm demonstrated relatively robust performance guarantees when facing data shortage.