Self-attention mechanism for dynamic multi-step ROP prediction under continuous learning structure

Abstract

The evaluation and prediction of the rate of penetration have been long-term challenging in real-time drilling operations due to, for example, the complexity of influence parameters and uncertainties from the subsurface. In this paper, we propose a machine learning structure combining with a conceptual framework of continuous learning and a deep learning network with the self-attention mechanism to further improve the practical ROP prediction. The self-attention mechanism, also the cornerstone of the Transformer model in the recent impressive GPT3.5 and 4 (Generative Pretrained Transformer), exhibits an enhanced capability to capture the long-dependence relation within sequential data. Compared with the other commonly used recurrent neural networks, the proposed self-attention network model shows significant improvement in reliability and accuracy, especially when one tries to forward predict a long sequence of ROP. This newly presented model can predict the real-time long-term ROP value and can further improve its efficiency by adjusting the controllable drilling parameters. Moreover, the continuous learning structure allows for the extension of single well application to a field scale in a continuous manner for each network model. Field case studies demonstrate the effectiveness and stability of the newly proposed model, which reaches a higher prediction accuracy than 90% for three testing wells. Hence, the proposed structure and network model can help with predictive analysis for real-time drilling performance as an accurate and robust model.