Multi-Label Classification of Daily Drill Reports (DDR) Utilizing Large Language Models (LLMs)

Abstract

Abstract In the oil and gas sector, precise identification and classification of drilling issues are crucial for safety and productivity. Analyzing historical drilling data enables insights into potential problems in similar wells drilling. From existing Electronic Drilling Management (EDM) tool, a dataset comprising nearly one hundred thousand text descriptions was compiled through keyword-based text mining alongside anti-keywords. Following the initial labeling process, the data was submitted to the business for label confirmation. Initially, basic machine learning models such as Long short-term memory (LSTM) were used. However, these had limitations related to spelling errors, acronyms, and miscellaneous symbols. Subsequently, the decision was made to transition to Large Language Models (LLMs). To address it, this paper proposes a novel approach using LLMs for multi-label drilling issue classification. Experiments were conducted with various LLMs from different providers and parameter sizes, leveraging GPUs. Challenges arose due to imbalanced data. To enhance the robustness of this method, proper data augmentation was carried out during LLM training to ensure broad coverage of drilling issues. With over 20 distinct classes, drilling descriptions often contain up to 5-6 classes, making achieving singular accuracy challenging. Thus, various accuracy metrics were experimented with to ensure robust multi-label classification (MLC) accuracy that addresses both false positives and false negatives. Regarding overall accuracy, model achieved a level surpassing 90%. Accuracy at the individual class level was evaluated, initially yielding zero accuracy for some classes due to limited occurrences. However, with data augmentation, both recall and precision accuracies improved significantly. Despite the recent surge in the popularity of LLMs, there remains a scarcity of projects effectively utilizing LLMs and Daily Drill Reports (DDR) to correctly identify issues in the well drilling process. This model utilizes state-of-the-art technology, employing suitable Transformer-based LLMs. This solution is built with open-source, on-premises models to address data privacy concerns. This novel approach holds promise to outperform historically provided solutions based on keyword extraction techniques, offering significantly better results. This method can be applied to both current and future drilling operations, leveraging the present condition of wells.