This study thoroughly investigates the complex origins of abnormal formation pressure in offshore oil and gas wells, taking the Rio del Rey Basin in Cameroon as a case study. Renowned for its abundant oil and gas resources, the area faces unique challenges in predicting overpressure due to its high-temperature and high-pressure reservoir characteristics. By quantitatively analyzing the main mechanisms such as undercompaction, high-temperature fluid expansion, and mud diapirism, the study addresses the complexities of overpressure prediction. This paper introduces an innovative analytical framework that combines hierarchical clustering algorithms with the LightGBM model. Further refined by the application of Bayesian optimization, the model intelligently adjusts hyperparameters to enhance predictive accuracy. Utilizing well logging data and applying machine learning techniques, the paper classifies and identifies different mechanisms causing abnormal pressures, achieving a model prediction accuracy of 0.942. The research findings highlight the predominant role of the undercompaction mechanism, accounting for approximately 70% of the abnormal high-pressure events in the study area. Fluid expansion and shale diapirism contribute smaller but significant proportions of 10% and 20%, respectively. These quantitative insights into the pressure mechanisms are vital for optimizing drilling operations and reducing risks in oil and gas exploration. The study’s hybrid approach, integrating geophysical analysis with advanced computational techniques, sets a precedent for future research. It provides new avenues for applying machine learning to understand complex geological phenomena in similar geological environments and makes a significant contribution to the strategic planning of hydrocarbon exploration and production activities.