Field Features Do Not Explain Greenfield Production Forecasting Bias

Abstract

Summary Over the past years, it has become clear that greenfield oil production forecasts are subject to strong optimism and overprecision biases: Significant early production shortfalls are the rule rather than the exception and the elicited uncertainty range is generally too narrow. This has large negative consequences for the net present valuation of such investments. A study from 2011 based on post-factum evaluation of greenfield production forecasts suggests that there is a causal relationship between certain project/field characteristics and production attainment (i.e., optimism bias). However, while self-reported causes of failure may provide interesting insights, such analyses are subject to cognitive hindsight bias. It is therefore necessary to test such claims more rigorously. Research on megaprojects in other industries suggests that forecasting bias is omnipresent but is stronger in certain circumstances (e.g., information and communication technology projects are subject to larger cost overruns than road construction projects). An important question is therefore whether there are combinations of field characteristics/features which can be measured objectively, such as field size, reservoir complexity, oil prices, and lack of drillstem tests (DSTs), etc., and which can be shown to have predictive power of overly optimistic and overconfident production forecasts. The data set in this study consists of 71 greenfield oil production forecasts at project sanction on the Norwegian Continental Shelf (NCS), with production starting between 1995 and 2020. Each forecast consists of a triplet of production curves which represent the statistical p10, the expectation, and the statistical p90. The forecasts are compared with actual production data. Metadata about the fields gathered from the Norwegian Petroleum Directorate (NPD) are used to establish 16 informative field features, from field reserves to the number of appraisal wells per unit area. These features are tested for predictive power, both individually and simultaneously, of optimism bias and of a general forecast quality metric. First, we show that value erosion caused by time overruns and production shortfalls are both significant, but that the relative importance of effects after production start is higher. Second, none of the tested machine learning models show any predictive power of forecasting bias. Because of this systematic presence of bias in the production forecasts, we argue that oil and gas companies need to make important changes to their decision-making workflows to take into account well-documented research findings on cognitive and organizational bias from the past decades, instead of the ever-increasing model complexity. Illustratively, as a final point, we show that a no-skills-involved reference class forecast based on empirical production curves from abandoned fields outperforms operators’ own greenfield forecasts. This approach may perhaps serve as a useful benchmark for future forecasts.