Using Bayesian Leave-One-Out and Leave-Future-Out Cross-Validation to Evaluate the Performance of Rate-Time Models to Forecast Production of Tight-Oil Wells

Abstract

Summary Production forecasting is usually performed by applying a single model from a classical statistical standpoint (point estimation). This approach neglects: (a) model uncertainty and (b) quantification of uncertainty of the model’s estimates. This work evaluates the predictive accuracy of rate-time models to forecast production from tight-oil wells using Bayesian methods. We apply Bayesian leave-one-out (LOO) and leave-future-out (LFO) cross-validation (CV) using an accuracy metric that evaluates the uncertainty of the models’ estimates: the expected log predictive density (elpd). We illustrate the application of the procedure to tight-oil wells of west Texas. This work assesses the predictive accuracy of rate-time models to forecast production of tight-oil wells. We use two empirical models, the Arps hyperbolic and logistic growth models, and two physics-based models: scaled slightly compressible single-phase and scaled two-phase (oil and gas) solutions of the diffusivity equation. First, we perform Bayesian inference to generate probabilistic production forecasts for each model using a Bayesian workflow in which we assess the convergence of the Markov chain Monte Carlo (MCMC) algorithm, calibrate, and evaluate the robustness of the models’ inferences. Second, we evaluate the predictive accuracy of the models using the elpd accuracy metric. This metric provides a measure of out-of-sample predictive performance. We apply two different CV techniques: LOO and LFO. The results of this study are the following. First, we evaluate the predictive performance of the models using the elpd accuracy metric, which accounts for the uncertainty of the models’ estimates assessing distributions instead of point estimates. Second, we perform fast CV calculations using an important sampling technique to evaluate and compare the results of the application of two CV techniques: leave-one-out cross-validation (LOO-CV) and leave-future-out cross-validation (LFO-CV). While the goal of LOO-CV is to evaluate the models’ ability to accurately resemble the structure of the production data, LFO-CV aims to assess the models’ capacity to predict future-time production (honoring the time-dependent structure of the data). Despite the difference in their prediction goals, both methods yield similar results on the set of tight-oil wells under study. The logistic growth model yields the best predictive performance for most of the wells in the data set, followed by the two-phase physics-based flow model. This work shows the application of new tools to evaluate the predictive accuracy of models used to forecast production of tight-oil wells using: (a) an accuracy metric that accounts for the uncertainty of the models’ estimates and (b) fast computation of two CV techniques, LOO-CV and LFO-CV. To our knowledge, the proposed approach is novel and suitable to evaluate and eventually select the rate-time model(s) with the best predictive accuracy of models to forecast hydrocarbon production.