Uncertainty In Economic Evaluations Research Articles

General Population Mortality Adjustment in Survival Extrapolation of Cancer Trials: Exploring Plausibility and Implications for Cost-Effectiveness Analyses in HER2-Positive Breast Cancer in Sweden.

In economic evaluations of novel therapies, assessing lifetime effects based on trial data often necessitates survival extrapolation, with the choice of model affecting outcomes. The aim of this study was to assess accuracy and variability between alternative approaches to survival extrapolation. Data on HER2-positive breast cancer patients from the Swedish National Breast Cancer Register were used to fit standard parametric distribution (SPD) models and excess hazard (EH) models adjusting the survival projections based on general population mortality (GPM). Models were fitted using 6-y data for stage I and II, 4-y data for stage III, and 2-y data for stage IV cancer reflecting an early data cutoff while maintaining sufficient events for comparison of model estimates with actual long-term outcomes. We compared model projections of 15-y survival and restricted mean survival time (RMST) to 15-y registry data and explored the variability between models in extrapolations of long-term survival. Among 11,224 patients compared with the observed registry 15-y RMST estimates across the disease stages, EH cure models provided the most accurate estimates in patients with stage I to III cancer, whereas EH models without cure most closely matched survival in patients with stage IV cancer, in which cure assumption was less plausible. The Akaike information criterion-averaged model projections varied as follows: -8.2% to +5.3% for SPD models, -4.9% to +5.2% for the EH model without a cure assumption, and -19.3% to -0.2% for the EH model with a cure assumption. EH models significantly reduced between-model variance in the predicted RMSTs over a 50-y time horizon compared with SPD models. EH models may be considered as alternatives to SPD models to produce more accurate and plausible survival extrapolation that accounts for general population mortality. Excess hazard (EH) methods have been suggested as an approach to incorporate background mortality rates in economic evaluation using survival extrapolation.We highlight that EH models with or without a cure assumption can produce more accurate survival projections and significantly reduce between-model variability in comparison with standard parametric distribution models across cancer stages.EH models may be a preferred modeling method to reduce model uncertainty in health economic modeling since models that would otherwise have produced implausible extrapolations are constrained by the EH framework.Reduced uncertainty in economic evaluations will enhance the application of evidence-based health care decision making.

Health Technology Reassessment: Addressing Uncertainty in Economic Evaluations of Oncology Drugs at Time of Reimbursement Using Long-Term Clinical Trial Data.

The evidence base to support reimbursement decision making for oncology drugs is often based on short-term follow-up trial data, and attempts to address this uncertainty are not typically undertaken once a reimbursement decision is made. To address this gap, we sought to conduct a reassessment of an oncology drug (pembrolizumab) for patients with advanced melanoma which was approved based on interim data with a median 7.9 months of follow-up and for which long-term data have since been published. We developed a three-health-state partitioned survival model based on the phase 3 KEYNOTE-006 clinical trial data using patient-level data reconstruction techniques based on an interim analysis. We used a standard survival analysis and parametric curve fitting techniques to extrapolate beyond the trial follow-up time, and the model structure and inputs were derived from the literature. Five-year long-term follow-up data from the trial were then used to re-evaluate the cost-effectiveness of pembrolizumab versus ipilimumab for treatment of advanced melanoma. The best fitting parametric curves and corresponding survival extrapolations for reconstructed interim data and long-term data reconstructed from KEYNOTE-006 were different. An analysis of the 5 year long-term follow-up data generated a base case incremental cost-effectiveness ratio (ICER) that was 28% higher than the ICER based on interim trial data. Our findings suggest that there may be a trade-off between certainty and the ICER. Conducting health technology re-assessments of certain oncology products on the basis of longer-term data availability, especially for those health technology adoption decisions made based on immature clinical data, may be of value to decision makers.

Impact of Extrapolation Model Choices on the Structural Uncertainty in Economic Evaluations for Cancer Immunotherapy: A Case Study of Checkmate067.

The aim of this study was to compare the performance of different extrapolation modeling techniques and analyze their impact on structural uncertainties in the economic evaluations of cancer immunotherapy. The individual patient data was reconstructed through published Checkmate067 Kaplan Meier curves. Standard parametric models and six flexible techniques were tested, including fractional polynomial, restricted cubic splines, Royston-Parmar models, generalized additive models, parametric mixture models, and mixture cure models. Mean square errors (MSE) and bias from raw survival plots were used to test the model fitness and extrapolation performance. Variability of estimated incremental cost-effectiveness ratios (ICERs) from different models was used to inform the structural uncertainty in economic evaluations. All indicators were analyzed and compared under cut-offs of 3 years and 6.5 years, respectively, to further discuss model impact under different data maturity. R Codes for reproducing this study can be found on GitHub. The flexible techniques in general performed better than standard parametric models with smaller MSE irrespective of the data maturity. Survival outcomes projected by long-term extrapolation using immature data differed from those with mature data. Although a best-performing model was not found because several models had very similar MSE in this case, the variability of modeled ICERs significantly increased when prolonging simulation cycles. Flexible techniques show better performance in the case of Checkmate067, regardless of data maturity. Model choices affect ICERs of cancer immunotherapy, especially when dealing with immature survival data. When researchers lack evidence to identify the 'right' model, we recommend identifying and revealing the model impacts on structural uncertainty.

Awareness of fetal movements and care package to reduce fetal mortality (AFFIRM): a trial-based and model-based cost-effectiveness analysis from a stepped wedge, cluster-randomised trial

BackgroundThe AFFIRM intervention aimed to reduce stillbirth and neonatal deaths by increasing awareness of reduced fetal movements (RFM) and implementing a care pathway when women present with RFM. Although there is uncertainty regarding the clinical effectiveness of the intervention, the aim of this analysis was to evaluate the cost-effectiveness.MethodsA stepped-wedge, cluster-randomised trial was conducted in thirty-three hospitals in the United Kingdom (UK) and Ireland. All women giving birth at the study sites during the analysis period were included in the study. The costs associated with implementing the intervention were estimated from audits of RFM attendances and electronic healthcare records. Trial data were used to estimate a cost per stillbirth prevented was for AFFIRM versus standard care. A decision analytic model was used to estimate the costs and number of perinatal deaths (stillbirths + early neonatal deaths) prevented if AFFIRM were rolled out across Great Britain for one year. Key assumptions were explored in sensitivity analyses.ResultsDirect costs to implement AFFIRM were an estimated £95,126 per 1,000 births. Compared to standard care, the cost per stillbirth prevented was estimated to be between £86,478 and being dominated (higher costs, no benefit). The estimated healthcare budget impact of implementing AFFIRM across Great Britain was a cost increase of £61,851,400/year.ConclusionsPerinatal deaths are relatively rare events in the UK which can increase uncertainty in economic evaluations. This evaluation estimated a plausible range of costs to prevent baby deaths which can inform policy decisions in maternity services.Trial registrationThe trial was registered with www.ClinicalTrials.gov, number NCT01777022.

Handling Uncertainty in Cost-Effectiveness Analysis: Budget Impact and Risk Aversion

Methods to handle uncertainty in economic evaluation have gained much attention in the literature, and the cost-effectiveness acceptability curve (CEAC) is the most widely used method to summarise and present uncertainty associated with program costs and effects in cost-effectiveness analysis. Some researchers have emphasised the limitations of the CEAC for informing decision and policy makers, as the CEAC is insensitive to radial shifts of the joint distribution of incremental costs and effects in the North-East and South-West quadrants of the cost-effective plane (CEP). Furthermore, it has been pointed out that the CEAC does not incorporate risk-aversion in valuing uncertain costs and effects. In the present article, we show that the cost-effectiveness affordability curve (CEAFC) captures both dimensions of the joint distribution of incremental costs and effects on the CEP and is, therefore, sensitive to radial shifts of the joint distribution on the CEP. Furthermore, the CEAFC also informs about the budget impact of a new intervention, as it can be used to estimate the joint probability that an intervention is both affordable and cost-effective. Moreover, we show that the cost-effectiveness risk-aversion curve (CERAC) allows the analyst to incorporate different levels of risk-aversion into the analysis and can, therefore, be used to inform decision-makers who are risk-averse. We use data from a published cost-effectiveness model of palbociclib in addition to letrozole versus letrozole alone for the treatment of oestrogen-receptor positive, HER-2 negative, advanced breast cancer to demonstrate the differences between CEAC, CEAFC and CERAC, and show how these can jointly be used to inform decision and policy makers.

EM4 Recommendations for Handling Uncertainty in Economic Evaluation: A Targeted Review of Pharmacoeconomic Guidelines

The appropriate handling of uncertainty is an essential element of economic evaluation of healthcare interventions. Accepted methods include deterministic and probabilistic analyses to characterise parametric uncertainty, and scenario analyses to test uncertainty propagated by methodological and structural assumptions. This targeted review examined which methods for handling uncertainty are recommended in pharmacoeconomic guidelines around the world.

Probabilistic One-Way Sensitivity Analysis with Multiple Comparators: The Conditional Net Benefit Frontier.

Although there have been substantial developments in the analysis of uncertainty in economic evaluations of health care programmes, the development of methods for one-way sensitivity analysis has been notably slower. Conditional incremental net benefit was recently proposed as an approach for implementing probabilistic one-way sensitivity analysis for economic evaluations comparing two strategies. In this paper, we generalise this approach to economic evaluations that compare three or more strategies. We find that ‘conditional net benefit’ may be used to conduct probabilistic one-way sensitivity analysis for economic evaluations comparing any number of strategies. We also propose the ‘conditional net benefit frontier’, which may be used to identify the most cost-effective of any number of strategies conditional upon the specific value of a parameter of interest.

Cohort versus patient level simulation for the economic evaluation of single versus combination immuno-oncology therapies in metastatic melanoma

Background: Model structure, despite being a key source of uncertainty in economic evaluations, is often not treated as a priority for model development. In oncology, partitioned survival models (PSMs) and Markov models, both types of cohort model, are commonly used, but patient responses to newer immuno-oncology (I-O) agents suggest that more innovative model frameworks should be explored.Objective: A discussion of the theoretical pros and cons of cohort level vs patient level simulation (PLS) models provides the background for an illustrative comparison of I-O therapies, namely nivolumab/ipilimumab combination and ipilimumab alone using patient level data from the CheckMate 067 trial in metastatic melanoma. PSM, Markov, and PLS models were compared on the basis of coherence with short-term clinical trial endpoints and long-term cost per QALY outcomes reported.Methods: The PSM was based on Kaplan-Meier curves from CheckMate 067 with 3-year data on progression free survival (PFS) and overall survival (OS). The Markov model used time independent transition probabilities based on the average trajectory of PFS and OS over the trial period. The PLS model was developed based on baseline characteristics hypothesized to be associated with disease as well as significant mortality and disease progression risk factors identified through a proportional hazards model.Results: The short-term Markov model outputs matched the 1–3 year clinical trial results approximately as well as the PSMs for OS but not PFS. The fixed (average) cohort PLS results corresponded as well as the PSMs for OS in the combination therapy arm and PFS in the monotherapy arm. Over the lifetime horizon, the PLS produced an additional 5.95 quality adjusted life years (QALYs) associated with combination therapy relative to ipilimumab alone, resulting in an incremental cost-effectiveness ratio (ICER) of £6,474 per QALY, compared with £14,194 for the PSMs which gave an incremental benefit of between 2.2 and 2.4 QALYs. The Markov model was an outlier (∼ £49,000 per QALY in the base case).Conclusions: The 4- and 5-state versions of the PSM cohort model estimated in this study deviate from the standard 3-state approach to better capture I-O response patterns. Markov and PLS approaches, by modeling state transitions explicitly, could be more informative in understanding I-O immune response, the PLS particularly so by reflecting heterogeneity in treatment response. However, both require a number of assumptions to capture the immune response effectively. Better I-O representation with surrogate endpoints in future clinical trials could yield greater model validity across all models.

Representing Uncertainty In Economic Evaluations: Getting More From PSA Results

Representing Uncertainty In Economic Evaluations: Getting More From PSA Results

Health State Utility Values for Age-Related Macular Degeneration: Review and Advice

Health state utility values are a major source of uncertainty in economic evaluations of interventions for age-related macular degeneration (AMD). This review identifies and critiques published utility values and methods for eliciting de novo utility values in AMD. We describe how utility values have been used in healthcare decision making and provide guidance on the choice of utility values for future economic evaluations for AMD. Literature was searched using PubMed, and health technology assessments (HTA) were searched using HTA agency websites to identify articles reporting utility values or approaches to derive utility values in AMD and articles applying utilities for use in healthcare decision making relating to treatments for AMD. A total of 70 studies qualified for data extraction, 22 of which were classified as containing utility values and/or elicitation methods, and 48 were classified as using utility values in decision making. A large number of studies have elicited utility values for AMD, although those applied to decision making have focused on a few of these. There is an appreciation of the challenges in the measurement and valuation of health states, with recent studies addressing challenges such as the insensitivity of generic health-related quality of life (HRQoL) questionnaires and utility in the worse-seeing eye. We would encourage careful consideration when choosing utility values in decision making and an explicit critique of their applicability to the decision problem.

Exploring Uncertainty in Economic Evaluations of Drugs and Medical Devices: Lessons from the First Review of Manufacturers' Submissions to the French National Authority for Health.

The objective of this paper was to evaluate how uncertainty has been accounted for in the cost-effectiveness analyses (CEAs) submitted by manufacturers to the French National Authority for Health (HAS) and to identify recurring concerns in these submissions. We used a cross-sectional design to evaluate manufacturers' submissions from the beginning of the evaluation process in October 2013 to the end of May 2015 (n=28). The sources of uncertainty attached to these CEAs were categorized and assessed. Relevant data were extracted independently by two assessors. Adherence to the HAS reference case was generally considered to be acceptable. Methodological uncertainty and parameter uncertainty were the sources of uncertainty that were most frequently explored by manufacturers. The quality of reporting of deterministic sensitivity analysis and probabilistic sensitivity analysis varied substantially across submissions, with a frequent lack of justification of the plausible range of parameter point estimates in 12 submissions (43%). Structural uncertainty was explored much less frequently. Concerns related to omission of either important clinical events or relevant health states or extrapolation of the effects of the technology beyond the time horizon of the clinical trials were identified in 16 submissions (57%). This study presented a characterization of the treatment of uncertainty for the first 28 manufacturers' submissions to the HAS. This work identified important concerns regarding the exploration of sources of uncertainty. The findings may help manufacturers to improve the quality of their submissions and may provide useful insights for extending guidelines on uncertainty analysis in CEAs submitted to the HAS.

Exploring Uncertainty in Economic Evaluation Of Medicines: A Review of The First Manufacturers' Submissions To the French National Authority For Health (Has).

Exploring Uncertainty in Economic Evaluation Of Medicines: A Review of The First Manufacturers' Submissions To the French National Authority For Health (Has).

RM41 - A novel approach to ranking parameter uncertainty in one-way sensitivity analysis: What tornado diagrams are missing

Economic evaluations of health technologies have long relied on one-way sensitivity analysis (SA) to examine the impact of parameter uncertainty on modeling outcomes. Traditionally, this impact has been measured and ranked based on absolute changes in the incremental cost-effectiveness ratio (ICER) across plausible parameter values and presented in a tornado diagram. This format does not adequately identify or prioritize parameters where the range of uncertainty causes the ICER to change quadrants in the cost-effectiveness (CE) plane. However, these quadrant changes, which represent fundamental changes to the CE conclusion, are arguably more meaningful than changes in the ICER within a quadrant. This research illustrates a novel approach to presenting one-way SA results that focuses on identifying parameters with the greatest potential to change the overall CE conclusion rather than narrowly focusing on changes to the ICER. We developed a comprehensive algorithm for ranking the parameters varied in a one-way SA. Broadly, we first prioritize parameters with the potential to qualitatively change the CE conclusion and then rank parameters based on quantitative changes to modeling outcomes. Changes to the CE conclusion are identified based on quadrant changes in the CE plane, and parameters are categorized as having the potential to change the conclusion both positively and negatively, only positively, only negatively, or not at all. Within these categories, a secondary ranking based on costs and health outcomes is used. Furthermore, visualization techniques anchored in the CE plane help assess whether conclusion changes are due primarily to changes in health, changes in costs, or both. This research demonstrates that the conclusions-based ranking algorithm works in more general settings than the traditional tornado diagram format. The conclusions-based approach is a powerful method that provides a more complete picture of the impact of parameter uncertainty in economic evaluations.

PRM25 Issues and Uncertainties in Combinations of Diagnostic Technologies: A Case Study in Epilepsy Surgery

There exists very little in the way of current guidelines or literature as to the combination and sequence of visualisation technologies (invasive or non-invasive) that should be used for the visualisation of seizure focus in patients with refractory epilepsy being considered for surgery. This lack of guidelines is typical in many diagnostic technologies. This study seeks to both create an explanatory analytical framework to demonstrate the variety of technologies, sequences and combinations available before epilepsy surgery is conducted, as well as focussing on the more general questions of the difficulties associated with the evaluation of diagnostic technologies and consider the role of methods to characterise uncertainty in economic evaluations. The decision analytical framework consisted of three different potential strategies that could be employed in order to elicit the most accurate information given an appropriate cost-effectiveness threshold. This framework was used to consider the wider difficulties and sensitivities of the analysis of diagnostic technologies in combination. The analytical framework section of this study found that (assuming an initial MRI and video-EEG) a strategy of FDG-PET being the first test being performed, followed by invasive EEG if the result of the test does not lead to a clear decision on whether the patient should proceed to surgery was optimal. In a general context, we found that in the evaluation of combinations of diagnostic technologies it is important not only to consider the added value of each individual test but also the correct combination and order in which the tests are employed. In addition the evaluation of diagnostics will always be highly sensitive to the cost-effectiveness of the subsequent treatment.

Communicating Uncertainty in Economic Evaluations

In cost-effectiveness analysis (CEA), it is common to compare a single, new intervention with 1 or more existing interventions representing current practice ignoring other, unrelated interventions. Sectoral CEAs, in contrast, take a perspective in which the costs and effectiveness of all possible interventions within a certain disease area or health care sector are compared to maximize health in a society given resource constraints. Stochastic league tables (SLT) have been developed to represent uncertainty in sectoral CEAs but have 2 shortcomings: 1) the probabilities reflect inclusion of individual interventions and not strategies and 2) data on robustness are lacking. The authors developed an extension of SLT that addresses these shortcomings. Analogous to nonprobabilistic MAXIMIN decision rules, the uncertainty of the performance of strategies in sectoral CEAs may be judged with respect to worst possible outcomes, in terms of health effects obtainable within a given budget. Therefore, the authors assessed robustness of strategies likely to be optimal by performing optimization separately on all samples and on samples yielding worse than expected health benefits. The approach was tested on 2 examples, 1 with independent and 1 with correlated cost and effect data. The method was applicable to the original SLT example and to a new example and provided clear and easily interpretable results. Identification of interventions with robust performance as well as the best performing strategies was straightforward. Furthermore, the robustness of strategies was assessed with a MAXIMIN decision rule. The SLT extension improves the comprehensibility and extends the usefulness of outcomes of SLT for decision makers. Its use is recommended whenever an SLT approach is considered.

Exploring Model Uncertainty in Economic Evaluation of Health Interventions: The Example of Rotavirus Vaccination in Vietnam

Motivated by observed discrepancies between 2 published studies on the cost-effectiveness of rotavirus vaccination in Vietnam, the authors' objectives were to illustrate a specific, systematic approach to assessing model (structure and process) uncertainty and to quantify explicitly the contributions of different sources of variation in the outputs of different studies that share the same research question. On the basis of a series of working definitions of key model elements, the authors developed 5 alternative computer simulation (state-transition) models of rotavirus disease. They examined how epidemiological outcomes and cost-effectiveness ratios associated with rotavirus vaccination would change as elements of model structure and modeling process were progressively modified. They also explicitly decomposed the relative contributions of different modeling elements to differences in the cost-effectiveness results between the 2 previous analyses motivating the present study. The findings suggest that within the category of a static, deterministic, aggregate-level model, different choices in model structure and process lead to relatively modest differences in the estimated cost-effectiveness of rotavirus vaccination, but that intermediate epidemiologic outcomes vary more substantially depending on the choice of model structure. The authors caution against generalizing the quantitative results in this study beyond the present example but suggest that the approach presented here may serve as a template for other examinations of model uncertainty. As new research questions arise after the introduction of rotavirus vaccination programs, a reevaluation of model uncertainty is likely to be needed.

Handling uncertainty in economic evaluations of patient level data: A review of the use of Bayesian methods to inform health technology assessments

Due to potential advantages (e.g., using all available evidence), Bayesian methods have been proposed to assist healthcare decision making. This review provides a detailed description of how Bayesian methods have been applied to economic evaluations of patient level data. The results serve both as a reference and as a means by which to examine the appropriate application of Bayesian methods to inform decision making. MEDLINE, EMBASE, and Cochrane Economic Evaluation databases were searched to identify studies, published up to November 2007, meeting three inclusion criteria: (i) the study conducted an economic evaluation, (ii) sampling uncertainty was incorporated using Bayesian methods, (iii) the likelihood function was informed by patient level data from a single source. Data were collected on key study characteristics (e.g., prior distribution, likelihood function, presentation of uncertainty). The search identified 366 potentially relevant studies, from which 103 studies underwent full-text review. Sixteen studies met the inclusion criteria. Half of the studies used uninformative priors; most studies incorporated the potential dependence between costs and effects, and presented cost-effectiveness acceptability curves. Results were sensitive to changes in the priors and likelihoods. Limited use of informative priors, among the included studies, gives policy makers little guidance on one of the main benefits of Bayesian methods, the ability to integrate all available evidence to capture the uncertainty inherent in decision making.

Approaches for Economic Evaluations of Health Care Technologies

Economic evaluations involve the comparison of alternative courses of action in terms of both their costs and their consequences. In response to increasing health care costs, economic evaluations of competing technologies, including radiologic interventions, are increasingly used to inform resource allocation decisions. It is therefore crucial that radiologists have a thorough understanding of the methods. The objective of this paper is to present a detailed overview of the principles and methods of economic evaluations of health technologies, including recent methodologic developments. For the purpose of this paper, the key elements of an economic evaluation are divided into 5 broad sections: 1) types of economic evaluations (eg, cost-effectiveness analysis, cost-utility analysis), 2) study perspectives (ie, temporal and cost perspectives), 3) analysis of costs and effects (eg, incremental cost-effectiveness ratios, cost-effectiveness planes), 4) conducting economic evaluations alongside trials or using decision-analytic models, and 5) dealing with the different forms of uncertainty in economic evaluations (eg, sampling uncertainty in trials, parameter uncertainty in models). Examples from the recent radiology literature are used to explain the key concepts. This review improves upon the previous educational papers published in radiologic journals, as it covers recent methodologic advances regarding the treatment of uncertainty.

Estimation, Power and Sample Size Calculations for Stochastic Cost and Effectiveness Analysis

Various methods have been proposed to address uncertainty in economic evaluations of healthcare programmes. One approach suggested in the literature is to estimate separate confidence intervals for the incremental costs and effects of a new health programme in comparison with an existing programme. These intervals are then combined to generate a rectangular confidence region in the cost-effectiveness plane that implicitly defines a corresponding confidence interval for the incremental cost-effectiveness ratio (ICER). The same approach has been used to calculate sample sizes and study power. This application of the rectangle method is consistent with the adoption of ICERs and a threshold as a decision rule, this being the most commonly used approach in empirical applications of cost-effectiveness analysis, as well as the one recommended by agencies that assess medical technology around the world. In this paper, we first outline the rectangle method, and then propose a modification that recognises that separate inferences are being drawn on the cost and effectiveness domains, and that corrects for multiple statistical comparisons. The confidence rectangle is otherwise too small, the corresponding confidence interval for the ICER is too narrow and sample sizes are under-estimated. Our modification corrects these problems. A further difficulty is that the placement of the confidence rectangle around the null value is somewhat arbitrary, and does not correspond to a unique value of ICERs. As a result, different values of sample size and power for the estimation of ICERs can be obtained, depending on the null values of the cost and effectiveness. We conclude that it is important to clearly identify the analytic goal in terms of estimating differential costs, differential effects or a combination of the two using the ICER index. These ideas are illustrated using numerical examples.

Quantification of Oil Price Uncertainty in Economic Evaluation Using Sequential Gaussian Simulation

In this paper, we propose improved methodology for quantifying the uncertainty in petroleum economic evaluations due to uncertainty in future oil prices. Conventional “hockey stick” price forecasts commonly used in industry significantly underestimate uncertainty because they do not reproduce the volatility inherent in oil prices. Some authors have proposed stochastic methods, such as the bootstrap method, to better model oil price volatility. A disadvantage of the bootstrap method, however, is that it can produce price realizations with unrealistically low or high prices. To address this shortcoming, we present two stochastic methods that honor both the historical distribution of oil prices as well as the historically observed variability in oil prices. The first is an approximate method in which multiple future price realizations are taken directly from different windows of historical uninflated price data. The method is easy to apply and can provide greater insights into the uncertainty and risks associated with project economic evaluation. However, this method can potentially overestimate uncertainty in oil prices and should be used judiciously. The second method uses sequential Gaussian simulation to generate equiprobable future oil price realizations consistent with both the frequency distribution and temporal variability of historical prices. Although the method is more difficult to apply, it provides a more statistically sound basis for quantifying uncertainty. The improved methods are compared to conventional methods for three synthetic cases derived from the literature and a field case. Results demonstrate that conventional methods underestimate oil price uncertainty more severely for projects with accelerated cash flow streams, which characterize most petroleum development projects. This indicates that probabilistic methods for quantifying uncertainty in oil prices, such as the sequential Gaussian simulation method proposed here, are needed to provide operators with reliable quantifications of the uncertainties and risks associated with individual projects, which should enable improved investment decision making.