PDG61 Incorporating Uncertainty in Clinical Benefit in Health Technology Assessments: A Review and Classification of Agency Approaches

Abstract

Relative effectiveness assessment (REA) conclusions inform HTA decisions. are based on evidence which is subject to uncertainty. This study outlined the ways in which HTA organizations around the world handle uncertainty in REAs. It analysed commonalities and differences based on HTA guidelines, using the GRADE framework as reference. Guidelines of 7 HTA organizations (IQWiG, NICE, HAS, ICER, ZIN, CADTH and EUnetHTA) regarding REA and uncertainty assessment methods were included. Three recent HTA reports of each organization were randomly selected to verify and clarify guidelines. Two analyses were performed: first, the role assigned to uncertainty by organizations was discussed while assessing uncertainty expression on four levels: individual study, body of evidence for one outcome, body of evidence across all outcomes and added benefit. Second, the extent to which HTA organizations considered the 8 domains of certainty of evidence defined by GRADE was assessed. Uncertainty was explicitly reported as a primary goal of REA alongside magnitude of added benefit by IQWiG, ICER, EUnetHTA and ZIN. NICE and CADTH include narrative critical appraisal sections and HAS incorporate certainty in added benefit rankings, without reporting uncertainty as primary goal. In the four predefined levels, ordinal scales, narrative approaches, and checklists were employed to express uncertainty. All organizations reported at least 4/5 downgrading domains of GRADE while the 3 upgrading domains were reported the least (by HAS, IQWiG, NICE, ICER, unclear operationalization for CADTH). ZIN and EUnetHTA reported on all domains. Assessment operationalization of some domains was unclear. This study provides insight in uncertainty assessment of clinical evidence within HTA. The approaches for expressing uncertainty vary from more structured to freer. The discrepancies in the domains included attest to the lack of a universal definition on uncertainty. More guidance on the best methods to incorporate uncertainty in HTA could lead to more aligned decision-making.