Quantifying the unquantifiable

Abstract

Applicability has been described as ‘‘inferences about the extent to which a causal relationship holds over variations in persons, settings, treatments, and outcomes’’ [1,2]. Consequently, to determine the applicability of any research result to a population of interest in a given setting, one has to take two aspects into consideration: first, whether, and to what extent, differences in patient characteristics, settings, treatments, and outcomes are present between a given body of evidence and a population of interest and second, whether existing differences can act as effect modifiers and have the potential to alter or even reverse the magnitude of an observed treatment effect. The first step, assessing differences in patient characteristics, settings, treatments, and outcomes, sounds straightforward but is, in reality, often hampered by lack of reporting [3]. The second step, determining the potential impact of differences on the magnitude of a treatment effect, is even more challenging. Such an assessment is frequently fraught with unknown factors such as the extent of effect modification caused by specific differences in patient characteristics, interventions, and so forth. In reality, assessments of applicability, therefore, often become a matter of assumptions and subjective judgments. In this issue of the journal, Koppenaal et al. [4] propose an adaptation of the Pragmatic-Explanatory Continuum Indicator Summary tool to be used to grade the applicability of systematic reviews of trials. The idea is intriguing because unless conclusions of systematic reviews are both valid and applicable to broader patient populations in realworld settings, decisions based on such systematic reviews may be flawed. Devising an applicability score that could complement the grading of the strength of a body of evidence would be a useful feature for users of systematic reviews. But is it possible to quantify and rate the applicability of an entire body of evidence in a systematic review? I am