M clinical trial results relevant to decision makers has been a theme in the Society for Medical Decision Making and this journal since their inception. Sample size calculations, ethical design, and the use of research results are also issues of long-standing interest to MDM and its readers. The “sufficiently important differences” (SID) proposed by Barrett and colleagues falls within this decades-long conversation and provides a potential solution that is patient oriented and decision-analytically coherent. To realize this potential, 3 methodologic issues need to be examined more closely: the relationship between SID and reporting of clinical trial results, the relationship between SID and the design of clinical trial results, and potential problems in elicitation. SID and reporting is straightforward. SID falls within the framework of providing methods to help individuals to make decisions that are well founded and coherent with their values and beliefs, which is a major goal of MDM. Based on 40 years of literature and a demonstration project, Lehmann and Goodman listed several desiderata for clinical-trial reporting that would employ a Bayesian and decision-analytic framework. They called for an interaction that would engage the reader’s prior knowledge and desired treatment threshold and convey the treatment effect’s size and variability. SID satisfies these goals handily. Lehmann and Goodman further raised the issue of communities of prior beliefs and communities of thresholds, which Barrett and colleagues address with their attention to the distribution of SIDs. Studies now in progress are heeding the call of Barrett and colleagues. For instance, DeBaun and colleagues, in a clinical trial of blood transfusion therapy for sickle-cell disease, are performing utility assessments and standard gambles for detecting SIDs prospectively on patients and providers in what they call a “risk-benefit analysis.” Implicit in Barrett and colleagues’ references but unstated in the article’s text, practitioners of evidencebased medicine often look to the number needed to treat (NNT) as a method of conveying the clinical significance of results, after the fact. In the terms used by Barrett and colleagues, a threshold maximum NNT is related to the minimum important difference. Similarly, for an adverse outcome, the number needed to harm (NNH) would lead to the maximum acceptable harm threshold, which Barrett and colleagues represent as a vertical line rising out of the x-axis in their Figure 2. Two points follow. First, threshold NNTs and NNHs lead to a quadrant of acceptability, and it is precisely the authors’ point that a truncated region better captures the thresholds. The 2nd, more subtle point is that NNT and NNH represent the perspective of a clinician or public health decision maker. It is not clear that patients understand the NNTs and the NNHs in the same way that clinicians do. Using SIDs in the design phase is a more contentious idea. Two recent books illuminate the high financial and medical stakes of the drug-approval process and also how, in this process, demonstrating that a new agent is more effective than the standard therapy is low on the list of concerns. (Evaluating whether it is as good at satisfying patients’ desires may be an even lower priority.) From a methodological perspective, it is crucial to choose a single outcome measure in a study, and a single threshold for that single outcome. If, in monitoring a trial, it becomes clear that the effect size is greater than the threshold, then it becomes unethical to continue that trial. On the other hand, if the effect size is so small in interim analyses that it would be futile to continue, it would likewise be unethical to continue the study. If an SID is used, then at least 2 outcomes must
Read full abstract