The Net Benefit of a treatment should take the correlation between benefits and harms into account

Eva Cantagallo,Marc Buyse,Julien Peron,Mickaël De Backer,Oriana Ciani,Jean-Christophe Chiem,Everardo D Saad

doi:10.1016/j.jclinepi.2021.03.018

Abstract

ObjectiveThe assessment of benefits and harms from experimental treatments often ignores the association between outcomes. In a randomized trial, generalized pairwise comparisons (GPC) can be used to assess a Net Benefit that takes this association into account. Study design and settingsWe use GPC to analyze a fictitious trial of treatment versus control, with a binary efficacy outcome (response) and a binary toxicity outcome, as well as data from two actual randomized trials in oncology. In all cases, we compute the Net Benefit for scenarios with different orders of priority between response and toxicity, and a range of odds ratios (ORs) for the association between these outcomes. ResultsThe GPC Net Benefit was quite different from the benefit/harm computed using marginal treatment effects on response and toxicity. In the fictitious trial using response as first priority, treatment had an unfavorable Net Benefit if OR < 1, but favorable if OR > 1. With OR = 1, the Net Benefit was 0. Results changed drastically using toxicity as first priority. ConclusionEven in a simple situation, marginal treatment effects can be misleading. In contrast, GPC assesses the Net Benefit as a function of the treatment effects on each outcome, the association between outcomes, and individual patient priorities.

Highlights

The assessment of benefits and harms from experimental treatments usually comprises qualitative and quantitative considerations, and relies on sources of data as varied as hospital records, pharmacovigilance databases, randomized and non-randomized clinical trials, etc. [1] A review of methods for benefit/harm assessment in systematic reviews identified four stages: a review of the reported benefits and harms of an intervention; quantitative assessments of the intervention’s benefits as compared with harms; decision-making at the population level; and decision-making at the individual level [2]
The generalized pairwise comparisons (GPC) Net Benefit was quite different from the benefit/harm computed using marginal treatment effects on response and toxicity
If only aggregate data were available for the fictitious trial described above, the higher response probability with the experimental treatment than with control would point to a benefit of the experimental treatment in terms of its efficacy, while the toxicity observed with the experimental treatment would point to harm of the experimental treatment in terms of its safety

Summary

Introduction

The assessment of benefits and harms from experimental treatments usually comprises qualitative and quantitative considerations, and relies on sources of data as varied as hospital records, pharmacovigilance databases, randomized and non-randomized clinical trials, etc. [1] A review of methods for benefit/harm assessment in systematic reviews identified four stages: a review of the reported benefits and harms of an intervention; quantitative assessments of the intervention’s benefits as compared with harms; decision-making at the population level; and decision-making at the individual level [2]. The assessment of benefits and harms from experimental treatments usually comprises qualitative and quantitative considerations, and relies on sources of data as varied as hospital records, pharmacovigilance databases, randomized and non-randomized clinical trials, etc. The quantitative assessment of benefit/risk is typically based on aggregate data (i.e., summary statistics) for the outcomes observed with the intervention as compared with standard of care or competing interventions. Efficacy and safety are usually analyzed separately, possibly using different data sources, and the results of these analyses are combined quantitatively into a benefit/harm ( called benefit/risk) assessment. Many methods have been proposed and used to perform this quantitative assessment, but it has long been recognized that a limitation of approaches based on aggregate data is that the association between the different outcomes of interest is ignored [3]. Buyse et al / Journal of Clinical Epidemiology 137 (2021) 148–158

Methods

Results

Conclusion