The fragility of trial results involves more than statistical significance alone

Abstract

ObjectivesThe fragility of clinical trial findings has been previously defined as the number of changes in outcomes that are required to change their statistical significance. We show that reliance on statistical significance alone provides only a limited and potentially misleading perspective, and an enhanced approach is developed. MethodsClinical importance of trial results and their quantitative stability are incorporated into an enhanced framework to assess fragility. ResultsExamples show that the small data changes required to affect statistical significance may actually be unlikely to occur. Recognizing this limitation, and because statistical significance conveys no information about the treatment effect size, our approach additionally takes into account the clinical importance of the results and their quantitative stability. The interpretation of studies with various combinations of these features is described. ConclusionThe concept of fragility should include clinical importance of trial findings and their quantitative stability, as well as statistical significance. Study results should be declared as stable only if they are statistically significant and quantitatively stable, but they can be either clinically important or unimportant; otherwise, the findings should be declared as unstable, or fragile.