The Fragility Index in Hand Surgery Randomized Controlled Trials

Abstract

Randomized controlled trials (RCTs) are the gold standard for comparing clinical interventions. Statistical significance as reported via a P value has been used to determine if a difference between clinical interventions exists in an RCT. However, P values do not clearly convey information about the robustness of a study's conclusions. An emerging metric, called the fragility index (the number of subjects who would need to change outcome category to raise the P value above the .05 threshold), is an indirect measure of how likely a repeat of the trial would reach the same conclusions. This study addressed the fragility of RCTs using dichotomous outcomes in hand surgery. Using systematic searching of the MEDLINE database, we identified hand surgery RCTs published in 11 high-impact journals published in the last decade (2007-2017). Studies were identified that involved 2 parallel arms, allocated patients to treatment and control in a 1:1 ratio, and reported statistical significance for a dichotomous variable. The fragility index was calculated using Fisher's exact test, using previously published methods. Five hand surgery RCTs were identified for inclusion reporting a range of fragility indices from 0 to 26. Two of the trials (40%) had a fragility index of 2 or less. Two of the trials (40%) reported that the number of patients lost to follow-up exceeded the fragility index, meaning that results of the patients lost to follow-up could theoretically completely reverse the study conclusions. The range of fragility indices reported in the recent hand surgery literature is consistent with previous reporting within orthopedic surgery. The fragility index is a useful metric to analyze the robustness of the study conclusions that should complement other methods of critical evaluation including the P value or effect sizes. Our results emphasize the need for future efforts to strengthen the robustness of RCT conclusions.