In reply:

Abstract

We thank Niforatos et al for their contribution to the discussion in regard to the limitations of the fragility index and fragility quotient. In their letter, they expand on some of the limitations to the application of the fragility index and fragility quotient discussed in our original work and ultimately question their utility. Although they highlight some important considerations, we believe that the measures have clinical utility. Here, we explore further some key points about their application and provide insight into how clinicians might use these tools. The fragility index is a metric that provides the absolute number of events necessary to reverse statistical significance. We note that because its calculation relies on P values, the fragility index is subject to the same limitations inherent to P values themselves. Carter et al1Carter R.E. McKie P.M. Storlie C.B. The fragility index: a P-value in sheep’s clothing?.Eur Heart J. 2017; 38: 346-348Google Scholar expanded on this notion and illustrated how P values are repackaged into the calculation of the fragility index to generate an inverse relationship between the two. Thus, as is true with P values, the fragility index should not be misinterpreted as a measure of clinical effect. Narayan et al2Narayan V.M. Gandhi S. Chrouser K. et al.The fragility of statistically significant findings from randomised controlled trials in the urological literature.BJU Int. 2018; 122: 160-166Google Scholar noted that “just as a P value of 0.001 compared with 0.04 does not signal a greater importance of the findings, a fragility index of 10 should not be interpreted to imply greater clinical effect than a fragility index of 1; rather, it simply illustrates the strength of the statistical significance itself.” Clinicians must therefore recognize that the fragility index represents an easily interpretable measure of the stability of statistically significant findings and not a measure of clinical effect. Because the fragility index provides the absolute number of events required to reverse significance, it has particular utility in interpreting study findings in the context of loss to follow-up. When the number of patients lost to follow-up exceeds the fragility index, results should be viewed with particular skepticism because factoring in the unknown outcomes of those patients might reverse significance.3Tignanelli C.J. Napolitano L.M. The fragility index in randomized clinical trials as a means of optimizing patient care.JAMA Surg. 2019; 154: 174-179Google Scholar Moreover, the fragility index should not be interpreted in isolation. Rather, it complements the P value and confidence interval.4Matics T.J. Khan N. Jani P. et al.The fragility index in a cohort of pediatric randomized controlled trials.J Clin Med Res. 2017; 6: 79Google Scholar When interpreted in conjunction with the aforementioned and paired with other statistical measures such as the number needed to treat, the fragility index may offer further insight into the reliability of statistically significant findings. Carter et al1Carter R.E. McKie P.M. Storlie C.B. The fragility index: a P-value in sheep’s clothing?.Eur Heart J. 2017; 38: 346-348Google Scholar recommended yet a broader approach to study interpretation that encompasses estimated efficacy, study design, and mitigation of biases. This may be best accomplished by abandoning the frequentist approach altogether in favor of alternative methodology such as Bayesian analysis. However, as long as researchers insist on a frequentist framework, added metrics such as the fragility index may provide further insight into the stability of statistically significant results. Although the fragility index has limitations, it also has clear utility as an intuitive measure of the reliability of study findings when interpreted in conjunction with the P value and confidence interval. We maintain that the fragility index should be normalized to sample size with the fragility quotient. Few studies have reported on the fragility quotient, however, and we are unaware of any that have investigated its reliability. Thus, we encourage further research on the concept of fragility as a whole. Fragility Measures: More Limitations ConsideredAnnals of Emergency MedicineVol. 73Issue 6PreviewWe commend the work by Brown et al1 that, using the fragility index and fragility quotient, assesses the fragility of randomized controlled trials in the emergency medicine literature. Although they provide a good overview of the limitations of the 2 metrics, we would like to further question the utility of fragility measures. Full-Text PDF The Results of Randomized Controlled Trials in Emergency Medicine Are Frequently FragileAnnals of Emergency MedicineVol. 73Issue 6PreviewRandomized controlled trials govern evidence-based clinical practice, and it is therefore critical that their results be robust. We aim to investigate the fragility of randomized controlled trials in emergency medicine by determining how often significance would be nullified with small changes in outcomes using the fragility index. Full-Text PDF