Median Fragility Index Research Articles

Evaluation of the robustness of randomized controlled trials for the treatment modalities of esophageal cancer using the fragility index - a systematic review.

Esophageal cancer remains a significant global health challenge. Several treatment modalities were explored in randomized controlled trials (RCTs) in recent decades. This study evaluates the robustness of RCTs focusing on esophageal cancer treatment using the fragility index (FI) and reverse fragility index (RFI). A systematic review of RCTs studying different treatment modalities for esophageal cancer from 2000 to 2023 was conducted. The FI and RFI were utilized to gauge the robustness of statistically significant and non-significant outcomes, respectively. The FI represents the minimal number of patient outcomes that would need to alter to overturn a trial's statistical significance, while RFI indicates the minimal changes required to achieve significance in non-significant results. Out of 4028 studies retrieved, 21 RCTs were included for final analysis. The studies spanned 2001 to 2023 with a mean followup of 66 months (range, 29-108 months) and median number of patients of 194 (range, 45-802). The most common treatment modalities examined in these studies were neoadjuvant chemoradiotherapy (n = 7, 33.3%), neoadjuvant chemotherapy (n = 4, 19.0%), and neoadjuvant immunotherapy (n = 2, 9.5%). Only 5 studies (23.8%) had a statistically significant primary outcome result with a median FI of 6 (IQR, 2.5-8.5). Non-significant primary outcomes were seen in 16 studies (76.2%) with a median RFI of 4 (IQR 1-11) and lost to followup of 0 (IQR 0-4). In the study with the highest FI (10), the FI was lower than the number of patients lost to followup (13). Our findings demonstrate that most RCTs on esophageal cancer treatments did not report significant primary outcomes. The few studies that reported significant results had a low fragility index, suggesting a vulnerability in their findings.

The Statistical Fragility of Outcomes on Breast Reconstruction with Acellular Dermal Matrix: A Systematic Review of Randomized Controlled Trials.

Acellular dermal matrix (ADM) is pivotal in breast surgery, yet the statistical robustness of surgical outcomes remains underexplored. This study employs the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to investigate the statistical fragility of ADM breast reconstruction outcomes. Randomized controlled trials (RCTs) (2013-present) with dichotomous outcomes were sourced from PubMed, Embase, SCOPUS, Medline, and Cochrane databases. FI and rFI-event reversals needed to alter outcome significance-and FQ-standardized fragility across trials-were computed and reported as median (IQR). Subgroup analysis focused on intervention types. Out of 33 studies screened, 19 RCTs comprising 204 outcomes were included, with a median FI of 4 (3 - 5) and FQ of 0.039 (0.029 - 0.070). Twenty-six outcomes achieved statistical significance, with a median FI of 3.5 (1 - 5) and FQ of 0.033 (0.010 - 0.073). The remaining 178 outcomes were nonsignificant, exhibiting a median FI of 4 (3 - 5) and FQ of 0.040 (0.030 - 0.070). Of the 204 outcomes, 18% had a number of patients lost to follow up equal to or surpassing the FI. By intervention type, the median FIs were similar in value but remained low. ADM-related breast reconstruction outcomes are statistically fragile, so reversal of a few outcomes or maintaining follow-up with patients may alter the significance of findings. Future researchers are thus recommended to report FI and FQ metrics with P-values to accurately portray reconstructive surgery outcomes.

Clinical benefit and fragility evaluation of systemic therapy trials for advanced soft tissue sarcoma.

The clinical benefit of systemic anticancer therapies can be unclear despite positive trials, and outcomes may not translate to real-world practice. This study evaluated the benefit of soft tissue sarcoma (STS) treatments using the European Society of Medical Oncology Magnitude of Clinical Benefit Scale (MCBS) v1.1 and measured the robustness of STS trial results using Fragility Index (FI). Database searches for adult phase II or III trials in advanced STS (January 1998-December 2023) were performed. Therapies with trial outcomes that met the criteria for MCBS were scored 1-5 (≥4 represents substantial clinical benefit). For randomized clinical trials with positive time-to-event endpoints, the number of additional events that would render results nonsignificant, FI, was calculated and expressed as a proportion of the experimental arm size (fragility quotient [FQ]). Higher FI/FQ implies more robust results. Among 194 trials, 19 (9.8%) were phase III. Most phase II trials (146/175; 83.4%) had single-arm or non-comparative design. Trials that were eligible for MCBS scoring (n=78; 40.2%) evaluated 56 different agents/regimens. Median MCBS score was 2. Only three agents/regimens (all cytotoxic therapies) had an MCBS score ≥4. Among 47 randomized clinical trials, 16 (8 phase II; 8 phase III) trials had positive outcomes. Median FI was 7 (range, 2-52) and 10 trials (62.5%) had an FQ < 10%, with median of 7% (range, 1%-59%). Most systemic therapies in STS trials did not confer substantial clinical benefit per European Society of Medical Oncology-MCBS. Additionally, positive randomized trials were often fragile. Novel STS therapy trials should use clinically meaningful endpoints and real-world efficacy confirmation is essential, especially for less robust trials.

The Fragility Index of Risk Factors for Hamstring Injuries.

OBJECTIVE: To determine the Fragility Index of hamstring injury risk factors, defined as the minimum number of participants who would need to change classification to make a hamstring injury risk factor statistically nonsignificant. DESIGN: Retrospective secondary data analysis. METHODS: Studies that investigated 1 or more risk factors for hamstring injury, and presented sufficient data to develop a 2 × 2 contingency table were included. A systematic literature search and reference screening of a recent hamstring injury systematic review were conducted to identify 78 articles. Relative risk and 95% confidence intervals were determined and then systematically recalculated by removing 1 observation from the high-risk injury count and adding it to the high-risk noninjury count. The Fragility Index for a risk factor was the number of observations required to be moved between groups until the relative risk was no longer significant. RESULTS: The median Fragility Index of all hamstring injury risk factors was 3 (Q1-Q3 = 2-6). The Fragility Index for nonmodifiable risk factors was 3 (Q1-Q3 = 2-6) and 3 (Q1-Q3 = 2-5) for modifiable risk factors. Over 35% of all included hamstring injury risk factors had a Fragility Index of ≤2. CONCLUSION: Most statistically significant hamstring injury risk factors are fragile associations. The interpretation of significant hamstring injury risk factors should consider a range of statistical metrics, and while the Fragility Index should never be considered in isolation, it is an intuitive measure to help assess the robustness of findings. J Orthop Sports Phys Ther 2024;54(10):672-678. Epub 4 September 2024. doi:10.2519/jospt.2024.12300.

How fragile the positive results of Chinese herbal medicine randomized controlled trials on irritable bowel syndrome are?

ObjectiveThe fragility index (FI), which is the minimum number of changes in status from “event” to “non-event” resulting in a loss of statistical significance, serves as a significant supplementary indicator for clinical physicians in interpreting clinical trial results and aids in understanding the outcomes of randomized controlled trials (RCTs). In this systematic literature survey, we evaluated the FI for RCTs evaluating Chinese herbal medicine (CHM) for irritable bowel syndrome (IBS), and explored potential associations between study characteristics and the robustness of RCTs.MethodsA comprehensive search was conducted in four databases in Chinese and four databases in English from their inception to January 1, 2023. RCTs encompassed 1:1 ratio into two parallel arms and reported at least one binary outcome that demonstrated statistical significance were included. FI was calculated by the iterative reduction of a target outcome event in the treatment group and concomitant subtraction of a non-target event from that group, until positive significance (defined as P < 0.05 by Fisher’s exact test) is lost. The lower the FI (minimum 1) of a trial outcome, the more fragile the positive result of the outcome was. Linear regression models were adopted to explore influence factors of the value of FI.ResultsA total of 30 trials from 2 4118 potentially relevant citations were finally included. The median FI of total trials included was 1.5 (interquartile range [IQR], 1–5), and half of the trials (n = 15) had a FI equal to 1. In 12 trials (40%), the total number of participants lost to follow-up surpassed the respective FI. The study also identified that increased FI was significantly associated with no TCM syndrome differentiation for inclusion criteria of the patients, larger total sample size, low risk of bias, and larger numbers of events.ConclusionsThe majority of CHM IBS RCTs with positive results were found to be fragile. Ensuring adequate sample size, scientifically rigorous study design, proper control of confounding factors, and a quality control calibration for consistency of TCM diagnostic results among clinicians should be addressed to increase the robustness of the RCTs. We recommend reporting the FI as one of the components of sensitivity analysis in future RCTs to facilitate the assessment of the fragility of trials.

The Statistical Fragility of Lateral Extra-articular Tenodesis Research: A Systematic Review.

A P value of <.05 is often used to denote statistical significance; however, in many scenarios, this threshold is vulnerable to a small number of outcome reversals. This study joins a body of studies within the orthopaedic literature that evaluate the statistical fragility of existing research via metrics such as fragility index (FI) and fragility quotient (FQ). The purpose of this study was to investigate the statistical fragility of randomized controlled trials (RCTs) and comparative studies on the topic, given the resurgent interest in lateral extra-articular tenodesis (LET) to augment primary or revision anterior cruciate ligament reconstruction (ACLR). It was hypothesized that the outcomes reported in these studies would be statistically fragile. Systematic review; Level of evidence, 4. Comparative studies and RCTs regarding LET as an adjunct procedure to ACLR published between 2000 and 2022 were analyzed. Descriptive characteristics, dichotomous outcomes, and continuous outcomes were extracted. The FI and continuous FI (CFI) were calculated by the number of event reversals to change significance; the FQ and continuous FQ (CFQ) were calculated to normalize the fragility metrics per sample size. Of 455 studies screened, 29 studies were included (9 RCTs, 20 comparative); 79.3% of included studies were published after 2020. A total of 48 dichotomous and 265 continuous outcomes were analyzed. The median FI was 9.0 (IQR, 7.0-13.3), with FQ of 0.1 (IQR, 0.04-0.17); the median CFI was 7.8 (IQR, 4.2-19.6), with CFQ of 0.12 (IQR, 0.08-0.19). The FQ and CFQ for studies on LET with revision ACLR were larger (0.117 and 0.113, respectively) than those focused on primary ACLR (0.042 and 0.095, respectively). Studies focused on LET with primary ACLR were more fragile than those on LET with revision, which suggests that further research on the indications for LET with primary ACLR is necessary. Future orthopaedic comparative research should include fragility metrics alongside traditional P values.

Exploring the fragility of meta-analyses in ophthalmology: a systematic review.

The fragility index (FI) of a meta-analysis evaluates the extent that the statistical significance can be changed by modifying the event status of individuals from included trials. Understanding the FI improves the interpretation of the results of meta-analyses and can help to inform changes to clinical practice. This review determined the fragility of ophthalmology-related meta-analyses. Meta-analyses of randomized controlled trials with binary outcomes published in a journal classified as 'Ophthalmology' according to the Journal Citation Report or an Ophthalmology-related Cochrane Review were included. An iterative process determined the FI of each meta-analysis. Multivariable linear regression modeling evaluated the relationship between the FI and potential predictive factors in statistically significant and non-significant meta-analyses. 175 meta-analyses were included. The median FI was 6 (Q1-Q3: 3-12). This meant that moving 6 outcomes from one group to another would reverse the study's findings. The FI was 1 for 18 (10.2%) of the included meta-analyses and was ≤5 for 75 (42.4%) of the included meta-analyses. The number of events (p < 0.001) and the p-value (p < 0.001) were the best predictors of the FI in both significant and non-significant meta-analyses. The statistical significance of meta-analyses in ophthalmology often hinges on the outcome of a few patients. The number of events and the p-value are the most important factors in determining the fragility of the evidence. The FI is an easily interpretable measure that can supplement the reader's understanding of the strength of the evidence being presented. CRD42022377589.

Fragility analysis and systematic review of patellar resurfacing versus non-patellar resurfacing in total knee arthroplasty.

Fragility analysis is a method of further characterising the robustness of statistical outcomes. This study evaluates the statistical fragility of randomised controlled trials (RCTs) comparing patellar resurfacing versus non-patellar surfacing in total knee arthroplasty (TKA). PubMed, MEDLINE and EMBASE were searched for RCTs comparing outcomes in TKA based on patellar resurfacing. Fragility index (FI) and reverse FI (collectively, "FI") were calculated for dichotomous outcomes as the number of outcome reversals needed to change statistical significance. Fragility quotient (FQ) was calculated by dividing the FI by the sample size for that outcome. Median FI and FQ were calculated for each individual outcome and for the overall study. Subanalyses were performed to assess FI and FQ based on outcome type, statistical significance and loss to follow-up. Twenty-one RCTs were included in the analysis, capturing 3910 subjects. The overall median FI was 5.0 (interquartile range, [IQR] 4.0-6.0), and the overall median FQ was 0.048 (IQR 0.022-0.065). The outcome of anterior knee pain has a median FI of 6.0 (IQR 4.0-6.0) and a median FQ of 0.057 (IQR 0.025-0.065). Only five (7%) outcomes were significant. The loss to follow-up was greater than the FI in 12 of 19 studies (63%) with available data. RCTs comparing patellar resurfacing in TKAs show significant statistical fragility; a few outcome reversals can alter findings. The majority of outcomes were nonsignificant, indicating that the choice to resurface the patella may not affect most clinical outcomes; however, clinical conclusions are limited by the statistical fragility of the analysed outcomes. Larger RCTs for this comparison are necessary, and we suggest adding FI and FQ to RCT reports with p values to improve the interpretability of results. Level II.

Poster 346: The Statistical Fragility of Lateral Extra-articular Tenodesis Research: A Systematic Review

Objectives: With reinterest in lateral extra-articular tenodesis (LET), there is an imperative need to evaluate the rigor of clinical studies regarding LET with anterior cruciate ligament reconstruction (ACLR). This study aims to present a comprehensive picture of the robustness of evidence from comparative studies regarding LET and to inform evidence-based medical decision-making for current practitioners. We hypothesized that the results of these analyses will show statistical fragility, consistent with similar evidence across the orthopaedic literature. Methods: Using the PubMed database, comparative studies and randomized controlled trials (RCTs) related to LET as an augmentation procedure to ACLR or revision ACLR published between 2000 and 2022 were identified and collected. Studies on the topic were broadly queried for relevance, and after screening, they were included if they were (1) pertaining to LET as an augmentation to ACLR or revision ACLR, and (2) designed as comparative studies or RCTs. Exclusion criteria included (1) cadaveric, nonhuman, in vitro, laboratory, or surgical technique (without patient outcomes); (2) commentary, editorial, letter to the editor, conference reports, future study design/published protocol; (3) only abstract available; (4) non-English; or (5) lacked the statistical basis for a fragility analysis. Manuscripts were independently reviewed by three authors and variables of interest of interest were extracted, including both dichotomous and continuous variables relevant to clinical decision making. Discrepancies were resolved via paired discussions. For each dichotomous outcome, we calculated a Fragility Index (FI). FI was calculated using a 2-by-2 contingency table and the Fisher's exact test using the method outlined in Figure 1. For each continuous outcome, we calculated a Continuous Fragility Index (CFI). CFI was calculated using Welch's t-test and the method proposed by Caldwell et al to expand fragility analysis to continuous variables. The analysis for each outcome was conducted with n = 5 simulations using synthetic, representative data generated from the reported sample mean, standard deviation, and sample size for both the experimental and control arms. For both dichotomous and continuous outcomes, the statistical fragility was reported using median and interquartile range. Comparisons of mean statistical fragility were conducted using a nonparametric t test. Data were analyzed using Python 3.7. Results: Out of the 455 initially identified studies, 178 full texts were screened. Ultimately, 29 studies were included in the final analysis, including 20 comparative studies and 9 RCTs. 18 studies reported dichotomous outcomes and 27 studies reported continuous outcomes, which resulted in a cumulative total of 48 dichotomous outcomes and 265 continuous outcomes for analysis. Figure 4 reports the distribution of FI and CFI values for all dichotomous and continuous outcomes. The median FI was 9.0 (25-75th percentile, 7.0-13.25). The median CFI was 7.8 (25-75th percentile, 4.2-19.6). Commonly reported dichotomous outcomes included clinical failure, graft rupture, and return to sport. The number of outcomes in which loss to follow-up (LTF) exceeded the FI was 13 (27%). Reported significant dichotomous outcomes were significantly more fragile than outcomes that were not significant (fragility quotient [FQ] = 0.02 vs 0.11, Welch t test p &lt; 0.001); however, most dichotomous outcomes were reported insignificant (79.2%) (Table 1). Commonly reported continuous outcomes across different studies included KOOS, IKDC, and Lysholm scores. LTF exceeded the CFI for 20 outcomes (8%). In contrast to dichotomous outcomes, continuous outcomes from RCTs and comparative studies were equivalently robust (continuous fragility quotient [CFQ] = 0.12 vs 0.12). The CFQ of both significant and nonsignificant continuous outcomes were similarly equal (0.12 vs 0.12) (Table 2). Reports on fragility quotients from studies that focused on LET for either primary or revision ACLR clinically are presented in Table 3. These subgroups excluded translational research and imaging studies. The FQ and CFQ for studies focused on revision ACLR were larger (0.117 and 0.113) than those focused on primary ACLR (0.042 and 0.095). Conclusions: Research regarding the usage of LET as an augmentation procedure for ACLR and revision ACLR is more statistically robust than many other topics within orthopaedics. Studies regarding LET with primary ACLR are more fragile than those for revision ACLR. Given the resurgent interest in this procedure and the current mixed evidence regarding its effectiveness and indications, we recommend the future reporting of fragility quotients alongside p-values, to assist clinicians in assessing the robustness of new evidence to inform decision making.

Fragility index analysis of systemic therapy clinical trials in soft tissue sarcoma (STS).

11581 Background: The fragility index (FI) measures the robustness of randomized clinical trials (RCT) that have positive results based on statistical p-values. It estimates the minimum number of events that are needed to reverse the trial results from positive to negative. Here, we perform FI analysis on RCTs evaluating systemic treatments for STS. Methods: A systematic search of the Medline and Embase databases for RCTs in adults with advanced STS (Jan 1998 to Dec 2023) was conducted. Gastrointestinal stromal tumor trials were excluded. The FI framework was adapted to allow the use of time-to-event (TTE) outcomes – progression-free survival (PFS) or overall survival (OS) as previously described (Desnoyers et al 2021). Survival tables were reconstructed from published data on positive TTE outcomes, using the Parmar Toolkit. Positive secondary endpoints were used only if the primary endpoint was negative. The number of additional events that would result in a non-significant effect for the hazard ratio (HR) of the positive endpoint of each trial (FI) was calculated and expressed as a proportion of the size of the experimental arm (fragility quotient; FQ). The number of censored patients – those who withdrew consent or were lost to follow-up (Cn) was noted. Results: Among 47 RCTs, 16 (8 phase II; 8 phase III) trials had positive outcomes. The primary endpoint was positive and used for FI analysis in 11/16 trials (68.8%). PFS was the most common positive outcome, evaluated in 13 trials (81.3%). The median FI was 6 (range 2 – 52), with FI &lt; 10 observed in 11 trials (68.8%). Median FQ was 7% (range 1 – 59%) and 10 trials (62.5%) had FQ &lt; 10%. Among 14 trials that reported data, Cn was ≥ FI in 7 trials (50%). Only 2 of 4 trials (50%) that led to regulatory approval had FI &gt; 10 or FQ &gt; 10%. Of the other 2, one drug approval was subsequently withdrawn (Table). Conclusions: Most positive clinical trials in STS were fragile and their outcomes may be confounded by the level of censoring. Real-world evaluation of approved systemic therapies and value scales such as ESMO Magnitude of Clinical Benefit scale or ASCO value framework are needed to confirm clinical benefit of systemic treatments in STS.[Table: see text]

Statistical fragility of findings from randomized phase 3 trials in pediatric oncology.

e23003 Background: The fragility index (FI) and survival-inferred fragility index (SIFI) have been proposed as adjunctive metrics to facilitate interpretation of p-values in clinical trials. FI/SIFI is the number of patients on a positive (p &lt; 0.05) trial’s experimental arm who would need to switch from “responder/non-event” to “non-responder/event” to cause p &gt; 0.05. Fragility indices have been reported previously in medical oncology, with median values &lt; 10 often seen in the literature. This metric has not been studied in pediatric oncology trials. Methods: PubMed was used to identify randomized phase 3 pediatric oncology trials published between 1980-2020. We report trial characteristics and calculate FI for trials with a binary outcome and SIFI for trials with a time-to-event outcome. We also report the fragility quotients (FQ and SFQ) to normalize FI and SIFI relative to trial size. Results: 113 trials included sufficient data for analysis. The median FI for trials with a binary outcome (n = 40) was 4.5 (range: 1-33). The median SIFI for trials with a time-to-event outcome (n = 73) was 13 (range: 0-61). The FI or SIFI was less than the number of patients lost to follow-up in 25% of 36 trials. Median FQ = 0.026 and SFQ = 0.03, indicating that revised outcome in 2.6% or 3% of trial participants would be sufficient to alter the trial conclusion. FQ and SFQ did not significantly vary according to trial characteristics. While sample sizes increased over time (mean 187, 1980s; mean 437, 2010s), FQ and SFQ remained stable between 2-3%. Conclusions: The statistical conclusions of pediatric oncology phase 3 trials hinge on a relatively small number and proportion of patients. Despite sample size limitations of low prevalence diseases, pediatric cancer trials appear similarly or less fragile than adult oncology trials. Fragility was similar for large versus small trials and remained constant over four decades. [Table: see text]

The fragility index and reverse fragility index of FDA investigational device exemption trials in spinal fusion surgery: a systematic review.

FDA investigational device exemption (IDE) studies are considered a gold standard of assessing safety and efficacy of novel devices through RCTs. The fragility index (FI) has emerged as a means to assess robustness of statistically significant study results and inversely, the reverse fragility index (RFI) for non-significant differences. Previous authors have defined results as fragile if loss to follow up is greater than the FI or RFI. The aim of this study was to assess the FI, RFI, and robustness of data supplied by IDE studies in spinal surgery. This was a systematic review of the literature. Inclusion criteria included randomized controlled trials with dichotomous outcome measures conducted under IDE guidelines between 2000 and 2023. FI and RFI were calculated through successively changing events to non-events until the outcome changed to non-significance or significance, respectively. The fragility quotient (FQ) and reverse fragility quotient (RFQ) were calculated by dividing the FI and RFI, respectively, by the sample size. Thirty-two studies met inclusion criteria with a total of 40 unique outcome measures; 240 outcomes were analyzed. Twenty-six studies reported 96 statistically significant results. The median FI was 6 (IQR: 3-9.25), and patients lost to follow up was greater than the FI in 99.0% (95/96) of results. The average FQ was 0.027. Thirty studies reported 144 statistically insignificant results and a median RFI of 6 (IQR: 4-8). The average RFQ extrapolated was 0.021, and loss to follow up was greater than the RFI in 98.6% (142/144) of results. IDE studies in spine surgery are surprisingly fragile given their reputations, large sample sizes, and intent to establish safety in investigational devices. This study found a median FI and RFI of 6. The number of patients lost to follow-up was greater than FIand RFI in 98.8% (237/240) of reported outcomes. FQ and RFQ tell us that changes of two to three patients per hundred can flip the significance of reported outcomes. This is an important reminder of the limitations of RCTs. Analysis of fragility in future studies may help clarify the strength of the relationship between reported data and their conclusions.

P.124 Assessing the fragility index of randomized controlled trials on carotid artery stenosis: systematic review

Background: The fragility index (FI) is the minimum number of patients whose status would have to change from a nonevent to an event to turn a statistically significant result to a non-significant result. We used this to measure the robustness of trials comparing carotid endarterectomy (CEA) to carotid artery stenting (CAS). Methods: A search was conducted in MEDLINE, Embase, and PubMed on RCTs comparing CEA to CAS. The trials need to have statistically significant results and dichotomous primary endpoints to be included. Results: Our literature search identified 10 RCTs which included 9382 patients (4734 CEA, 4648 CAS). The primary end points of all included trials favoured CEA over CAS. The median FI was 9.5 (interquartile range 2.25 - 21.25). All of the studies that reported lost-to-follow-up (LTFU) had LTFU greater than its fragility index, which raises concern that the missing data could change the results of the trial from statistically significant to statistically insignificant. Conclusions: A small number of events (FI, median 9.5) were required to render the results of carotid artery stenosis RCTs comparing CEA to CAS statistically insignificant. All of the studies that reported LTFU had LTFU greater than its fragility index.

The Fragility of Statistical Findings in the Femoral Neck Fracture Literature: A Systematic Review of Randomized Controlled Trials.

Randomized controlled trials (RCTs) in the femoral neck fracture literature frequently report P -values for outcomes, which have substantial implications in guiding surgical management. This study used the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to assess the statistical stability of outcomes reported in RCTs evaluating the management and treatment of femoral neck fractures. PubMed, Embase, and MEDLINE were queried for RCTs (January 1, 2010 to February 28, 2023). RCTs that evaluated surgical management or treatment of femoral neck fractures were included. RCTs with 2 treatment arms reporting categorical dichotomous outcomes were included. Non-RCT studies, RCTs with greater than 2 treatment arms, and RCTs without a femoral neck fracture cohort were excluded. The FI and rFI were calculated as the number of outcome event reversals required to alter statistical significance for significant ( P < 0.05) and nonsignificant ( P ≥ 0.05) outcomes, respectively. The FQ was calculated by dividing the FI by the sample size for the study. Nine hundred eighty-five articles were screened, with 71 studies included for analysis. The median FI across a total of 197 outcomes was 4 [interquartile range (IQR) 2-5] with an associated FQ of 0.033 (IQR 0.017-0.060). Forty-seven outcomes were statistically significant with a median FI of 2 (IQR 1-4) and associated FQ of 0.02 (IQR 0.014-0.043). One hundred fifty outcomes were statistically nonsignificant with a median rFI of 4 (IQR 3-5) and associated FQ of 0.037 (IQR 0.019-0.065). Statistical findings in femoral neck fracture RCTs are fragile, with reversal of a median 4 outcomes altering significance of study findings. The authors thus recommend standardized reporting of P -values with FI and FQ metrics to aid in interpreting the robustness of outcomes in femoral neck fracture RCTs. Therapeutic Level III. See Instructions for Authors for a complete description of levels of evidence.

Fragility index analysis for randomized controlled trials of approved biologicals and small molecule drugs in inflammatory bowel diseases

IntroductionBiologics and small molecules have been increasingly applied in Crohn’s disease (CD) and ulcerative colitis (UC). But the robustness of their trials has not been evaluated. MethodsWe initially collected all the approved biologics or small molecules for CD or UC up to December 1, 2022. Databases were then queried by keywords in chemical name and CD or UC. Randomized controlled trials (RCTs) in the two-arm, 1:1 design were included. Fragility index (FI) and fragility quotient (FQ) were subsequently calculated. ResultsWe included twenty-eight RCTs, including nine pivotal trials listed in approval labels, nineteen non-pivotal trials not included in the labels. The median sample size was 99 [IQR, 60–262] and the median number of loss-of-follow-up (LFU) was 14 [IQR, 8–43]. Pivotal trials in the labels had the median FI of 8 [IQR, 4–14, n = 6] that was marginally higher than non-pivotal trials (3 [IQR, 2–4], p = 0.08). The median FQ was 0.0330 [IQR, 0.1220–0.0466] and 0.0310 [IQR, 0.0129–0.0540] for pivotal and non-pivotal trials, respectively (p = 1.0). The sample size and FI were significantly correlated (Spearman correlation coefficient [r] = 0.56, 95 %CI 0.21–0.78, p = 0.003). The number of total events was also significantly correlated with FI (r = 0.53, 95 %CI 0.17–0.77, p = 0.006). Study p-values were significantly associated with FI (p = 0.01): trials with p-values < 0.001 had the highest median FI of 10 [IQR, 6–17]. No factor was found strongly correlated with FQ. ConclusionResults from trials assessing administration-approved biologics or small molecules for treating CD or UC were vulnerable to small changes by measuring FI or FQ. Pivotal studies contributing to regulatory approvals exhibited a relatively higher degree of resilience compared to non-pivotal trials.

The Statistical Fragility of Marrow Stimulation for Cartilage Defects of the Knee: A Systematic Review of Randomized Controlled Trials.

Marrow stimulation is used to address knee cartilage defects. In this study, we used the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to evaluate statistical fragility of outcomes reported in randomized controlled trials (RCTs) evaluating marrow stimulation. PubMed, Embase, and MEDLINE were queried for recent RCTs (January 1, 2010-September 5, 2023) assessing marrow stimulation for cartilage defects of the knee. The FI and rFI were calculated as the number of outcome event reversals required to alter statistical significance for significant and nonsignificant outcomes, respectively. The FQ was determined by dividing the FI by the study sample size. Across 155 total outcomes from 21 RCTs, the median FI was 3 (interquartile range [IQR], 2-5), with an associated median FQ of 0.067 (IQR, 0.033-0.010). Thirty-two outcomes were statistically significant, with a median FI of 2 (IQR, 1-3.25) and FQ of 0.050 (IQR, 0.025-0.069). Ten of the 32 (31.3%) outcomes reported as statistically significant had an FI of 1. In total, 123 outcomes were nonsignificant, with a median rFI of 3 (IQR, 2-5). Studies assessing stem cell augments were the most fragile, with a median FI of 2. In 55.5% of outcomes, the number of patients lost to follow-up was greater than or equal to the FI. Statistical findings in RCTs evaluating marrow stimulation for cartilage defects of the knee are statistically fragile. We recommend combined reporting of P-values with FI and FQ metrics to aid in the interpretation of clinical findings in comparative trials assessing cartilage restoration.

Statistical fragility of outcomes in acellular dermal matrix literature: A systematic review of randomized controlled trials

BackgroundAcellular dermal matrix (ADM) is commonly utilized in plastic and reconstructive surgery (PRS). With the abundance of randomized controlled trials (RCTs) reporting P-values for ADM outcomes, this study uses the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to evaluate the statistical stability of outcomes in ADM RCTs. MethodsPubMed, Embase, SCOPUS, Medline, and Cochrane databases were reviewed for ADM RCTs (2003-present) reporting a dichotomous, categorical outcome. FI and rFI (event reversals influencing outcome significance) and FQ (standardized fragility) were calculated and reported as median (IQR). Subgroup analysis was performed based on intervention types. ResultsOut of 127 studies screened, 56 RCTs with 579 outcomes were included. The median FI stood at 4 (3 - 5) and FQ was 0.04 (0.03 - 0.07). Only 101 outcomes were statistically significant with a median FI of 3 (1 - 6) and FQ of 0.04 (0.02 - 0.08). The nonsignificant outcomes had a median FI of 4 (3 - 5) and FQ of 0.04 (0.03 - 0.07). Notably, 26% of outcomes had a number of patients lost to follow up equal to or surpassing the FI. By intervention type, the median FIs showed minor fluctuations but remained low. ConclusionsOutcomes from ADM-related RCTs are statistically fragile. Slight outcome reversals or maintaining patient follow-up can alter the significance of results. Therefore, future researchers are recommended to jointly report FI, FQ, and P-values to offer a comprehensive view of the robustness in ADM literature.

Interpreting the Current Literature on Outcomes of Robotic-Assisted Versus Conventional Total Knee Arthroplasty Using Fragility Analysis: A Systematic Review and Cross-Sectional Study of Randomized Controlled Trials

BackgroundFragility analysis is a method of further characterizing outcomes in terms of the stability of statistical findings. This study assesses the statistical fragility of recent randomized controlled trials (RCTs) evaluating robotic-assisted versus conventional total knee arthroplasty (RA-TKA versus C-TKA). MethodsWe queried PubMed for RCTs comparing alignment, function, and outcomes between RA-TKA and C-TKA. Fragility index (FI) and reverse fragility index (RFI) (collectively, “FI”) were calculated for dichotomous outcomes as the number of outcome reversals needed to change statistical significance. Fragility quotient (FQ) was calculated by dividing the FI by the sample size for that outcome event. Median FI and FQ were calculated for all outcomes collectively as well as for each individual outcome. Subanalyses were performed to assess FI and FQ based on outcome event type and statistical significance, as well as study loss to follow-up and year of publication. ResultsThe overall median FI was 3.0 (interquartile range, [IQR] 1.0 to 6.3) and the median reverse fragility index was 3.0 (IQR 2.0 to 4.0). The overall median FQ was 0.027 (IQR 0.012 to 0.050). Loss to follow-up was greater than FI for 23 of the 38 outcomes assessed. ConclusionsA small number of alternative outcomes is often enough to reverse the statistical significance of findings in RCTs evaluating dichotomous outcomes in RA-TKA versus C-TKA. We recommend reporting FI and FQ alongside P values to improve the interpretability of RCT results.

Applying the fragility index to randomized controlled trials evaluating total neoadjuvant therapy for rectal cancer.

32 Background: A relatively novel summary measure that is not commonly reported in randomized controlled trials (RCTs) is the fragility index (FI). FI describes the number of additional events required for an outcome to lose statistical significance. It has been applied to a number of medical subspecialities such as critical care, orthopedic surgery and colorectal surgery. Recently, there has been significant interest in, and adoption of, total neoadjuvant therapy (TNT) for locally advanced rectal cancer (LARC). A number of RCTs have assessed TNT, but the robustness of these practice changing RCTs has never been evaluated. As such, we designed the present study to assess the robustness of the RCTs evaluating TNT for LARC using the FI. Methods: Relevant articles were identified through a recently published review article by Johnson et al. in the Canadian Journal of Surgery, that narratively reviewed all of the previously published RCTs evaluating TNT for LARC. We manually searched Google Scholar and PubMed to identify any other relevant RCTs. Outcomes within these RCTs that were either dichotomous outcomes or time to event outcomes were eligible for inclusion if the reported effect size had an associated p-value of less than 0.05. The main outcome was the FI for each statistically significant outcome. Walsh et al.’s method of calculating FI was utilized. A RCTs results were considered fragile if the FI was less than the loss to follow up for a given outcome. Correlations between FI and research characteristics were assessed using the Spearman’s rank correlation coefficients. Results: Ten RCTs were identified with 25 outcomes having statistically significant differences between groups (p-values &lt; 0.05). Eleven outcomes were time-to-event outcomes, while the remainder were dichotomous outcomes. About half (n=13) were oncologic outcomes (i.e., survival, recurrence), while the rest (n=12) were short- and long-term complications. The median FI was 2 (interquartile range [IQR] 1-16). The number of patients lost to follow-up exceeded the FI in 17 outcomes (68.0%) and thus these results were considered “fragile”. Lower FI was associated with high risk of bias (rho=-0.5594) and higher loss to follow-up (i.e., greater than 5% vs. less than 5%) (rho=-0.4394), while higher FI was associated with large sample size (i.e., greater than 500 patients vs. less than 500 patients) (rho=0.5120). Conclusions: The robustness of outcomes from trials assessing TNT for LARC was found to be questionable. Most of these outcomes were fragile, as determined by the FI. In most cases, two or less additional events would have resulted in a loss of statistical significance of the reported results. Those using the results of these studies, including clinicians and health policy experts, should apply caution when interpreting these types of trials.

The Fragility of Landmark Randomized Controlled Trials in the Plastic Surgery Literature.

Randomized controlled trials (RCTs) are integral to the progress of evidenced-based medicine and help guide changes in the standards of care. Although results are traditionally evaluated according to their corresponding P value, the universal utility of this statistical metric has been called into question. The fragility index (FI) has been developed as an adjunct method to provide additional statistical perspective. In this study, we aimed to determine the fragility of 25 highly cited RCTs in the plastic surgery literature. A PubMed search was used to identify the 25 highest cited RCTs with statistically significant dichotomous outcomes across 24 plastic surgery journals. Article characteristics were extracted, and the FI of each article was calculated. Additionally, Altmetric scores were determined for each study to determine article attention across internet platforms. The median FI score across included studies was 4 (2-7.5, interquartile range). The two highest FI scores were 208 and 58, respectively. Four studies (16%) had scores of 0 or 1. Three studies (12%) had scores of 2. All other studies (72%) had FI scores of 3 or higher. The median Altmetric score was 0 (0-3). The FI can provide additional perspective on the robustness of study results, but like the P value, it should be interpreted in the greater context of other study elements.