Exploring discordance in evidence from meta-analyses and subsequent large-scale randomized controlled trials in perioperative medicine

The Need for Large Clinical Studies in Perioperative Medicine

Poor agreement in significant findings between meta-analyses and subsequent large randomized trials in perioperative medicine

3152 Background: Efficacy endpoints of randomized controlled trials (RCT) are commonly used as the basis of regulatory drug approvals. Recently, promising results in early phase trials have resulted in approval of biomarker-targeted therapies. We examined if early phase trial results were associated with efficacy in subsequent biomarker-enriched RCTs. Methods: All cancer drug RCTs conducted between January 2006 and March 2021 were identified through Clinicaltrials.gov. Trials were eligible if a biomarker was used to select a patient population for treatment with a targeted agent. Associated early phase trials were included if they matched the RCT in treatment setting and patient population. Trials pairs were compared using objective response rate (ORR) and progression-free survival (PFS). We assessed difference in endpoints using summary measures (e.g., average, range). We examined whether early phase trials results were associated with RCT results using logistic regression. Results: The search yielded 2,157 unique phase III RCTs and 27 RCTs met eligibility criteria pairing with associated early phase trials, where 17 RCTs met their primary endpoint. The most common biomarkers were EGFR+ (n = 8), HER2+ (n = 5) and PD-L1 (n = 5). Based on average difference of trial pairs, ORR was similar between trials (1.59%, 95% CI = -2.5-5.6, p = 0.50) and median PFS was slightly higher in early phase trials (1.95 months, 95% CI = 0.91-2.99, p < 0.05). On an individual pair basis, there was large range of variability in the difference between early phase trials and RCTs for ORR (range = -23.9-20.2%) and median PFS (range = -0.8-7.4 months). The probability of the RCT meeting its primary endpoint is 50% or 95%, when the early phase trial ORR is 41.2% (95% CI = 35.2-47.1%) or 77.7% (95% CI = 71.7-83.6%), respectively. Conclusions: Through comparison of early phase trials and subsequent phase III RCT, we found that, overall, ORR has minimal bias in early phase trials, and median PFS appears to be slightly overestimated. Substantial variability in results for trial pairs suggests that, on an individual basis, results in early phase trial can be inconsistent with results in subsequent RCT. Early phase trial results may be associated with RCTs meeting their primary endpoint when ORR is very high; however, caution must be exercised when using early phase trials as representative of RCTs for decision-making as the predictive ability of early phase trials is limited.

Trial sequential analysis: adding a new dimension to meta-analysis.

Many scientific articles are written merely to get something published, neglecting the clinician who would like the medical literature to guide their practice. Evidence-based medicine is expected to help in clinical decision-making. Systematic reviews of the literature followed by a meta-analysis of randomized, controlled trials (RCT) have claimed to represent the highest strength of evidence. However, the results published in meta-analyses have not always been confirmed in subsequent large RCTs. An analysis of 12 large RCTs and 19 meta-analyses addressing the same questions found that the outcomes of these large RCTs were not predicted accurately 35% of the time by previously published meta-analyses. Therefore, meta-analyses of several small RCTs do not obviate the need for large, multicentre RCTs, which can still be considered as a gold standard for the development of clinical guidelines or practice plans. Moreover, large RCTs using a factorial design can be highly efficient because they can answer several clinical questions at the same time and offer the only systematic approach to investigate an interaction of combinations in multimodal approaches.

Acute metabolic acidosis is frequent in critical illness and may be managed with volume replacement therapy, sodium bicarbonate, and renal replacement therapy (RRT). BICARICU-1 and BICARICU-2 trials provided new evidence on bicarbonate therapy. We aimed to systematically evaluate the effects of sodium bicarbonate in critically ill patients. We systematically searched PubMed, Embase, Medline, and the Cochrane Library for randomized controlled trials (RCTs) evaluating sodium bicarbonate vs. placebo in critically ill adults with acute metabolic acidosis. Only RCTs were selected. Primary outcome was RRT; secondary outcomes were mortality, ICU length of stay, ventilator-free days, and vasopressor-free days. We performed Hartung-Knapp random-effects meta-analyses and trial sequential analysis (TSA) for primary outcomes. Bayesian random-effects meta-analyses with noninformative and weakly informative priors were used for sensitivity analysis. Four trials enrolling 1,111 patients were included. Sodium bicarbonate showed a nonsignificant trend toward lower mortality (risk ratio [RR], 0.84; 95% CI, 0.55-1.30); TSA indicated that the cumulative sample size remained far below the required information size, and the mortality effect is inconclusive. Sodium bicarbonate significantly reduced RRT use (RR, 0.69; 95% CI, 0.61-0.78), and TSA supported firm evidence of benefit. Bayesian analyses estimated posterior probabilities of any mortality reduction (RR < 1) and reduced RRT use of 90.4% and 94.6%, respectively. Secondary outcomes remained imprecise, with a trend of toward more vasopressor-free days. In critically ill adults with acute metabolic acidosis, sodium bicarbonate significantly reduces RRT requirement, with Bayesian analyses suggesting a possibility of mortality benefit, although current evidence remains inconclusive. Larger RCTs or high-quality real-world studies are needed to confirm efficacy of sodium bicarbonate for mortality.

BackgroundResearch waste can occur when trials are conducted in the wrong populations. Vitamin D deficient populations are most likely to benefit from vitamin D supplementation. We investigated waste attributable to randomised controlled trials (RCTs) of supplementation in populations that were not vitamin D deficient.MethodsIn December 2015, we searched Pubmed, recent systematic reviews, and three trial registries for RCTs of vitamin D with clinical endpoints in adults, and 25-hydroxvitamin D (25OHD) survey data relevant to large (N ≥ 1000) RCTs. We investigated the proportion of RCTs that studied vitamin D deficient populations, temporal trends in baseline 25OHD, and whether investigators in large RCTs considered relevant 25OHD survey data or systematic reviews in their trial justifications.ResultsOf 137 RCTs of vitamin D with clinical endpoints, 118 (86%) reported baseline mean/median 25OHD, which was < 25, 25–49, 50–74, and ≥ 75 nmol/L in 12 (10%), 62 (53%), 36 (31%), and 8 (7%) RCTs, respectively. In 70% of RCTs, baseline 25OHD was > 40 nmol/L. Baseline 25OHD increased over time. Before 2006, 38%, 62%, 0% and 0% of RCTs had baseline 25OHD < 25, 25–49, 50–74, and ≥ 75 nmol/L respectively; in 2011–15, the respective proportions were 9%, 49%, 37%, and 6%. Of 12 RCTs with baseline 25OHD < 25 nmol/L, 8 had neutral findings. Of 25 large RCTs (18 completed, 7 ongoing), 1 was undertaken in a vitamin D deficient population, 3 in vitamin D insufficient populations, and 17 had, or probably will have, baseline 25OHD > 40 nmol/L. 44% (8/18) of large completed RCTs cited relevant prior population 25OHD data, and only 3/10 (30%) relevant prior systematic reviews.ConclusionsUp to 70% of RCTs of vitamin D with clinical endpoints, 71% of large completed RCTs, and 100% of ongoing large RCTs could be considered research waste because they studied cohorts that were not vitamin D deficient.

Promising early phase trial results of biomarker-targeted therapies have occasionally led to regulatory approval. We examined if early phase trials were predictive of efficacy in randomized controlled trials (RCTs) with matching treatment settings. Cancer drug RCTs conducted between January 2006 and March 2021 were identified through Clinicaltrials.gov. Biomarker-enriched RCTs and associated matching early phase trials were included. Trial pairs were compared using objective response rate (ORR) and progression-free survival (PFS). We examined whether early phase trials results were associated with RCT results using logistic regression. The search yielded 2157 unique RCTs and 27 RCTs pairing with early phase trials were included. Based on average difference of trial pairs, ORR was similar (1.6%; 95% confidence interval (CI) - 2.5 to 5.6, p = 0.50) and median PFS was higher in early phase trials (2.0months; 95% CI 0.9-3.0, p < 0.05). On an individual pair basis, there was large variability in difference for ORR (range - 23.9 to 20.2%) and median PFS (range - 0.8 to 7.4months). The probability of the RCT meeting its primary endpoint is 95% (95% prediction interval (PI) 72.8-99.3%) when the early phase trial ORR is 77.7%. Overall, in early phase trials, ORR has minimal bias and median PFS appears to be slightly overestimated. Substantial variability between trials suggests early phase trial results may be inconsistent with subsequent RCT. Early phase trial results may be associated with RCTs meeting their primary endpoint when ORR is very high; however, caution must be exercised when using early phase trials as representative of RCTs.

Dear Editor, Sepsis represents a global problem with high economic burden for health care systems. Since 2002, the Surviving Sepsis Campaign has recommended early quantitative resuscitation for patients with severe sepsis and septic shock with a recent update [1]. However, the optimal goals for quantitative resuscitation remain uncertain. Lactate clearance, defined by the change of lactate levels between two points in time, as a more rapid and less costly parameter, has the potential to be such a promising goal for quantitative resuscitation. We performed a meta-analysis of randomized controlled trials (RCTs) to evaluate the effect of early lactate clearance-guided therapy on mortality and other outcomes in patients with sepsis. We searched PubMed, Embase, and Cochrane Central Register of Controlled Trials to identify RCTs that evaluated the effect of early lactate clearance-guided therapy on clinical outcomes in adults with sepsis. The search terms used were ‘‘lactate clearance’’, and ‘‘sepsis’’, or ‘‘severe sepsis’’ or ‘‘septic shock’’. We used the Cochrane collaboration tool to assess risk of bias, and the GRADE (Grades of Recommendation, Assessment, Development and Evaluation) approach to evaluate the quality of evidence. The primary outcome was all-cause mortality. Secondary outcomes included length of hospital stay and length of intensive care unit (ICU) stay. We calculated risk ratios (RRs) or mean differences (MDs) and 95 % confidence intervals (CIs) using a randomeffects model. A two-tailed p value less than 0.05 was considered a significant level except for where a certain p value has been given. All statistical analyses were performed using RevMan 5.2 (Nordic Cochrane Centre). We conducted trial sequential analysis (TSA) using a diversityadjusted required information size calculated from an alpha error of 0.05, a beta error of 0.20, a control event proportion obtained from the results of the meta-analysis, and a relative risk reduction of 20 % in all-causes mortality, using standard software TSA version 0.9 Beta (http://www.ctu.dk/tsa). Four RCTs enrolling 547 patients were included in the meta-analysis [2–5]. The main characteristics of the four included RCTs are presented in Table 1 in the Electronic Supplementary Material (ESM). Assessment of the risk of bias is summarized in Table 2 (ESM). Overall, two RCTs were categorized as at lower risk of bias [2, 3], and two as at unclear risk of bias [4, 5]. Data on primary outcome were provided in all four trials (547 patients) [2–5]. Early lactate clearance-guided therapy was associated with a reduction in mortality (RR 0.65, 95 % CI 0.49–0.85, p = 0.002, I = 0 %, Fig. 1). TSA showed that 34.5 % of the required information size of 1586 patients were accrued. The cumulative z curve crossed the conventional boundary for benefit but did not cross the trial sequential monitoring boundary for benefit, showing that currently cumulative evidence is inconclusive, as shown in Fig. 3 (ESM). For secondary outcomes, early lactate clearance-guided therapy had no effect on length of hospital stay (weighted mean difference, WMD –0.13 days, 95 % CI –4.58 to 4.31, three RCTs [2, 3, 5]) and length of ICU stay (WMD –1.54 days, 95 % CI –3.22 to 0.15, four RCTs [2–5]), as shown in Fig. 2 (ESM). The GRADE evidence

When may systematic reviews and meta-analyses be considered reliable?

BackgroundThe optimal tube size for managing traumatic hemothorax, pneumothorax, or hemopneumothorax remains debated. While large-bore chest tubes (LCTs—≥ 28 Ch) are traditionally favored, emerging evidence suggests that small-caliber tubes (SCTs—≤ 14 Ch), such as pigtail catheters and small straight tubes, may offer similar efficacy with fewer complications. This study aimed to evaluate the comparative effectiveness and safety of SCTs versus LCTs from Randomized Controlled Trials (RCTs) in adult trauma patients and to assess the conclusiveness of the current evidence using trial sequential analysis (TSA).MethodsThe study was conducted according to the Cochrane recommendations, searching the PubMed, Scopus, and EMBASE datasets up to 25th March 2025 without language restrictions (PROSPERO ID: CRD420251023165). The primary outcome was treatment failure; secondary outcomes included insertion-related complications, duration of drainage, and length of hospital stay. Random effects models based on restricted maximum likelihood and Hartung-Knapp correction were developed. Sensitivity analysis was conducted to detect sources of heterogeneity. The risk of bias was assessed using the Cochrane RoB 2 tool. TSA was used to evaluate the risk of random error and to determine whether the required information size (RIS) had been reached.ResultsFour RCTs (n = 676 patients) were included. Pooled analysis showed no significant difference in failure rates between SCTs and LCTs (RR 0.95, 95% CI 0.66–1.35, I2 = 0%). No significant differences were observed in complication rates or hospital stay. Duration of tube placement was significantly shorter in the SCT group (MD − 0.49 days, p = 0.02). TSA indicated that the cumulative evidence was underpowered, achieving only 22% of the RIS (3110 patients). The Z-curve did not cross thresholds for benefit, harm, or futility.ConclusionSCTs appear to be as effective and safe as LCTs for selected trauma patients with uncomplicated thoracic injuries. However, due to limited sample size and heterogeneity across trials, current evidence is inconclusive. Larger, high-quality RCTs are warranted to confirm these findings and guide clinical practice.Supplementary InformationThe online version contains supplementary material available at 10.1186/s13017-025-00655-x.

Background To evaluate the association between albumin administration as volume replacement and mortality in adult ARDS patients, we performed this meta-analysis and trial sequential analysis (TSA). Methods We searched databases including PubMed, Science Direct, Scopus, Web of Science databases and Cochrane Central Register of Controlled Trials up to 12 December 2024. We screened trials that included adult ARDS patients and compared albumin with crystalloid. The 28-day mortality served as the primary endpoint, while the oxygenation change, the length of ICU stay and the length of hospital stay were designated as secondary outcomes. To clarify the differing concentrations of albumin, we formed two distinct subgroups: the hyper-oncotic albumin subgroup (≥20%) and the iso-oncotic albumin subgroup (4%∼5%). Statistical synthesis was performed with Cochrane Review Manager 5.4.1, employing random-effects models. To mitigate random errors, TSA was implemented with α = 0.05 and β = 0.20 parameters. Results The analysis incorporated 5 publications: 3 randomized controlled trials (RCTs) and 2 non-randomized studies (NRSs). Overall mortality was lower in the albumin group (33.2%, 97/292) than in the crystalloid group (44.9%, 133/296) (OR = 0.61, 95%CI 0.43–0.85, p = 0.004). RCTs (n = 204) showed no benefit (OR = 0.83, p = 0.54), but NRSs (n = 384) demonstrated reduced mortality (OR = 0.52, p = 0.002). Hyper-oncotic albumin was associated with lower mortality in NRSs (OR = 0.40, p = 0.02) but not in RCTs (OR = 0.74, p = 0.57). Iso-oncotic albumin showed no benefit (OR = 0.88, p = 0.72). Regarding the impact of albumin on oxygenation, significant improvements in oxygenation were observed only on the first (p = 0.05) and second days (p < 0.0001). The TSA indicated a continued need for high-quality RCTs. Conclusions Our analysis suggests that hyper-oncotic albumin may reduce mortality and improve early oxygenation in ARDS patients compared to crystalloids. Larger RCTs are urgently needed to validate these findings and define their potential role in clinical management.

Few randomized controlled trials (RCTs) have compared ticagrelor to clopidogrel after thrombolytic therapy in patients with ST-segment elevation myocardial infarction (STEMI). To assess the quality of the current evidence, a trial sequential analysis (TSA) of all the available RCTs was performed. A literature search through electronic databases for relevant RCTs was completed. Trial sequential boundaries were applied to the meta-analysis to guard against statistical error, calculate the information size (IS), and assess the quality of the currently available evidence. The safety outcome was bleeding at 30-days and the efficacy outcome was major adverse cardiovascular events at 30-days. There were 3 RCTs with a total of 3999 patients were included. For the safety and efficacy outcomes, there was no difference between the ticagrelor and clopidogrel groups (RR 0.94; 95% CI 0.56-1.60, p = 0.83) and (RR 0.87; 95% CI 0.49-1.52, p = 0.62), respectively. The corresponding TSA revealed an IS of 20,928 and 37,266 for safety and efficacy outcomes, respectively. The Z-curves for both outcomes failed to cross the conventional boundary of significance and TSA boundary, indicating no statistical difference between the ticagrelor and clopidogrel group and lack of firm evidence from the currently available RCTs to draw conclusion. Based on the current available RCTs, there is not enough evidence to support or refute better outcomes with ticagrelor in patients with STEMI treated with thrombolytics. Larger RCTs with enough power are needed before firm recommendations can be applied.

BackgroundCOVID-19 is a rapidly spreading disease that has caused extensive burden toindividuals, families, countries, and the world. Effective treatments ofCOVID-19 are urgently needed. This is the second edition of a livingsystematic review of randomized clinical trials assessing the effects of alltreatment interventions for participants in all age groups withCOVID-19.Methods and findingsWe planned to conduct aggregate data meta-analyses, trial sequentialanalyses, network meta-analysis, and individual patient data meta-analyses.Our systematic review was based on PRISMA and Cochrane guidelines, and oureight-step procedure for better validation of clinical significance ofmeta-analysis results. We performed both fixed-effect and random-effectsmeta-analyses. Primary outcomes were all-cause mortality and serious adverseevents. Secondary outcomes were admission to intensive care, mechanicalventilation, renal replacement therapy, quality of life, and non-seriousadverse events. According to the number of outcome comparisons, we adjustedour threshold for significance to p = 0.033. We used GRADEto assess the certainty of evidence. We searched relevant databases andwebsites for published and unpublished trials until November 2, 2020. Tworeviewers independently extracted data and assessed trial methodology. Weincluded 82 randomized clinical trials enrolling a total of 40,249participants. 81 out of 82 trials were at overall high risk of bias.Meta-analyses showed no evidence of a difference between corticosteroidsversus control on all-cause mortality (risk ratio [RR] 0.89; 95% confidenceinterval [CI] 0.79 to 1.00; p = 0.05; I2 =23.1%; eight trials; very low certainty), on serious adverse events (RR0.89; 95% CI 0.80 to 0.99; p = 0.04; I2 = 39.1%;eight trials; very low certainty), and on mechanical ventilation (RR 0.86;95% CI 0.55 to 1.33; p = 0.49; I2 = 55.3%; twotrials; very low certainty). The fixed-effect meta-analyses showedindications of beneficial effects. Trial sequential analyses showed that therequired information size for all three analyses was not reached.Meta-analysis (RR 0.93; 95% CI 0.82 to 1.07; p = 0.31;I2 = 0%; four trials; moderate certainty) and trialsequential analysis (boundary for futility crossed) showed that we couldreject that remdesivir versus control reduced the risk of death by 20%.Meta-analysis (RR 0.82; 95% CI 0.68 to 1.00; p = 0.05;I2 = 38.9%; four trials; very low certainty) and trialsequential analysis (required information size not reached) showed noevidence of difference between remdesivir versus control on serious adverseevents. Fixed-effect meta-analysis showed indications of a beneficial effectof remdesivir on serious adverse events. Meta-analysis (RR 0.40; 95% CI 0.19to 0.87; p = 0.02; I2 = 0%; two trials; very lowcertainty) showed evidence of a beneficial effect of intravenousimmunoglobulin versus control on all-cause mortality, but trial sequentialanalysis (required information size not reached) showed that the result wasseverely underpowered to confirm or reject realistic intervention effects.Meta-analysis (RR 0.63; 95% CI 0.35 to 1.14; p = 0.12;I2 = 77.4%; five trials; very low certainty) and trialsequential analysis (required information size not reached) showed noevidence of a difference between tocilizumab versus control on seriousadverse events. Fixed-effect meta-analysis showed indications of abeneficial effect of tocilizumab on serious adverse events. Meta-analysis(RR 0.70; 95% CI 0.51 to 0.96; p = 0.02; I2 =0%; three trials; very low certainty) showed evidence of a beneficial effectof tocilizumab versus control on mechanical ventilation, but trialsequential analysis (required information size not reached) showed that theresult was severely underpowered to confirm of reject realistic interventioneffects. Meta-analysis (RR 0.32; 95% CI 0.15 to 0.69; p< 0.00; I2 = 0%; two trials; very low certainty) showedevidence of a beneficial effect of bromhexine versus standard care onnon-serious adverse events, but trial sequential analysis (requiredinformation size not reached) showed that the result was severelyunderpowered to confirm or reject realistic intervention effects.Meta-analyses and trial sequential analyses (boundary for futility crossed)showed that we could reject that hydroxychloroquine versus control reducedthe risk of death and serious adverse events by 20%. Meta-analyses and trialsequential analyses (boundary for futility crossed) showed that we couldreject that lopinavir-ritonavir versus control reduced the risk of death,serious adverse events, and mechanical ventilation by 20%. All remainingoutcome comparisons showed that we did not have enough information toconfirm or reject realistic intervention effects. Nine single trials showedstatistically significant results on our outcomes, but were underpowered toconfirm or reject realistic intervention effects. Due to lack of data, itwas not relevant to perform network meta-analysis or possible to performindividual patient data meta-analyses.ConclusionsNo evidence-based treatment for COVID-19 currently exists. Very low certaintyevidence indicates that corticosteroids might reduce the risk of death,serious adverse events, and mechanical ventilation; that remdesivir mightreduce the risk of serious adverse events; that intravenous immunoglobinmight reduce the risk of death and serious adverse events; that tocilizumabmight reduce the risk of serious adverse events and mechanical ventilation;and that bromhexine might reduce the risk of non-serious adverse events.More trials with low risks of bias and random errors are urgently needed.This review will continuously inform best practice in treatment and clinicalresearch of COVID-19.Systematic review registrationPROSPERO CRD42020178787.

BackgroundBiologic therapies targeting inflammatory cytokines have transformed the management of plaque psoriasis; however, their use is limited by high cost, parenteral administration, and monitoring requirements. Icotrokinra (JNJ-77242113) is a first-in-class oral peptide that selectively inhibits the interleukin-23 (IL-23) receptor and represents a potential oral alternative. We conducted a systematic review and meta-analysis to evaluate the efficacy and safety of oral icotrokinra in moderate-to-severe plaque psoriasis.MethodsA comprehensive search of PubMed, Scopus, Web of Science, and the Cochrane Library was performed to identify randomized controlled trials (RCTs) published up to November 2025. Eligible studies enrolled adolescents or adults with moderate-to-severe plaque psoriasis treated with oral icotrokinra (200 mg once daily) versus placebo. A random-effects model was applied, and dichotomous outcomes were pooled as risk ratios (RRs) with 95% confidence intervals (CIs). Risk of bias was assessed using RoB 2, and trial sequential analysis (TSA) was performed to evaluate the conclusiveness of evidence.ResultsFive RCTs comprising 1,951 participants were included. At week 16, icotrokinra significantly improved Investigator’s Global Assessment (IGA) 0/1 (RR = 7.27, 95% CI 5.62–9.40) and Psoriasis Area and Severity Index (PASI) 75 responses (RR = 6.70, 95% CI 5.20–8.62) compared with placebo (both p < 0.001). Higher levels of skin clearance were also achieved, including PASI 90 (RR = 13.82, 95% CI 8.75–21.84) and PASI 100 (RR = 31.65, 95% CI 12.56–79.76). Significant benefits were observed in scalp-specific disease (ss-IGA; RR = 4.27) and patient-reported outcomes, including complete symptom resolution on the Psoriasis Symptom Scale Diary (RR = 9.76). Adverse event rates did not differ significantly between icotrokinra and placebo, and heterogeneity across outcomes was minimal. TSA indicated that current evidence remains insufficient to confirm definitive conclusions.ConclusionOral icotrokinra demonstrates potential efficacy across multiple clinical and patient-reported endpoints with a safety profile comparable to placebo in moderate-to-severe plaque psoriasis. However, TSA indicates that the required information size has not been reached. Therefore, current evidence remains insufficient to draw definitive conclusions. Future RCTs with long-term follow-up are required to confirm these findings.Systematic Review Registrationhttps://www.crd.york.ac.uk/prospero/display_record.php?RecordID=1237937, identifier CRD420251237937