Questionable interim analyses in the Goal-Directed Perfusion Trial study of goal-directed perfusion

Abstract

Author has nothing to disclose with regard to commercial support. Author has nothing to disclose with regard to commercial support. The recently published article by Ranucci and colleagues1Ranucci M. Johnson I. Willcox T. Baker R.A. Boer C. Baumann A. et al.Goal-directed perfusion to reduce acute kidney injury: a randomized trial.J Thorac Cardiovasc Surg. 2018; 156: 1918-1927.e2Abstract Full Text Full Text PDF PubMed Scopus (99) Google Scholar is a randomized trial of a protocol for goal-directed perfusion of patients undergoing cardiac surgery that suggests efficacy of a goal-directed perfusion strategy aimed at maintaining oxygen delivery at a minimum of 280 mL · min−1 · m−2. I have serious concerns, however, regarding the conduct of interim analyses and early stopping of the trial. In brief, Ranucci and colleagues1Ranucci M. Johnson I. Willcox T. Baker R.A. Boer C. Baumann A. et al.Goal-directed perfusion to reduce acute kidney injury: a randomized trial.J Thorac Cardiovasc Surg. 2018; 156: 1918-1927.e2Abstract Full Text Full Text PDF PubMed Scopus (99) Google Scholar planned 4 analyses: 3 interim analyses (25%, 50%, and 75% completion) and a final analysis at 100% completion. At 25%, an efficacy analysis was performed at an α level of 0.01 (per online supplement). I presume that results were not significant at this first interim analysis with P > .01 as the study continued; however, investigators and statisticians at the sponsoring institution may have observed suggestive interim efficacy results such that they changed their stopping boundary for the P value at the 50% interim analysis from .005 to .05 (per statistical analysis section, main text) for the efficacy analysis. This decision was not grounded in statistical or ethical principles for data monitoring of an ongoing clinical trial. Among many issues with this decision, the combination of the first 2 interim analyses tested at α levels 0.01 and 0.05 levels, respectively, leads to an inflated type I error rate greater than 0.05 for this study design. I note that the actual type I error rate for this approach depends on the rule used to define suggestive results that would lead to an increase in subsequent efficacy boundaries; however, with certainty the type I error rate for this design is greater than 0.05. The 75% interim analysis, had it been needed, was also modified to stopping with P ≤ .05, and I presume that the final analysis would also have tested at P ≤ .05, leading to an overall type I error rate as high as 0.101—a doubling of the type I error rate! Ranucci and colleagues should post their ethics board–approved study protocols before and after modification in a public repository and address the following questions: Were interim analyses specified before the first subject was enrolled (limited details were added to clinicaltrials.gov registration on February 9, 2017, well after the study was already halted for efficacy)? If so, were efficacy stopping boundaries specified in advance for interim and final analyses, with a plan to control the type I error rate,2O’Brien P.C. Fleming T.R. A multiple testing procedure for clinical trials.Biometrics. 1979; 35: 549-556Crossref PubMed Scopus (2378) Google Scholar, 3Pocock S.J. Interim analyses for randomized clinical trials: the group sequential approach.Biometrics. 1982; 38: 153-162Crossref PubMed Scopus (192) Google Scholar at the time that the study enrolled the first patient? Was the ethics board notified that modification would break any previously specified control of the type I error rate? At the 50% interim analysis, the primary end point was analyzed with the result of P = .036 from a χ2 test, which would not have met stopping rules for efficacy if not for this questionable change in the stopping rules. Unfortunately, because the trial was stopped early, we do not know whether results would still be compelling after observing another 350 subjects. Goal-directed perfusion to reduce acute kidney injury: A randomized trialThe Journal of Thoracic and Cardiovascular SurgeryVol. 156Issue 5PreviewTo determine whether a goal-directed perfusion (GDP) strategy aimed at maintaining oxygen delivery (DO2) at ≥280 mL·min−1·m−2 reduces the incidence of acute kidney injury (AKI). Full-Text PDF Open ArchivePerfusion-induced acute kidney injury: Critiques that do not roll off the tongues of thoracic surgeonsThe Journal of Thoracic and Cardiovascular SurgeryVol. 157Issue 5PreviewThe letter to the Editor by Schulte1 in this issue of the Journal raises some important critiques regarding a previously published randomized trial by Ranucci and coauthors.2 The article by Ranucci and coauthors2 was controversial from the day it was published, and likely before publication as well. The editorial process encompassed some difficult decisions both before and after publication of the afrticle.3-5 There were questions raised about how oxygen delivery should be measured. There are at least 5 unmeasured variables that affect oxygen delivery at the cellular level and that confound the study of Ranucci and coworkers2: Full-Text PDF Open ArchiveOn the premature termination of the Goal-directed perfusion trialThe Journal of Thoracic and Cardiovascular SurgeryVol. 157Issue 5PreviewI thank Dr Schulte for his letter, in which he strongly debates the reasons for the premature termination of the Goal-Directed Perfusion Trial (GIFT).1 His contribution gives me the opportunity to clarify some aspects of the GIFT study, and even to provide additional data. Full-Text PDF Open Archive