Changes in trial parameters

Abstract

Michael Levin and colleagues (Sept 16, p 961)1Levin M Quint PA Goldstein B et al.Recombinant bactericidal/permeabilityincreasing protein (rBPI21) as adjunctive tretment for children with severe meningococcal sepsis: a randomised trial.Lancet. 2000; 356: 961-967Summary Full Text Full Text PDF PubMed Scopus (347) Google Scholar report results of a placebo controlled trial of bactericidal/permeability-increasing protein (BPI) in children with meningococcal sepsis. Although the original primary endpoint of the trial, 60 day mortality, and a subsequently specified composite primary endpoint do not differ significantly between study groups, they conclude, on the basis of selected endpoints, that the trial demonstrated efficacy. Levin and colleagues are to be commended for studying this challenging and important issue, but their conclusion remains speculative. The trial was designed to be able to detect a reduction in mortality from 25% in controls to 10% with BPI. During the study it became clear mortality was strikingly lower than expected, and, despite an expansion in size, the trial could detect no difference of the magnitude sought. As a result, the sponsor, XOMA, proposed to change the primary endpoint from mortality alone to composite of mortality and morbidities to increase the event rate and, hopefully, the power. Since the elements of the composite were of dissimilar clinical severity, Levin and colleagues chose a ranked outcome assessment, with death as the worst outcome, followed by recovery with specified moderate to severe morbidities, followed by recovery with mild or no morbidities. Given the rarity of this disease and that the database remained masked, the Food and Drug Administration accepted the sponsor's proposal. Levin and colleagues state that the sponsor did not consult them about the change of endpoints and that the case-report forms were not optimum for assessment of the new primary endpoint. We concur that these deficiencies are unfortunate, and note that it is the sponsor's responsibility to ensure communication with trial investigators and to properly collect trial data. Despite the inability to detect significant differences between BPI and placebo by the original intentionto-treat mortality analysis and the modified primary endpoints, as well as many other secondary outcomes, Levin and colleagues conclude from analyses of selected retrospectively defined, secondary outcome measures, including mortality rates in patients who completed treatment, that BPI has beneficial effects. However, although perhaps representing a trend toward benefit, neither this analysis nor the purported reduction in patients needing severe amputations were significant, despite being described as “much lower” and “a striking reduction”, respectively. When there is good cause (in this case, an unexpectedly low mortality rate and inability to adequately expand the trial), the Food and Drug Administration might allow changes to analytical plans before revealing data. When there are many potential analyses (eg, various outcome measures, subsets, statistical tests, and analytic approaches), even an ineffective therapy can, by chance, seem effective on one or more analyses. Thus, selection of analyses after data are revealed frequently introduces bias. When prospectively defined endpoints show no significant differences from placebo, as occurred in this trial, other analyses, even if encouraging, may be judged suggestive of benefit but should not alone lead to major and definitive conclusions, such as that a product is effective.