Negative Studies in Cancer Research: Why the Negativity?

Abstract

Shortly after the US Congress appropriated $5 million to establish theNational Cancer Institute (NCI) in 1955, cooperative groups were organized to accelerate the fight against cancer. Their coordinated efforts have been very successful, generating evidence that has influenced the entire spectrum of cancer care. Yet NCI cooperative groups face significant challenges in the current austere research funding environment.With limited funds, assessing the scientific impactofpreviously fundedresearchoffers thepotential toguide optimal and efficient use of scarce research dollars. One approach for assessing the impact of a research portfolio is to explore positive and negative trials. Although a positive result showing superiority of one treatment arm is typically thegoalwhendesigningaclinical trial,negative trials, or those failing to demonstrate a statistically significant differencebetweentreatmentarms in theirprimaryoutcome, still producemeaningfulknowledge.Forexample,numerousnegative cooperative group trials have concluded treatmentswith less morbidity offered at least comparable clinical outcomes tomore intensive therapies.Without thesestudies,manymore patientswithbreast cancerwouldhaveundergoneradicalmastectomies instead of breast conservation surgery (NSABP B-06),1 and12monthsofadjuvant therapyforcoloncancermay have persisted rather than the current 6 months standard (Intergroup 0089).2 In this issue of JAMAOncology, Unger and colleagues3 set out to evaluate the impact of phase 3 clinical trials led by the SouthwestOncologyGroup (SWOG). Recognizing that a negative trial may be regarded as a failure or poor public investment, the authors sought to compare the scientific impact of positive and negative phase 3 treatment trials. Analyzing the outcomes and subsequent citations of 94 phase 3 randomized trials conducted by SWOG during a 30-year period, Unger and colleagues3 report a number of important findings. First, the authors found that only 28% of trials yielded positive results. Negative trials were far more common, accounting for 42%, with approximately 28% of studies closing early due to failure to accrue patients.3 It is striking that a phase 3 clinical trial was just as likely to fail due to insufficient accrual, as toyield apositive result. Thisunderscores the need to not only redouble efforts to understand and improve study accrual but also to ensure that all completed studies are used to theirmaximalbenefit.Knowing that amajorityof trials are likely tobenegative, it isessential that thedataderivedfrom these trials be shared with the scientific community to accelerate learning. That each SWOG data set was used for an average of 4 publications per study suggests that the data are beingusedby the scientific community, butgiven the timeand expense invested in generating thesedata, theremaybe room for improvement. For example, theNCI could consider a platform similar to National Heart, Lung, and Blood Institute’s BioLINCC (Biologic Specimen and Data Repository InformationCoordinatingCenter), ensuringpatient level trial data are discoverable, accessible, usable, and widely shared.4 Ungerandcolleagues3alsoprovideanempiricalassessment of SWOG’s phase 3 trials’ scientific impact. The SWOG group’s recordofpublishing the resultsof theirworkwasoutstanding— publishing96%of their completed studies comparedwith the publication rate of less than 70% for completed clinical trials fundedby theNational InstitutesofHealth.5Asecondmeasure of scientific impact is citation rate. The authors found that the primarypublicationofpositivetrialsweremorefrequentlycited thannegativestudies (averageperyear43vs21; relativerisk,2.0; 95%CI, 1.1-3.9; P = .03), butwhen the authors considered primaryandsecondarymanuscripts, theoverall scientific impact of positive and negative phase 3 trials by citation counts were similar (55 vs 45; relative risk, 1.2; 95%CI, 0.7-2.3; P = .53).3 As the authors acknowledge, their approach to assessing the impact of phase 3 trials did not include the most importantoutcome: impactonpatient care. Scientific citationsalone are an imperfect measure of a clinical trial’s worth. For example, theprimaryarticle for thephase3study (PBT-1)6widely recognized as confirming the lack of benefit for high-dose chemotherapy with stem cell rescue for breast cancer has a citation rate of 30 per year using the authors’ methods. This negative studyhadaprofound impact on the clinicalmanagement of patients with breast cancer, but it carries less scientific weight than the average positive SWOG study based on manuscript citations alone (43 citations per year). Future efforts to assess the clinical impact of completed trials should incorporate changes in consensus clinical guidelines, payer policies, or direct patient care patterns. The NCI has adopted a number of initiatives to augment the scientific impact of their treatment trials, including universal biorepositories for molecular analyses and incorporation of patient reported outcomes. These changes will increase the scientific yield of each trial.However, basedon this analysis of SWOG data, it is likely that negative trials will Related article page 875 Opinion