Does quality of treatment-related harms reporting affect conclusions about superiority of treatment?

Abstract

6607 Background: It is often not known if harms were considered in the overall benefit-risk evaluation in deciding about the superiority of treatments. Here we present a study examining the relationship between researchers’ conclusions about the superiority of treatment with quality of treatment-related harms reporting. Methods: We reviewed all consecutive phase III randomized clinical trials (RCTs) conducted by Southwest Oncology Group from 1960 to 2003 (117 RCTs involving 139 comparisons enrolling 58,908 patients). We extracted data on primary outcomes [overall survival (OS), event free survival (EFS)], and treatment-related mortality (TRM). We classified the quality of reporting as good, intermediate or poor. Association of superiority of treatment and harms reporting was evaluated using chi-square test and meta-analytic techniques. Results: 76% of studies (106/139) reported TRM. Of these, the quality of reporting of harms was considered as “good” in 36% (38/106) of studies, “intermediate” in 55% (58/106), and “poor” in 9% (10/106). Investigators judged experimental treatments to be superior in 34% of trials (48/139) while standard treatment was superior in 66% (91/139). TRM data was reported in 75% (36/48) of results favoring experimental treatments and 78% (70/91) of results favoring standard treatments. There was no association between superiority of a treatment and TRM reporting (p = 0.80) or quality of harms reporting (p = 0.83). The pooled hazard ratio (HR) for OS in RCTs reporting TRM was 0.94 (95% CI: 0.90, 0.99) and for RCTs not reporting TRM was 0.99 (95% CI: 0.89, 1.09) (test of heterogeneity p = 0.06). The pooled HR for EFS among RCTs reporting TRM was 0.88 (95% CI: 0.83, 0.94) and for RCTs not reporting TRM it was 1.01 (95% CI: 0.79, 1.29) (test of heterogeneity p = 0.07). Additionally, the pooled HR for OS and EFS did not show any bias in reporting of harms according to harms reporting quality. Conclusions: Investigators’ conclusion regarding the superiority of experimental or standard treatment does not appear to be associated with outcome reporting bias for harms. Investigators judge both harms and benefits when they draw conclusions about treatment superiority. No significant financial relationships to disclose.