Validity of Using Cancer Registry Data in Comparative Effectiveness Research Involving Radiation Therapy

Abstract

Randomized clinical trials represent the gold standard in comparative effectiveness research, yet many clinical scenarios lack randomized data therefore turning researchers to analyses using other data sources. A large body of comparative effectiveness research uses cancer registry data to compare survival among different treatments. However, the non-randomized nature of these analyses is subject to potential confounding and selection bias. The purpose of this study was to determine whether comparative effectiveness analyses using cancer registry data from the National Cancer Database (NCDB) produces results concordant with randomized trials in radiation oncology. We identified randomized clinical trials involving radiation therapy referenced in the National Comprehensive Cancer Network (NCCN) Clinical Practice Guidelines for central nervous system, head and neck, breast, lung, esophageal, gastric, pancreas, rectal, anal, bladder, prostate, lymphoma, uterine, and cervical cancers. We included only randomized trials with treatment arms identifiable within NCDB. For each trial we extracted patients from NCDB matching the eligibility criteria of the randomized trial whenever possible. For each trial we used three Cox regression models to determine hazard ratios (HRs) for overall survival: univariable, multivariable, and propensity score adjusted models. Multivariable analyses controlled for potential confounders including demographic, comorbidity, clinical, treatment and tumor-related variables. Each NCDB survival analysis was defined as discordant if the HR for the NCDB analysis fell outside the 95% confidence interval of the corresponding randomized trial. Separately, we also assessed for disagreement with statistical significance, with p<0.05 for NCDB and p>0.05 for the clinical trial (or vice versa) defined as discordant. Thirty-two randomized trials met inclusion criteria. NCDB analyses produced hazard ratios for survival discordant with randomized trials in 16 (50%) univariable analyses, 10 (31%) multivariable analyses, and 12 (37%) propensity score analyses. NCDB analyses produced p-values discordant with randomized trials with 26 (81%) univariable analyses, 22 (69%) multivariable analyses, and 20 (62%) propensity score analyses. We did not identify any clinical trial characteristics specifically associated with discordance between NCDB analyses and randomized trials including disease site, severity of cancer, or era of trial. Comparative effectiveness research involving radiation therapy using NCDB frequently produces results discordant to randomized data. Multivariable or propensity score analysis modestly improves concordance between NCDB and clinical trials. Caution should be used when interpreting comparative effective research with cancer registry data.