Reforming Subgroup Analysis

Abstract

The FDA largely approves or disapproves drugs based on average treatment effects. Given widespread heterogeneity in treatment response, this approach can result in the approval of drugs with significant negative effects for identifiable subgroups (false positives) and in the non-approval of drugs with significant positive effects for identifiable subgroups (false negatives). Despite the FDA's position, drug companies frequently conduct post hoc subgroup analysis - a search for responsive subgroups - after their clinical trials find no positive average treatment effects. The FDA rejects such analysis due to the risk of spurious results. With sufficient covariate measurements, a drug company can always find some subgroup that benefits from a drug. This paper asks whether there workable compromise between the FDA and drug companies. Specifically, we seek a drug approval process that can use post hoc subgroup analysis to eliminate false negatives but does not risk opportunistic behavior and spurious correlation. The primary reform we recommend is a statistical analysis of a random subset of the data set from a clinical trial by an independent researcher. The subsample examined by the independent researcher can eliminates the risk of spurious findings due to multiple testing in the remainder of the sample. We apply our approach to the results of a recent clinical trial of a cancer drug, Xcytrin, that failed to find positive average treatment effects, and discover positive treatment effects for an important subset of patients.