To Replicate or Not to Replicate: The Case of Pharmacogenetic Studies

Abstract

Replication of positive findings from genetic association studies has long served as the gold standard for establishing validity of observed genotype–phenotype associations.1–3 In addition to adequate statistical power and sound epidemiological design aimed at minimizing biases, suggested criteria for successful replication studies include: (1) same genetic variant, (2) same direction of association, (3) same definition of the phenotype, and (4) same ethnic group as reported in the discovery study.1 However, in the context of pharmacogenetics, the phenotype requirement is further constrained by the drug and intervention strategy of the initial investigation. As a result, and because many pharmacogenetic and pharmacogenomic studies arise from randomized clinical trials, replication of gene–drug response associations has proven to be a costly and logistically difficult endeavor.4,5 In light of these constraints, this review presents the case for rethinking the replication requirement and proposes several alternative approaches to validating pharmacogenomic findings. Response by Ioannidis on p 412 The prevailing view is that the requirement of replication, currently imposed or strongly suggested by most peer-reviewed journals, has improved the quality of published studies by reducing false-positive findings and overcoming the winner’s curse.6 At first glance, replication rates for findings from both candidate gene studies and agnostic approaches, such as genome-wide association studies (GWAS) and more commonly whole exome or whole genome scans, appear low. For example, of 70 previously reported candidate loci for common diseases, only 22 had P values <10−7 across 100 subsequently conducted GWAS.7 Similarly, a recent analysis of large-scale genetic studies including at least 50 variants and published between 2007 and 2010 suggests that only 1.2% of initial findings are replicated.8 In pharmacogenetic studies of cardiovascular outcomes, where the phenotype is defined as response to drug treatment, the replication rate (3.4%) is comparable to …