Assessment of pharmacogenomic agreement.

Zhaleh Safikhani,Christos Hatzis,Christopher Mason,Rene Quevedo,Leming Shi,Nicolai Juul Birkbak,Hugo Jwl Aerts,Nehme El-Hachem,Anna Goldenberg,John Quackenbush,Petr Smirnov,Benjamin Haibe-Kains

doi:10.12688/f1000research.8705.1

Abstract

In 2013 we published an analysis demonstrating that drug response data and gene-drug associations reported in two independent large-scale pharmacogenomic screens, Genomics of Drug Sensitivity in Cancer (GDSC) and Cancer Cell Line Encyclopedia (CCLE), were inconsistent. The GDSC and CCLE investigators recently reported that their respective studies exhibit reasonable agreement and yield similar molecular predictors of drug response, seemingly contradicting our previous findings. Reanalyzing the authors' published methods and results, we found that their analysis failed to account for variability in the genomic data and more importantly compared different drug sensitivity measures from each study, which substantially deviate from our more stringent consistency assessment. Our comparison of the most updated genomic and pharmacological data from the GDSC and CCLE confirms our published findings that the measures of drug response reported by these two groups are not consistent. We believe that a principled approach to assess the reproducibility of drug sensitivity predictors is necessary before envisioning their translation into clinical settings.

Highlights

Pharmacogenomic studies correlate genomic profiles and sensitivity to drug exposure in a collection of samples to identify molecular predictors of drug response
The groundbreaking release of the Genomics of Drug Sensitivity in Cancer[1] (GDSC) and Cancer Cell Line Encyclopedia[2] (CCLE) datasets enables the assessment of pharmacogenomic data consistency, a necessary requirement for developing robust drug sensitivity predictors
We briefly describe the fundamental analytical differences between our initial comparative study[3] and the recent assessment of pharmacogenomic agreement published by the GDSC and CCLE investigators[4]

Summary

Introduction

Pharmacogenomic studies correlate genomic profiles and sensitivity to drug exposure in a collection of samples to identify molecular predictors of drug response. By investigating the authors’ code and methods, we identified key shortcomings in their analysis protocol, which have contributed to the authors’ assertion of consistency between drug sensitivity predictors derived from GDSC and CCLE For their ANOVA analyses, the authors used drug activity area (1-AUC) values independently generated in GDSC and CCLE, but used the same GDSC mutation data across the two different datasets (Figure 1b; see Methods). By using the same mutation calls for both GDSC and CCLE, the authors have disregarded the noise in the molecular profiles, while creating an information leak between the two studies For their ElasticNet analysis, the authors followed a similar design by reusing the CCLE genomic data across the two datasets, but comparing different drug sensitivity measures that are IC50 in GDSC vs AUC in CCLE (Figure 1c; see Methods). Of copy number data is quite remarkable (Figure 3a) and could be partly attributed to the fact that CCLE investigators used their SNP array data to compare cell line fingerprints with those of the GDSC project prior to publication and removed the discordant cases from their dataset[2]

Conclusions

Findings

Methods