Sequencing Quality Control Project Research Articles

RNA-Seq Reproducibility Assessment of the Sequencing Quality Control Project.

With the widespread RNA-seq applications of different sequencing platforms in biomedical science research in recent years, a systematic evaluation of RNA-seq data quality is crucial and timely. The Sequencing Quality Control (SEQC) project is a large-scale community effort for assessing the performance of RNA-seq technology across different platforms and multiple laboratories, where reference RNA samples with multiple replicates were sequenced at 12 laboratories using 3 sequencing platforms. Different from the SEQC project, we performed an independent and comprehensive analysis of RNA-seq data of the SEQC project to assess sequencing reproducibility across platforms, sequencing sites, sample replicates, and FlowCells, respectively. With the employment of graphical tools and statistical models, our systemic analysis supports a distinctive conclusion that reproducibility across platforms and sequencing sites are not acceptable, whereas reproducibility across sample replicates and FlowCells are acceptable.

Abstract 1623: Inter-laboratory harmonization of next generation sequencing somatic mutation assays for cancer response prediction

Abstract Background: The potential for next generation sequencing (NGS) analysis in personalized diagnosis and treatment of cancer is advancing rapidly. Measurement of mutation fraction less than 5% may be clinically relevant in analysis of circulating tumor DNA and/or small cytologic samples. However, due to uncertainty introduced by variation in sample quality, the mutation fraction lower limit of detection (LOD) reported by clinical laboratories does not typically extend beyond this 5% threshold. Even with a more conservative LOD criterion, (e.g., 20% mutation fraction), poor harmony may result from inter-laboratory variation in skills to assess sample quality as well as specimen characteristics that are not evident even to expert clinical pathologists. Thus, a critical unmet need is methods that harmonize reporting of mutation fraction measured by NGS by accurately estimating specimen-specific confidence limits for each measurement. Methods: We designed a mixture of spike-in competitive internal standards for tumor somatic mutations in 32 currently actionable genes that enable quality-controlled measurement by targeted NGS using either PCR-amplicon or hybrid-capture enrichment. We also developed a data analysis pipeline suitable to separate target and internal standard sequencing reads prior to analysis of target copy number and sequencing errors. Through participation in the FDA-led SEquencing Quality Control Project Phase 2 (SEQC2), where a set of highly characterized cell line mixtures was developed to assess cross-site reproducibility and mutation detection accuracy of multiple targeted sequencing platforms, we aim to comprehensively evaluate the utility of the spike-in mixture with various targeted NGS panels to measure 95% confidence limits around each somatic mutation fraction measurement. Results: In pilot studies, combining the mixture of internal standards with specimens prior to NGS library preparation enabled reliable measurement of 95% confidence limits around each somatic mutation measurement. Further, using the internal standards to control for specimen adequacy enabled reliable measurement of actionable mutation fraction at as low as 0.01% in suitable specimens. Conclusion: We expect that the full cross-site SEQC2 study will provide further support for the role of competitive spike-in mixtures to provide reliable 95% confidence limits around each measurement of somatic mutation fraction in a clinical specimen. Disclaimer: The views expressed here are those of the authors only and do not necessarily express the views/policies of the FDA. Citation Format: James C. Willey, Thomas M. Blomquist, Erin L. Crawford, Joshua Xu, Leihong Wu, Thomas Morrison. Inter-laboratory harmonization of next generation sequencing somatic mutation assays for cancer response prediction [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2018; 2018 Apr 14-18; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2018;78(13 Suppl):Abstract nr 1623.

Comprehensive Assessments of RNA-seq by the SEQC Consortium: FDA-Led Efforts Advance Precision Medicine.

Studies on gene expression in response to therapy have led to the discovery of pharmacogenomics biomarkers and advances in precision medicine. Whole transcriptome sequencing (RNA-seq) is an emerging tool for profiling gene expression and has received wide adoption in the biomedical research community. However, its value in regulatory decision making requires rigorous assessment and consensus between various stakeholders, including the research community, regulatory agencies, and industry. The FDA-led SEquencing Quality Control (SEQC) consortium has made considerable progress in this direction, and is the subject of this review. Specifically, three RNA-seq platforms (Illumina HiSeq, Life Technologies SOLiD, and Roche 454) were extensively evaluated at multiple sites to assess cross-site and cross-platform reproducibility. The results demonstrated that relative gene expression measurements were consistently comparable across labs and platforms, but not so for the measurement of absolute expression levels. As part of the quality evaluation several studies were included to evaluate the utility of RNA-seq in clinical settings and safety assessment. The neuroblastoma study profiled tumor samples from 498 pediatric neuroblastoma patients by both microarray and RNA-seq. RNA-seq offers more utilities than microarray in determining the transcriptomic characteristics of cancer. However, RNA-seq and microarray-based models were comparable in clinical endpoint prediction, even when including additional features unique to RNA-seq beyond gene expression. The toxicogenomics study compared microarray and RNA-seq profiles of the liver samples from rats exposed to 27 different chemicals representing multiple toxicity modes of action. Cross-platform concordance was dependent on chemical treatment and transcript abundance. Though both RNA-seq and microarray are suitable for developing gene expression based predictive models with comparable prediction performance, RNA-seq offers advantages over microarray in profiling genes with low expression. The rat BodyMap study provided a comprehensive rat transcriptomic body map by performing RNA-Seq on 320 samples from 11 organs in either sex of juvenile, adolescent, adult and aged Fischer 344 rats. Lastly, the transferability study demonstrated that signature genes of predictive models are reciprocally transferable between microarray and RNA-seq data for model development using a comprehensive approach with two large clinical data sets. This result suggests continued usefulness of legacy microarray data in the coming RNA-seq era. In conclusion, the SEQC project enhances our understanding of RNA-seq and provides valuable guidelines for RNA-seq based clinical application and safety evaluation to advance precision medicine.

Abstract 4150: Quantitative sequencing following PCR-driven library preparation with internal standard mixtures has improved analytical performance and lower cost.

Abstract Background: Next-generation sequencing (NGS) is amenable to a multitude of clinical applications by virtue of its automated and highly parallelized analysis of nucleic acid templates. However, prior studies have identified non-systematic biases introduced during preparation of NGS libraries as the primary source of technical variation preventing immediate application for measuring nucleic acid abundance in the clinical setting. We reasoned that a PCR-based NGS library preparation protocol that incorporated competitive internal amplification control (IAC) mixtures (i.e. internal standards) would control for the majority of bias introduced during NGS library preparation, enabling clinical laboratories to offer cost effective moderately complex diagnostic panels from quantitative NGS data. Methods: In order to test this approach, we obtained reference material RNA titration pools used in the FDA-sponsored Sequencing Quality Control (SEQC) project that have been characterized for nucleic acid abundance by multiple qPCR, Microarray and NGS platforms. Using Multiplex-PCR with primers and competitive IAC for 150-gene targets we prepared NGS libraries from: 1) gDNA to test general analytical performance, and 2) cDNA from reverse transcribed SEQC project reference material to determine accuracy in detecting fold change. Using gDNA mixed with serially titrated IAC mixtures as input, we observed a linear dynamic range over 106 orders of magnitude, with an average R2 = 0.995 (0.993 – 0.997; 95% CI). There was a high correlation coefficient (R2= 0.96) between measured values for copies of nucleic acid abundance in two separate library preparations (separate reverse transcriptions and Multiplex PCR-based library preparations) from the same reference RNA material (FDA SEQC project Sample A). Because the SEQC project RNA Samples C and D represent a known cross titration between SEQC project RNA Samples A and B, by comparing measured to expected values for expression of each gene in Samples C and D it is possible to determine accuracy of the method. In preliminary studies, the correlation coefficient of expected versus observed for Sample C was R2 = 0.96, with an ROC curve-determined accuracy to detect a 3-fold change of 97% (95 – 99%; 95% CI). Inter-platform and inter-laboratory comparisons are ongoing. Conclusion: The approach described here overcomes key sources of non-systematic bias introduced during NGS library preparation. This should enable reproducible inter-laboratory and inter-platform quantitative NGS results, and a clear path to regulatory approval for clinical diagnostic applications. Citation Format: Thomas Blomquist, Erin L. Crawford, James C. Willey. Quantitative sequencing following PCR-driven library preparation with internal standard mixtures has improved analytical performance and lower cost. [abstract]. In: Proceedings of the 104th Annual Meeting of the American Association for Cancer Research; 2013 Apr 6-10; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2013;73(8 Suppl):Abstract nr 4150. doi:10.1158/1538-7445.AM2013-4150