How many biological replicates are needed in an RNA-seq experiment and which differential expression tool should you use?

Nicholas J Schurch,Nicola Wrobel,Pietá Schofield,Gordon G Simpson,Marek Gierliński,Vijender Singh,Christian Cole,Tom Owen-Hughes,Karim Gharbi,Mark Blaxter,Geoffrey J Barton,Alexander Sherstnev

doi:10.1261/rna.053959.115

Abstract

RNA-seq is now the technology of choice for genome-wide differential gene expression experiments, but it is not clear how many biological replicates are needed to ensure valid biological interpretation of the results or which statistical tools are best for analyzing the data. An RNA-seq experiment with 48 biological replicates in each of two conditions was performed to answer these questions and provide guidelines for experimental design. With three biological replicates, nine of the 11 tools evaluated found only 20%–40% of the significantly differentially expressed (SDE) genes identified with the full set of 42 clean replicates. This rises to >85% for the subset of SDE genes changing in expression by more than fourfold. To achieve >85% for all SDE genes regardless of fold change requires more than 20 biological replicates. The same nine tools successfully control their false discovery rate at ≲5% for all numbers of replicates, while the remaining two tools fail to control their FDR adequately, particularly for low numbers of replicates. For future RNA-seq experiments, these results suggest that at least six biological replicates should be used, rising to at least 12 when it is important to identify SDE genes for all fold changes. If fewer than 12 replicates are used, a superior combination of true positive and false positive performances makes edgeR and DESeq2 the leading tools. For higher replicate numbers, minimizing false positives is more important and DESeq marginally outperforms the other tools.

Highlights

RNA-seq has supplanted microarrays as the technology of choice for genome-wide differential gene expression (DGE) experiments
The performance of each DGE tool as a function of replicate number and expression fold change was evaluated by comparing the DGE results from subsets of these replicates against the “gold standard” set of DGE results obtained for each tool with the full set of clean replicates
The tool-specific gold standards were computed by running the tool on the read-count-per-gene measurements from the full set of clean data and marking as “significantly differentially expressed” (SDE) those differentially expressed genes with multiple testing corrected P-values or false discovery rate (FDR) ≤0.05

Summary

Introduction

RNA-seq has supplanted microarrays as the technology of choice for genome-wide differential gene expression (DGE) experiments. For microarray methods it has been shown that low replicate experiments often have insufficient statistical power to call DGE correctly (Pan et al 2002) and cannot accurately measure the natural biological variability (Churchill 2002). It is widely appreciated that increasing the number of replicates in an RNA-seq experiment usually leads to more robust results (Auer and Doerge 2010; Hansen et al 2011; Busby et al 2013; Liu et al 2014), the precise relationship between replicate number and the ability to correctly identify the differentially expressed genes (i.e., the statistical power of the experiment) has not been fully explored.

Methods

Results

Conclusion