An improved filtering algorithm for big read datasets and its application to single-cell assembly

Axel Wedemeyer,Anand Srivastav,Thorsten B Reusch,Philip Rosenstiel,Christian Schielke,Lasse Kliemann

doi:10.1186/s12859-017-1724-7

Axel Wedemeyer, Anand Srivastav + Show 4 more

Open Access

https://doi.org/10.1186/s12859-017-1724-7

Copy DOI

Abstract

BackgroundFor single-cell or metagenomic sequencing projects, it is necessary to sequence with a very high mean coverage in order to make sure that all parts of the sample DNA get covered by the reads produced. This leads to huge datasets with lots of redundant data. A filtering of this data prior to assembly is advisable. Brown et al. (2012) presented the algorithm Diginorm for this purpose, which filters reads based on the abundance of their k-mers.MethodsWe present Bignorm, a faster and quality-conscious read filtering algorithm. An important new algorithmic feature is the use of phred quality scores together with a detailed analysis of the k-mer counts to decide which reads to keep.ResultsWe qualify and recommend parameters for our new read filtering algorithm. Guided by these parameters, we remove in terms of median 97.15% of the reads while keeping the mean phred score of the filtered dataset high. Using the SDAdes assembler, we produce assemblies of high quality from these filtered datasets in a fraction of the time needed for an assembly from the datasets filtered with Diginorm.ConclusionsWe conclude that read filtering is a practical and efficient method for reducing read data and for speeding up the assembly process. This applies not only for single cell assembly, as shown in this paper, but also to other projects with high mean coverage datasets like metagenomic sequencing projects.Our Bignorm algorithm allows assemblies of competitive quality in comparison to Diginorm, while being much faster. Bignorm is available for download at https://git.informatik.uni-kiel.de/axw/Bignorm.

Highlights

For single-cell or metagenomic sequencing projects, it is necessary to sequence with a very high mean coverage in order to make sure that all parts of the sample DNA get covered by the reads produced
In order to speed up the assembly process — or in extreme cases to make it possible in the first place, given certain restrictions on available RAM and/or time — a subdataset of the sequencing dataset is to be determined such that an assembly based on this sub-dataset works as good as possible
Each of the blue boxes corresponds to Bignorm with a particular Q0, while Diginorm is represented as the wide orange box in the background

Summary

Introduction

For single-cell or metagenomic sequencing projects, it is necessary to sequence with a very high mean coverage in order to make sure that all parts of the sample DNA get covered by the reads produced. This leads to huge datasets with lots of redundant data. Conclusions: We conclude that read filtering is a practical and efficient method for reducing read data and for speeding up the assembly process This applies for single cell assembly, as shown in this paper, and to other projects with high mean coverage datasets like metagenomic sequencing projects. For a formal description of the problem, see Additional file 1: Section S1

Methods

Results

Discussion

Conclusion