Massive Amounts Of Sequencing Data Research Articles

Knowledge Gaps, Research Needs, and Opportunities in Plant Disease Diagnostic Assay Development and Validation

Sequence-based identification and detection technologies have revolutionized plant disease diagnostics in the last few decades, opening the door to highly selective and specific diagnostic assays. However, diagnostic assay development and validation remain hindered by many factors that would be ameliorated by research prioritization and coordination. Massive amounts of sequence data, in the forms of evolutionarily informative markers and now whole genomes, are available yet not easily accessible or properly organized to guide efficient assay development and deployment. For many key pathogen groups, particularly those that are not easily culturable, data are scant. Identification of pathogen-specific sequences in microbial genomes, the ultimate diagnostic markers, requires massive data processing and is slowed by a lack of quality data curation pipelines that efficiently translate data into problem solving. New technologies have the potential to enable detection at extremely low titers, but many of these have not yet found their way into diagnostic labs, much less the field. Here, we review the current status of diagnostics readiness in select groups of pathogens and pathosystems and define research priorities to fill knowledge and resource gaps to support scientists engaged in diagnostics. These priorities include improved coordination of data generation and curation and validation of new technologies, such as LAMP, MiFi, RPA, and CRISPR-Cas, to accelerate the implementation of more specific and sensitive assays. Finally, novel approaches, such as detection of volatile profiles and microbiome analyses, have the potential to complement pathogen diagnostics focusing on specific pathogens. [Formula: see text] The author(s) have dedicated the work to the public domain under the Creative Commons CC0 “No Rights Reserved” license by waiving all of his or her rights to the work worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law, 2023.

Analysis of Polyadenylation Signal Usage with Full-Length Transcriptome in Spodoptera frugiperda (Lepidoptera: Noctuidae).

Simple SummaryRNA polyadenylation is an important process in mRNA maturation. The process is controlled by various cis-acting elements surrounding the cleavage site, and their binding factors. Recently high sequencing technology especially full-length transcriptome provides a large amount of sequencing data for us to explore the variations of poly(A) signals, alternative polyadenylation (APA) in Spodoptera frugiperda. We studied 50,616 polyadenylation signals using full-length transcriptome and EST data. These data show that 51.64% of the 50,616 pA sites had the conserved AAUAAA hexamers, while 10.13% of pA sites had none of the AAUAAA-like hexamers. Among these genes, more than 64.76% have more than one pA site. Our results also support that APA plays a significant role in increasing transcriptome diversity and gene expression regulation in Spodoptera frugiperda. Our dataset was the first polyadenylation signal analysis in the Lepidoptera and would provide a theoretical basis for pest control.During the messenger RNA (mRNA) maturation process, RNA polyadenylation is a key step, and is coupled to the termination of transcription. Various cis-acting elements near the cleavage site and their binding factors would affect the process of polyadenylation, and AAUAAA, a highly conserved hexamer, was the most important polyadenylation signal (PAS). PAS usage is one of the critical modification determinants targeted at mRNA post-transcription. The full-length transcriptome has recently generated a massive amount of sequencing data, revealing poly(A) variation and alternative polyadenylation (APA) in Spodoptera frugiperda. We identified 50,616 polyadenylation signals in Spodoptera frugiperda via analysis of full-length transcriptome combined with expression Sequence Tags Technology (EST). The polyadenylation signal usage in Spodoptera frugiperda is conserved, and it is similar to that of flies and other animals. AAUAAA and AUUAAA are the most highly conserved polyadenylation signals of all polyadenylation signals we identified. Additionally, we found the U/GU-rich downstream sequence element (DSE) in the cleavage site. These results demonstrate that APA in Spodoptera frugiperda plays a significant role in root growth and development. This is the first polyadenylation signal usage analysis in agricultural pests, which can deepen our understanding of Spodoptera frugiperda and provide a theoretical basis for pest control.

EasyMicroPlot: An Efficient and Convenient R Package in Microbiome Downstream Analysis and Visualization for Clinical Study.

Advances in next-generation sequencing (NGS) have revolutionized microbial studies in many fields, especially in clinical investigation. As the second human genome, microbiota has been recognized as a new approach and perspective to understand the biological and pathologic basis of various diseases. However, massive amounts of sequencing data remain a huge challenge to researchers, especially those who are unfamiliar with microbial data analysis. The mathematic algorithm and approaches introduced from another scientific field will bring a bewildering array of computational tools and acquire higher quality of script experience. Moreover, a large cohort research together with extensive meta-data including age, body mass index (BMI), gender, medical results, and others related to subjects also aggravate this situation. Thus, it is necessary to develop an efficient and convenient software for clinical microbiome data analysis. EasyMicroPlot (EMP) package aims to provide an easy-to-use microbial analysis tool based on R platform that accomplishes the core tasks of metagenomic downstream analysis, specially designed by incorporation of popular microbial analysis and visualization used in clinical microbial studies. To illustrate how EMP works, 694 bio-samples from Guangdong Gut Microbiome Project (GGMP) were selected and analyzed with EMP package. Our analysis demonstrated the influence of dietary style on gut microbiota and proved EMP package's powerful ability and excellent convenience to address problems for this field.

Hidden Viral Sequences in Public Sequencing Data and Warning for Future Emerging Diseases.

ABSTRACTRNA viruses cause numerous emerging diseases, mostly due to transmission from mammalian and avian reservoirs. Large-scale surveillance of RNA viral infections in these animals is a fundamental step for controlling viral infectious diseases. Metagenomic analysis is a powerful method for virus identification with low bias and has contributed substantially to the discovery of novel viruses. Deep-sequencing data have been collected from diverse animals and accumulated in public databases, which can be valuable resources for identifying unknown viral sequences. Here, we screened for infections of 33 RNA viral families in publicly available mammalian and avian sequencing data and found approximately 900 hidden viral infections. We also discovered six nearly complete viral genomes in livestock, wild, and experimental animals: hepatovirus in a goat, hepeviruses in blind mole-rats and a galago, astrovirus in macaque monkeys, parechovirus in a cow, and pegivirus in tree shrews. Some of these viruses were phylogenetically close to human-pathogenic viruses, suggesting the potential risk of causing disease in humans upon infection. Furthermore, infections of five novel viruses were identified in several different individuals, indicating that their infections may have already spread in the natural host population. Our findings demonstrate the reusability of public sequencing data for surveying viral infections and identifying novel viral sequences, presenting a warning about a new threat of viral infectious disease to public health.

Analysis of Patterns of Co-Occurrence and Mutual Exclusivity of Gene Mutations in Philadelphia-Negative Myeloproliferative (Ph-negative MPNs) and Myelodysplastic/Myeloproliferative Neoplasms (MDS/MPNs)

Introduction: Next-generation sequencing (NGS) has led to the generation of massive amounts of sequencing data from cancer samples, including MPNs and MDS/MPNs. Current publications a vast heterogeneity among tumor genotypes across different patients. However, analysis of patterns of mutations correlation can reveal some genes that are more likely to co-occur in the same patients, while other genes tend to be mutually exclusive and are almost never found in the same case. Analysis of patterns of co-occurrence and mutual exclusivity of gene mutations can lead to insights in the pathogenesis of these diseases and can help identify putative novel driver genes that are mutually exclusive from known driver genes.Patients and Methods:We analyzed data of 646 patients with MPN or MDS/MPN, including 210 patients from our center and 436 patients with data that was available in the literature. Patients from our center were analyzed through either WES (N=123) or targeted sequencing panel of 253 genes involved in myeloid malignancies (N=87). Among the 436 cases selected from published studies in the literature, 278 had been analyzed by WES and 158 were evaluated by large sequencing panels containing the majority of myeloid related genes. We sought to analyze a common denominator of 26 genes that found to be recurrently patients: JAK2, TET2, ASXL1, SRSF2, CALR, DNMT3A, SF3B1, CBL, NRAS, KRAS, PTPN11, EZH2, U2AF1, RUNX1, CUX1, MPL, ZRSR2, TP53, IDH2, SH2B3,PHF6, GATA2, IDH1, SETBP1, ETV6, STAG2 . For the 210 patients included from our center, reads were aligned and processed following Genome Analysis Toolkit best practices, and somatic variants were called combining the output of SomaticSniper, Mutect and Pindel for WES data and Mutect/Mutect2 for targeted panel data. Regarding public data, we selected 7 studies that evaluated patients with MPNs and MDS/MPNs through WES/targeted sequencing and where sample-level data on all individual mutations that were identified were available. Pairwise analysis of occurrence and mutual exclusivity used Fisher tests with p-values adjusted for false discovery rate using the Benjamini-Hochberg method with a q-value cutoof of 0.1, while Multi-Dendrix (Raphael Lab, Brown University) was used for determination of groups of mutual exclusive genes.Results:Diagnosis of the cohort included essential thrombocythemia (ET; N=134), polycythemia vera (PV; N=113), myelofibrosis (MF; N=123), chronic myelomonocytic leukemia (CMML; N=228), atypical CML (aCML; N=15) and more rare disorders (MPN-unclassified [N=4], MDS/MPN-unclassified [N=10], refractory anemia with ring sideroblasts and thrombocytosis [N=18] and chronic neutrophilic leukemia [N=1]). Since some genes tend to co-occur more frequently with a specific disease type (e.g. JAK2 in Polycythemia Vera) we analyzed cases of MPNs and MDS/MPNs separately. Results are summarized in the Table. Among patients with MPNs, some novel associations appear: ASXL1 tends to co-occur with CUX1, EZH2, SRSF2 and U2AF1, while CALR and TET2 mutations tend to be mutually exclusive. SRSF2 mutations tend to co-occur with TET2 and CUX1 mutations, and U2AF1 with STAG2 mutations. In patients with MDS/MPNs, different associations are seen, including a tendency for mutual exclusivity of ASXL1 and DNMT3A mutations (Odds Ratio [OR]=0.09) and TET2 and SF3B1 mutations (OR=0.21), while TET2 t ends to co-occur with KRAS, SRSF2 and ZRSR2, while MPL associates strongly with U2AF1 mutations (OR=9.97). Analysis of sets of mutual exclusive genes with Multi-Dendrix revealed the following sets of genes that tend to be mutually exclusive (considering the whole cohort): Set 1- CALR, CBL, CUX1, JAK2, NRAS, RUNX1; Set 2 - GATA2, IDH2, MPL, SETBP1, TET2; Set 3- EZH2, SF3B1, SRSF2, TP53, U2AF1, ZRSR2; Set 4- ASXL1, DNMT3A, IDH1, KRAS, PHF6. Set 2 is illustrated in Figure 1.Conclusions:Despite a vast array of different mutations and possible combinations in MPNs and MDS/MPNs, it is clear that some combinations of gene mutations occur more frequently than others, while some genes almost never co-occur together. Analysis of the functional consequences of frequent gene associations might lead to improvements in diagnostic and molecular classification of these neoplasms. Larger cohorts, in the range of thousands of patients, are needed to better determine the more prevalent patterns of co-association and mutual exclusivity. [Display omitted] DisclosuresNo relevant conflicts of interest to declare.

A roadmap for metagenomic enzyme discovery.

Covering: up to 2021Metagenomics has yielded massive amounts of sequencing data offering a glimpse into the biosynthetic potential of the uncultivated microbial majority. While genome-resolved information about microbial communities from nearly every environment on earth is now available, the ability to accurately predict biocatalytic functions directly from sequencing data remains challenging. Compared to primary metabolic pathways, enzymes involved in secondary metabolism often catalyze specialized reactions with diverse substrates, making these pathways rich resources for the discovery of new enzymology. To date, functional insights gained from studies on environmental DNA (eDNA) have largely relied on PCR- or activity-based screening of eDNA fragments cloned in fosmid or cosmid libraries. As an alternative, shotgun metagenomics holds underexplored potential for the discovery of new enzymes directly from eDNA by avoiding common biases introduced through PCR- or activity-guided functional metagenomics workflows. However, inferring new enzyme functions directly from eDNA is similar to searching for a ‘needle in a haystack’ without direct links between genotype and phenotype. The goal of this review is to provide a roadmap to navigate shotgun metagenomic sequencing data and identify new candidate biosynthetic enzymes. We cover both computational and experimental strategies to mine metagenomes and explore protein sequence space with a spotlight on natural product biosynthesis. Specifically, we compare in silico methods for enzyme discovery including phylogenetics, sequence similarity networks, genomic context, 3D structure-based approaches, and machine learning techniques. We also discuss various experimental strategies to test computational predictions including heterologous expression and screening. Finally, we provide an outlook for future directions in the field with an emphasis on meta-omics, single-cell genomics, cell-free expression systems, and sequence-independent methods.

The potential of genome-wide RAD sequences for resolving rapid radiations: a case study in Cactaceae

The reconstruction of relationships within recently radiated groups is challenging even when massive amounts of sequencing data are available. The use of restriction site-associated DNA sequencing (RAD-Seq) to this end is promising. Here, we assessed the performance of RAD-Seq to infer the species-level phylogeny of the rapidly radiating genus Cereus (Cactaceae). To examine how the amount of genomic data affects resolution in this group, we used datasets and implemented different analyses. We sampled 52 individuals of Cereus, representing 18 of the 25 species currently recognized, plus members of the closely allied genera Cipocereus and Praecereus, and other 11 Cactaceae genera as outgroups. Three scenarios of permissiveness to missing data were carried out in iPyRAD, assembling datasets with 30% (333 loci), 45% (1440 loci), and 70% (6141 loci) of missing data. For each dataset, Maximum Likelihood (ML) trees were generated using two supermatrices, i.e., only SNPs and SNPs plus invariant sites. Accuracy and resolution were improved when the dataset with the highest number of loci was used (6141 loci), despite the high percentage of missing data included (70%). Coalescent trees estimated using SVDQuartets and ASTRAL are similar to those obtained by the ML reconstructions. Overall, we reconstruct a well-supported phylogeny of Cereus, which is resolved as monophyletic and composed of four main clades with high support in their internal relationships. Our findings also provide insights into the impact of missing data for phylogeny reconstruction using RAD loci.

Deconvolution of nucleic-acid length distributions: a gel electrophoresis analysis tool and applications.

Next-generation DNA-sequencing (NGS) technologies, which are designed to streamline the acquisition of massive amounts of sequencing data, are nonetheless dependent on various preparative steps to generate DNA fragments of required concentration, purity and average size (molecular weight). Current automated electrophoresis systems for DNA- and RNA-sample quality control, such as Agilent’s Bioanalyzer® and TapeStation® products, are costly to acquire and use; they also provide limited information for samples having broad size distributions. Here, we describe a software tool that helps determine the size distribution of DNA fragments in an NGS library, or other DNA sample, based on gel-electrophoretic line profiles. The software, developed as an ImageJ plug-in, allows for straightforward processing of gel images, including lane selection and fitting of univariate functions to intensity distributions. The user selects the option of fitting either discrete profiles in cases where discrete gel bands are visible or continuous profiles, having multiple bands buried under a single broad peak. The method requires only modest imaging capabilities and is a cost-effective, rigorous alternative characterization method to augment existing techniques for library quality control.

Shouji: a fast and efficient pre-alignment filter for sequence alignment.

The ability to generate massive amounts of sequencing data continues to overwhelm the processing capability of existing algorithms and compute infrastructures. In this work, we explore the use of hardware/software co-design and hardware acceleration to significantly reduce the execution time of short sequence alignment, a crucial step in analyzing sequenced genomes. We introduce Shouji, a highly parallel and accurate pre-alignment filter that remarkably reduces the need for computationally-costly dynamic programming algorithms. The first key idea of our proposed pre-alignment filter is to provide high filtering accuracy by correctly detecting all common subsequences shared between two given sequences. The second key idea is to design a hardware accelerator that adopts modern field-programmable gate array (FPGA) architectures to further boost the performance of our algorithm. Shouji significantly improves the accuracy of pre-alignment filtering by up to two orders of magnitude compared to the state-of-the-art pre-alignment filters, GateKeeper and SHD. Our FPGA-based accelerator is up to three orders of magnitude faster than the equivalent CPU implementation of Shouji. Using a single FPGA chip, we benchmark the benefits of integrating Shouji with five state-of-the-art sequence aligners, designed for different computing platforms. The addition of Shouji as a pre-alignment step reduces the execution time of the five state-of-the-art sequence aligners by up to 18.8×. Shouji can be adapted for any bioinformatics pipeline that performs sequence alignment for verification. Unlike most existing methods that aim to accelerate sequence alignment, Shouji does not sacrifice any of the aligner capabilities, as it does not modify or replace the alignment step. https://github.com/CMU-SAFARI/Shouji. Supplementary data are available at Bioinformatics online.

ASAP: <underline>A</underline>ccelerated <underline>S</underline>hort-Read <underline>A</underline>lignment on <underline>P</underline>rogrammable Hardware

The proliferation of high-throughput sequencing machines ensures rapid generation of up to billions of short nucleotide fragments in a short period of time. This massive amount of sequence data can quickly overwhelm today's storage and compute infrastructure. This paper explores the use of hardware acceleration to significantly improve the runtime of short-read alignment, a crucial step in preprocessing sequenced genomes. We focus on the Levenshtein distance (edit-distance) computation kernel and propose the ASAP accelerator, which utilizes the intrinsic delay of circuits for edit-distance computation elements as a proxy for computation. Our design is implemented on an Xilinx Virtex 7 FPGA in an IBM POWER8 system that uses the CAPI interface for cache coherence across the CPU and FPGA. Our design is $200\times$ faster than the equivalent C implementation of the kernel running on the host processor and $2.2\times$ faster for an end-to-end alignment tool for 120-150 base-pair short-read sequences. Further the design represents a $3760\times$ improvement over the CPU in performance/Watt terms.

GLOSSary: the GLobal Ocean 16S subunit web accessible resource

BackgroundEnvironmental metagenomics is a challenging approach that is exponentially spreading in the scientific community to investigate taxonomic diversity and possible functions of the biological components. The massive amount of sequence data produced, often endowed with rich environmental metadata, needs suitable computational tools to fully explore the embedded information. Bioinformatics plays a key role in providing methodologies to manage, process and mine molecular data, integrated with environmental metagenomics collections. One such relevant example is represented by the Tara Ocean Project.ResultsWe considered the Tara 16S miTAGs released by the consortium, representing raw sequences from a shotgun metagenomics approach with similarities to 16S rRNA genes. We generated assembled 16S rDNA sequences, which were classified according to their lengths, the possible presence of chimeric reads, the putative taxonomic affiliation. The dataset was included in GLOSSary (the GLobal Ocean 16S Subunit web accessible resource), a bioinformatics platform to organize environmental metagenomics data. The aims of this work were: i) to present alternative computational approaches to manage challenging metagenomics data; ii) to set up user friendly web-based platforms to allow the integration of environmental metagenomics sequences and of the associated metadata; iii) to implement an appropriate bioinformatics platform supporting the analysis of 16S rDNA sequences exploiting reference datasets, such as the SILVA database. We organized the data in a next-generation NoSQL “schema-less” database, allowing flexible organization of large amounts of data and supporting native geospatial queries. A web interface was developed to permit an interactive exploration and a visual geographical localization of the data, either raw miTAG reads or 16S contigs, from our processing pipeline. Information on unassembled sequences is also available. The taxonomic affiliations of contigs and miTAGs, and the spatial distribution of the sampling sites and their associated sequence libraries, as they are contained in the Tara metadata, can be explored by a query interface, which allows both textual and visual investigations. In addition, all the sequence data were made available for a dedicated BLAST-based web application alongside the SILVA collection.ConclusionsGLOSSary provides an expandable bioinformatics environment, able to support the scientific community in current and forthcoming environmental metagenomics analyses.

ISeq: Web-Based RNA-seq Data Analysis and Visualization.

Transcriptome sequencing (RNA-seq) is becoming a standard experimental methodology for genome-wide characterization and quantification of transcripts at single base-pair resolution. However, downstream analysis of massive amount of sequencing data can be prohibitively technical for wet-lab researchers. A functionally integrated and user-friendly platform is required to meet this demand. Here, we present iSeq, an R-based Web server, for RNA-seq data analysis and visualization. iSeq is a streamlined Web-based R application under the Shiny framework, featuring a simple user interface and multiple data analysis modules. Users without programming and statistical skills can analyze their RNA-seq data and construct publication-level graphs through a standardized yet customizable analytical pipeline. iSeq is accessible via Web browsers on any operating system at http://iseq.cbi.pku.edu.cn .

ACRES: efficient query answering on large compressed sequences

With the advances in next generation sequencing, the amount of genomic sequence data being produced continues to grow at an exponential rate. It is estimated that the entire genome of each individual human, each containing about 3 billion letters, could be made available in the next a few years. An increasingly pressing issue in genomics and medicine is how to efficiently store and query these massive amounts of sequence data. Recently a lossless compression technique has been proposed to drastically reduce the storage space of genomic sequences, taking advantage of the fact that any two genomes from the same species are highly similar and therefore only their differences need to be encoded. In this paper we study how to efficiently answer queries on the compressed sequences without first decompressing them. We study three important types of queries, including retrieving a subsequence, finding subsequences matching a given pattern, and finding subsequences similar to a pattern. We propose an index structure, filtering techniques, and efficient algorithms for answering these queries. We further demonstrate the utility of these algorithms using a real dataset.

NGS-pipe: a flexible, easily extendable and highly configurable framework for NGS analysis

MotivationNext-generation sequencing is now an established method in genomics, and massive amounts of sequencing data are being generated on a regular basis. Analysis of the sequencing data is typically performed by lab-specific in-house solutions, but the agreement of results from different facilities is often small. General standards for quality control, reproducibility and documentation are missing.ResultsWe developed NGS-pipe, a flexible, transparent and easy-to-use framework for the design of pipelines to analyze whole-exome, whole-genome and transcriptome sequencing data. NGS-pipe facilitates the harmonization of genomic data analysis by supporting quality control, documentation, reproducibility, parallelization and easy adaptation to other NGS experiments.Availability and implementation https://github.com/cbg-ethz/NGS-pipe

Genome-wide joint analysis of single-nucleotide variant sets and gene expression for hypertension and related phenotypes.

BackgroundWith the advance of next-generation sequencing technologies, the study of rare variants in targeted genome regions or even the whole genome becomes feasible. Nevertheless, the massive amount of sequencing data brings great computational and statistical challenges for association analyses. Aside from sequencing variants, other high-throughput omic data (eg, gene expression data) also become available, and can be incorporated into association analysis for better modeling and power improvement. This motivates the need of developing computationally efficient and powerful approaches to model the joint associations of multilevel omic data with complex human diseases.MethodsA similarity-based weighted U approach is used to model the joint effect of sequencing variants and gene expression. Using a Mexican American sample provided by Genetic Analysis Workshop 19 (GAW19), we performed a whole-genome joint association analysis of sequencing variants and gene expression with systolic (SBP) and diastolic blood pressure (DBP) and hypertension (HTN) phenotypes.ResultsThe whole-genome joint association analysis was completed in 80 min on a high-performance personal computer with an i7 4700 CPU and 8 GB memory. Although no gene reached statistical significance after adjusting for multiple testing, some top-ranked genes attained a high significance level and may have biological plausibility to hypertension-related phenotypes.ConclusionsThe weighted U approach is computationally efficient for high-dimensional data analysis, and is capable of integrating multiple levels of omic data into association analysis. Through a real data application, we demonstrate the potential benefit of using the new approach for joint association analysis of sequencing variants and gene expression.

CROMqs: an infinitesimal successive refinement lossy compressor for the quality scores.

Massive amounts of sequencing data are being generated thanks to advances in sequencing technology and a dramatic drop in the sequencing cost. Much of the data are comprised of nucleotides and the corresponding quality scores that indicate their reliability. The latter are more difficult to compress and are themselves noisy. As a result, lossy compression of the quality scores has recently been proposed to alleviate the storage costs. Further, it has been shown that lossy compression, at some specific rates, can achieve a performance on variant calling similar to that achieved with the lossless compressed data. We propose CROMqs, a new lossy compressor for the quality scores with the property of "infinitesimal successive refinability". This property allows the decoder to decompress the data iteratively without the need of agreeing with the encoder on a specific rate prior to compression. This characteristic is particularly amenable in practice, as in most cases the appropriate rate at which the lossy compressor should operate can not be established prior to compression. Further, this property can be of interest in scenarios involving streaming of genomic data. CROMqs is the first infinitesimal successive refinement lossy compressor for the quality scores in the literature, and we show that it obtains a comparable rate-distortion performance to previously proposed algorithms. Moreover, we also show that CROMqs achieves a comparable performance on variant calling to that of the lossless compressed data.

A cluster-based approach to compression of Quality Scores.

Massive amounts of sequencing data are being generated thanks to advances in sequencing technology and a dramatic drop in the sequencing cost. Storing and sharing this large data has become a major bottleneck in the discovery and analysis of genetic variants that are used for medical inference. As such, lossless compression of this data has been proposed. Of the compressed data, more than 70% correspond to quality scores, which indicate the sequencing machine reliability when calling a particular basepair. Thus, to further improve the compression performance, lossy compression of quality scores is emerging as the natural candidate. Since the data is used for genetic variants discovery, lossy compressors for quality scores are analyzed in terms of their rate-distortion performance, as well as their effect on the variant callers. Previously proposed algorithms do not do well under all performance metrics, and are hence unsuitable for certain applications. In this work we propose a new lossy compressor that first performs a clustering step, by assuming all the quality scores sequences come from a mixture of Markov models. Then, it performs quantization of the quality scores based on the Markov models. Each quantizer targets a specific distortion to optimize for the overall rate-distortion performance. Finally, the quantized values are compressed by an entropy encoder. We demonstrate that the proposed lossy compressor outperforms the previously proposed methods under all analyzed distortion metrics. This suggests that the effect that the proposed algorithm will have on any downstream application will likely be less noticeable than that of previously proposed lossy compressors. Moreover, we analyze how the proposed lossy compressor affects Single Nucleotide Polymorphism (SNP) calling, and show that the variability introduced on the calls is considerably smaller than the variability that exists between different methodologies for SNP calling.

Denoising of Quality Scores for Boosted Inference and Reduced Storage.

Massive amounts of sequencing data are being generated thanks to advances in sequencing technology and a dramatic drop in the sequencing cost. Much of the raw data are comprised of nucleotides and the corresponding quality scores that indicate their reliability. The latter are more difficult to compress and are themselves noisy. Lossless and lossy compression of the quality scores has recently been proposed to alleviate the storage costs, but reducing the noise in the quality scores has remained largely unexplored. This raw data is processed in order to identify variants; these genetic variants are used in important applications, such as medical decision making. Thus improving the performance of the variant calling by reducing the noise contained in the quality scores is important. We propose a denoising scheme that reduces the noise of the quality scores and we demonstrate improved inference with this denoised data. Specifically, we show that replacing the quality scores with those generated by the proposed denoiser results in more accurate variant calling in general. Moreover, a consequence of the denoising is that the entropy of the produced quality scores is smaller, and thus significant compression can be achieved with respect to lossless compression of the original quality scores. We expect our results to provide a baseline for future research in denoising of quality scores. The code used in this work as well as a Supplement with all the results are available at http://web.stanford.edu/~iochoa/DCCdenoiser_CodeAndSupplement.zip.

PopGeV: a web-based large-scale population genome browser.

The development of high-throughput sequencing technology has made it possible for more and more researchers to use population sequencing data to mine genes associated with specific traits. However, the massive amounts of sequencing data have also brought new challenges to the researchers. The question of how to browse population genomic data in an easy and intuitive manner must be addressed. Web-based genome browsers allow user to conveniently view the results of genomic analyses, but heavy usage can reduce the response speed of the webpage, which limits its usefulness in the display of large-scale genome data. IndexedDB technology is a good solution to this problem; it supports web browsers and so creates local databases. In this way, data can be read from the local storage, achieving a smooth display of population genomic data. PopGeV has the following characteristics. First, it uses a new encoding method for compression of population SNP and INDEL data. IndexedDB technology is used to download the results to local storage so that users can browse the results smoothly even when the network traffic is heavy. Second, PopGeV identify similar genomic regions between two individuals based on SNP data. Population diversity indexes are calculated when comparing two populations. Third, user defined annotation information can be integrated for user-friendly mining of gene functions. Simulation shows that PopGeV can smoothly display analysis results of population genome containing over 500 individuals with 2 millions SNP data. PopGeV is available at www.soyomics.com/popgev/ yuanxh@iga.ac.cn.

CLAST: CUDA implemented large-scale alignment search tool.

BackgroundMetagenomics is a powerful methodology to study microbial communities, but it is highly dependent on nucleotide sequence similarity searching against sequence databases. Metagenomic analyses with next-generation sequencing technologies produce enormous numbers of reads from microbial communities, and many reads are derived from microbes whose genomes have not yet been sequenced, limiting the usefulness of existing sequence similarity search tools. Therefore, there is a clear need for a sequence similarity search tool that can rapidly detect weak similarity in large datasets.ResultsWe developed a tool, which we named CLAST (CUDA implemented large-scale alignment search tool), that enables analyses of millions of reads and thousands of reference genome sequences, and runs on NVIDIA Fermi architecture graphics processing units. CLAST has four main advantages over existing alignment tools. First, CLAST was capable of identifying sequence similarities ~80.8 times faster than BLAST and 9.6 times faster than BLAT. Second, CLAST executes global alignment as the default (local alignment is also an option), enabling CLAST to assign reads to taxonomic and functional groups based on evolutionarily distant nucleotide sequences with high accuracy. Third, CLAST does not need a preprocessed sequence database like Burrows–Wheeler Transform-based tools, and this enables CLAST to incorporate large, frequently updated sequence databases. Fourth, CLAST requires <2 GB of main memory, making it possible to run CLAST on a standard desktop computer or server node.ConclusionsCLAST achieved very high speed (similar to the Burrows–Wheeler Transform-based Bowtie 2 for long reads) and sensitivity (equal to BLAST, BLAT, and FR-HIT) without the need for extensive database preprocessing or a specialized computing platform. Our results demonstrate that CLAST has the potential to be one of the most powerful and realistic approaches to analyze the massive amount of sequence data from next-generation sequencing technologies.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-014-0406-y) contains supplementary material, which is available to authorized users.