K-mer Distribution Research Articles

The Genome of Caenorhabditis bovis.

The free-living nematode Caenorhabditis elegans is akey laboratory model for metazoan biology. C.elegans has also become a model for parasitic nematodes despite being only distantly related to most parasitic species. All of the ∼65 Caenorhabditis species currently in culture are free-living, with most having been isolated from decaying plant or fungal matter. Caenorhabditis bovis is a particularly unusual species that has been isolated several times from the inflamed ears of Zebu cattle in Eastern Africa, where it is associated with thedisease bovine parasitic otitis. C.bovis is therefore of particular interest to researchers interested in the evolution of nematode parasitism. However, as C.bovis is not in laboratory culture, it remains little studied. Here, by sampling livestock markets and slaughterhouses in Western Kenya, we successfully reisolated C.bovis from the ear of adult female Zebu. We sequenced the genome of C.bovis using the Oxford Nanopore MinION platform in a nearby field laboratory and used the data to generate a chromosome-scaledraft genome sequence. We exploited this draft genome sequence to reconstruct the phylogenetic relationships of C.bovis to other Caenorhabditis species and reveal the changes in genome size and content that have occurred during its evolution. We also identified expansions in several gene families that have been implicated in parasitism in other nematode species. The high-quality draft genome and our analyses thereof represent a significant advancement in our understanding of this unusual Caenorhabditis species.

STRAIN: an R package for multi-locus sequence typing from whole genome sequencing data

BackgroundMulti-locus sequence typing (MLST) is a standard typing technique used to associate a sequence type (ST) to a bacterial isolate. When the output of whole genome sequencing (WGS) of a sample is available the ST can be assigned directly processing the read-set.Current approaches employ reads mapping (SRST2) against the MLST loci, k-mer distribution (stringMLST), selective assembly (GRAbB) or whole genome assembly (BIGSdb) followed by BLASTn sequence query.Here we present STRAIN (ST Reduced Assembly IdentificatioN), an R package that implements a hybrid strategy between assembly and mapping of the reads to assign the ST to an isolate starting from its read-sets.ResultsAnalysis of 540 publicly accessible Illumina read sets showed STRAIN to be more accurate at correct allele assignment and new alleles identification compared to SRTS2, stringMLST and GRAbB. STRAIN assigned correctly 3666 out of 3780 alleles (capability to identify correct alleles 97%) and, when presented with samples containing new alleles, identified them in 3730 out of 3780 STs (capability to identify new alleles 98.7%) of the cases. On the same dataset the other tested tools achieved lower capability to identify correct alleles (from 28.5 to 96.9%) and lower capability to identify new alleles (from 1.1 to 97.1%).ConclusionsSTRAIN is a new accurate method to assign the alleles and ST to an isolate by processing the raw reads output of WGS. STRAIN is also able to retrieve new allele sequences if present. Capability to identify correct and new STs/alleles, evaluated on a benchmark dataset, are higher than other existing methods. STRAIN is designed for single allele typing as well as MLST. Its implementation in R makes allele and ST assignment simple, direct and prompt to be integrated in wider pipeline of downstream bioinformatics analyses.

LtrDetector: A tool-suite for detecting long terminal repeat retrotransposons de-novo

BackgroundLong terminal repeat retrotransposons are the most abundant transposons in plants. They play important roles in alternative splicing, recombination, gene regulation, and defense mechanisms. Large-scale sequencing projects for plant genomes are currently underway. Software tools are important for annotating long terminal repeat retrotransposons in these newly available genomes. However, the available tools are not very sensitive to known elements and perform inconsistently on different genomes. Some are hard to install or obsolete. They may struggle to process large plant genomes. None can be executed in parallel out of the box and very few have features to support visual review of new elements. To overcome these limitations, we developed LtrDetector, which uses techniques inspired by signal-processing.ResultsWe compared LtrDetector to LTR_Finder and LTRharvest, the two most successful predecessor tools, on six plant genomes. For each organism, we constructed a ground truth data set based on queries from a consensus sequence database. According to this evaluation, LtrDetector was the most sensitive tool, achieving 16–23% improvement in sensitivity over LTRharvest and 21% improvement over LTR_Finder. All three tools had low false positive rates, with LtrDetector achieving 98.2% precision, in between its two competitors. Overall, LtrDetector provides the best compromise between high sensitivity and low false positive rate while requiring moderate time and utilizing memory available on personal computers.ConclusionsLtrDetector uses a novel methodology revolving around k-mer distributions, which allows it to produce high-quality results using relatively lightweight procedures. It is easy to install and use. It is not species specific, performing well using its default parameters on genomes of varying size and repeat content. It is automatically configured for parallel execution and runs efficiently on an ordinary personal computer. It includes a k-mer scores visualization tool to facilitate manual review of the identified elements. These features make LtrDetector an attractive tool for future annotation projects involving long terminal repeat retrotransposons.

Alignment-free approaches for predicting novel Nuclear Mitochondrial Segments (NUMTs) in the human genome

The nuclear human genome harbors sequences of mitochondrial origin, indicating an ancestral transfer of DNA from the mitogenome. Several Nuclear Mitochondrial Segments (NUMTs) have been detected by alignment-based sequence similarity search, as implemented in the Basic Local Alignment Search Tool (BLAST). Identifying NUMTs is important for the comprehensive annotation and understanding of the human genome. Here we explore the possibility of detecting NUMTs in the human genome by alignment-free sequence similarity search, such as k-mers (k-tuples, k-grams, oligos of length k) distributions. We find that when k=6 or larger, the k-mer approach and BLAST search produce almost identical results, e.g., detect the same set of NUMTs longer than 3 kb. However, when k=5 or k=4, certain signals are only detected by the alignment-free approach, and these may indicate yet unrecognized, and potentially more ancestral NUMTs. We introduce a “Manhattan plot” style representation of NUMT predictions across the genome, which are calculated based on the reciprocal of the Jensen-Shannon divergence between the nuclear and mitochondrial k-mer frequencies. The further inspection of the k-mer-based NUMT predictions however shows that most of them contain long-terminal-repeat (LTR) annotations, whereas BLAST-based NUMT predictions do not. Thus, similarity of the mitogenome to LTR sequences is recognized, which we validate by finding the mitochondrial k-mer distribution closer to those for transposable sequences and specifically, close to some types of LTR.

Degenerative Expansion of a Young Supergene.

Long-term suppression of recombination ultimately leads to gene loss, as demonstrated by the depauperate Y and W chromosomes of long-established pairs of XY and ZW chromosomes. The young social supergene of the Solenopsis invicta red fire ant provides a powerful system to examine the effects of suppressed recombination over a shorter timescale. The two variants of this supergene are carried by a pair of heteromorphic chromosomes, referred to as the social B and social b (SB and Sb) chromosomes. The Sb variant of this supergene changes colony social organization and has an inheritance pattern similar to a Y or W chromosome because it is unable to recombine. We used high-resolution optical mapping, k-mer distribution analysis, and quantification of repetitive elements on haploid ants carrying alternate variants of this young supergene region. We find that instead of shrinking, the Sb variant of the supergene has increased in length by more than 30%. Surprisingly, only a portion of this length increase is due to consistent increases in the frequency of particular classes of repetitive elements. Instead, haplotypes of this supergene variant differ dramatically in the amounts of other repetitive elements, indicating that the accumulation of repetitive elements is a heterogeneous and dynamic process. This is the first comprehensive demonstration of degenerative expansion in an animal and shows that it occurs through nonlinear processes during the early evolution of a region of suppressed recombination.

Phylogenetic analysis of protein sequences based on a novel k-mer natural vector method

Based on the k-mer model for protein sequence, a novel k-mer natural vector method is proposed to characterize the features of k-mers in a protein sequence, in which the numbers and distributions of k-mers are considered. It is proved that the relationship between a protein sequence and its k-mer natural vector is one-to-one. Phylogenetic analysis of protein sequences therefore can be easily performed without requiring evolutionary models or human intervention. In addition, there exists no a criterion to choose a suitable k, and k has a great influence on obtaining results as well as computational complexity. In this paper, a compound k-mer natural vector is utilized to quantify each protein sequence. The results gotten from phylogenetic analysis on three protein datasets demonstrate that our new method can precisely describe the evolutionary relationships of proteins, and greatly heighten the computing efficiency.

P103 The promise of cost-efficient full-length HLA class I genotyping: Advances using nanopore sequencing

Aim Nanopore based sequencing has seen rapid advancement in the recent years with iterations in pore structure, sequencing chemistries and base callers. This has led to increasingly accurate sequencing of extremely long DNA molecules. Since current HLA genotyping algorithms are not optimized for nanopore data we developed a genotyping algorithm based on read grouping and subsequent mapping to a reduced reference database. Methods High raw sequencing errors present a challenge for genotyping based on direct mapping to a reference database. Our algorithm, Poretyper, obviates this initial mapping by first grouping the raw reads based on the distributions of k-mers within each read. A multiple sequence alignment derived from the groups of raw reads results in a set of consensus sequences which represent potential alleles. These consensus sequences are then used to create a culled reference database dramatically reducing the search space, thus reducing artefactual raw read mappings. Results HLA-A, HLA-B and HLA-C for a hundred samples drawn from the DKMS donor registry were sequenced using the MinION with R9.5 chemistry. All hundred samples were then genotyped using Poretyper and existing G-group pretypings could be recapitulated failing only for those alleles where no full length sequences were available. Conclusions Nanopore sequencing presents a viable and accurate platform for cost-efficient full-length HLA Class I genotyping. For those alleles where full length reference sequences are not available, an in silico extension of such allele sequences using the full-length sequence of the next closest allele presents a viable approach for full-length genotyping.

MicroPheno: predicting environments and host phenotypes from 16S rRNA gene sequencing using a k-mer based representation of shallow sub-samples.

Author(s): Asgari, Ehsaneddin; Garakani, Kiavash; McHardy, Alice C; Mofrad, Mohammad RK | Abstract: MotivationMicrobial communities play important roles in the function and maintenance of various biosystems, ranging from the human body to the environment. A major challenge in microbiome research is the classification of microbial communities of different environments or host phenotypes. The most common and cost-effective approach for such studies to date is 16S rRNA gene sequencing. Recent falls in sequencing costs have increased the demand for simple, efficient and accurate methods for rapid detection or diagnosis with proved applications in medicine, agriculture and forensic science. We describe a reference- and alignment-free approach for predicting environments and host phenotypes from 16S rRNA gene sequencing based on k-mer representations that benefits from a bootstrapping framework for investigating the sufficiency of shallow sub-samples. Deep learning methods as well as classical approaches were explored for predicting environments and host phenotypes.ResultsA k-mer distribution of shallow sub-samples outperformed Operational Taxonomic Unit (OTU) features in the tasks of body-site identification and Crohn's disease prediction. Aside from being more accurate, using k-mer features in shallow sub-samples allows (i) skipping computationally costly sequence alignments required in OTU-picking and (ii) provided a proof of concept for the sufficiency of shallow and short-length 16S rRNA sequencing for phenotype prediction. In addition, k-mer features predicted representative 16S rRNA gene sequences of 18 ecological environments, and 5 organismal environments with high macro-F1 scores of 0.88 and 0.87. For large datasets, deep learning outperformed classical methods such as Random Forest and Support Vector Machine.Availability and implementationThe software and datasets are available at https://llp.berkeley.edu/micropheno.Supplementary informationSupplementary data are available at Bioinformatics online.

MicroPheno: predicting environments and host phenotypes from 16S rRNA gene sequencing using a k-mer based representation of shallow sub-samples.

MotivationMicrobial communities play important roles in the function and maintenance of various biosystems, ranging from the human body to the environment. A major challenge in microbiome research is the classification of microbial communities of different environments or host phenotypes. The most common and cost-effective approach for such studies to date is 16S rRNA gene sequencing. Recent falls in sequencing costs have increased the demand for simple, efficient and accurate methods for rapid detection or diagnosis with proved applications in medicine, agriculture and forensic science. We describe a reference- and alignment-free approach for predicting environments and host phenotypes from 16S rRNA gene sequencing based on k-mer representations that benefits from a bootstrapping framework for investigating the sufficiency of shallow sub-samples. Deep learning methods as well as classical approaches were explored for predicting environments and host phenotypes.ResultsA k-mer distribution of shallow sub-samples outperformed Operational Taxonomic Unit (OTU) features in the tasks of body-site identification and Crohn’s disease prediction. Aside from being more accurate, using k-mer features in shallow sub-samples allows (i) skipping computationally costly sequence alignments required in OTU-picking and (ii) provided a proof of concept for the sufficiency of shallow and short-length 16S rRNA sequencing for phenotype prediction. In addition, k-mer features predicted representative 16S rRNA gene sequences of 18 ecological environments, and 5 organismal environments with high macro-F1 scores of 0.88 and 0.87. For large datasets, deep learning outperformed classical methods such as Random Forest and Support Vector Machine.Availability and implementationThe software and datasets are available at https://llp.berkeley.edu/micropheno.Supplementary information Supplementary data are available at Bioinformatics online.

A draft genome sequence of the rose black spot fungus Diplocarpon rosae reveals a high degree of genome duplication.

BackgroundBlack spot is one of the most severe and damaging diseases of garden roses. We present the draft genome sequence of its causative agent Diplocarpon rosae as a working tool to generate molecular markers and to analyze functional and structural characteristics of this fungus.ResultsThe isolate DortE4 was sequenced with 191x coverage of different read types which were assembled into 2457 scaffolds. By evidence supported genome annotation with the MAKER pipeline 14,004 gene models were predicted and transcriptomic data indicated that 88.5% of them are expressed during the early stages of infection. Analyses of k-mer distributions resulted in unexpectedly large genome size estimations between 72.5 and 91.4 Mb, which cannot be attributed to its repeat structure and content of transposable elements alone, factors explaining such differences in other fungal genomes. In contrast, different lines of evidences demonstrate that a huge proportion (approximately 80%) of genes are duplicated, which might indicate a whole genome duplication event. By PCR-RFLP analysis of six paralogous gene pairs of BUSCO orthologs, which are expected to be single copy genes, we could show experimentally that the duplication is not due to technical error and that not all isolates tested possess all of the paralogs.ConclusionsThe presented genome sequence is still a fragmented draft but contains almost the complete gene space. Therefore, it provides a useful working tool to study the interaction of D. rosae with the host and the influence of a genome duplication outside of the model yeast in the background of a phytopathogen.

MetaGen: reference-free learning with multiple metagenomic samples

A major goal of metagenomics is to identify and study the entire collection of microbial species in a set of targeted samples. We describe a statistical metagenomic algorithm that simultaneously identifies microbial species and estimates their abundances without using reference genomes. As a trade-off, we require multiple metagenomic samples, usually ≥10 samples, to get highly accurate binning results. Compared to reference-free methods based primarily on k-mer distributions or coverage information, the proposed approach achieves a higher species binning accuracy and is particularly powerful when sequencing coverage is low. We demonstrated the performance of this new method through both simulation and real metagenomic studies. The MetaGen software is available at https://github.com/BioAlgs/MetaGen.

KmerPyramid: an interactive visualization tool for nucleobase and k-mer frequencies.

Bioinformatics methods often incorporate the frequency distribution of nulecobases or k-mers in DNA or RNA sequences, for example as part of metagenomic or phylogenetic analysis. Because the frequency matrix with sequences in the rows and nucleobases in the columns is multi-dimensional it is hard to visualize. We present the R-package 'kmerPyramid' that allows to display each sequence, based on its nucleobase or k-mer distribution projected to the space of principal components, as a point within a 3-dimensional, interactive pyramid. Using the computer mouse, the user can turn the pyramid's axes, zoom in and out and identify individual points. Additionally, the package provides the k-mer frequency matrices of about 2000 bacteria and 5000 virus reference sequences calculated from the NCBI RefSeq genbank. The 'kmerPyramid' can particularly be used for visualization of intra- and inter species differences. The R-package 'kmerPyramid' is available from the GitHub website at https://github.com/jkruppa/kmerPyramid. klaus.jung@tiho-hannover.de. Supplementary data are available at Bioinformatics online.

Mismatch repair (MMR) detection in urothelial carcinoma (UC) and correlation with immune checkpoint blockade (ICB) response.

4511 Background: High mutation burden correlates with response to ICB in UC. Loss of function alterations or epigenetic silencing of MMR genes results in MMR deficient (MMR-D) UC, leading to a microsatellite instability (MSI) mutation signature. We used a CLIA-certified pipeline (MSISensor) to interrogate Next Generation sequencing (NGS) data from UC tumors to identify MMR-D patients (pts). We correlated MMR-D with mutation load and response to ICB. Methods: 447 tumors from 424 UC pts underwent prospective NGS using the MSK-IMPACT exon capture assay and genomic interrogation of microsatellite (MS) sites using MSIsensor, which assesses the number/length of MS within the targeted regions of tumor-normal sample pairs. Loci are considered unstable (somatic) if k-mer distributions are significantly different between tumor and matched normal using a standard multiple testing correction of χ2 p-values. The fraction of unstable sites is reported as an MSIsensor score. MSI high tumors have scores >10 while <3 are denoted MS stable. Scores from 3-10 were categorized as MS intermediate. Results: Thirteen pts (3%) had an MSI score >10 and a median mutation count of 52 (36.5-73.5) vs 8 (5-13) in 410 non-MMR-D pts (p<0.01). Ten pts (71%) had upper tract UC. Of 9 pts with germline sequencing performed, 8 (89%) had heritable loss of function mutations in MMR proteins (Lynch syndrome, LS). One pt had a somatic MSH2 mutation. Fifteen pts had MS scores from 3-10: 3 had LS, one a BRCA1 germline alteration, and 9 did not have germline testing available. Two pts with MSI scores <3 had extremely high mutation loads (213 and 414) and both had POLE mutations. Five pts received ICB therapy for metastatic and all achieved near-complete or complete responses. No MMR-D pt has died at 27 months follow-up vs 125 non-MMR-D pts (p=0.014). Conclusions: The MSI Sensor assay can discriminate MSI high from MMR proficient UC. While rare, MMR-D UC is characterized by a high mutation load, strong association with Lynch syndrome, and durable responses to ICB, similar to data in colon cancer. An MMR-D signature should trigger genetic testing for Lynch syndrome. ICB should be considered early in the treatment course of patients with MMR-D metastatic UC.

Monte Carlo simulation of evaporation-driven self-assembly in suspensions of colloidal rods.

The vertical drying of a colloidal film containing rodlike particles was studied by means of kinetic Monte Carlo (MC) simulation. The problem was approached using a two-dimensional square lattice, and the rods were represented as linear k-mers (i.e., particles occupying k adjacent sites). The initial state before drying was produced using a model of random sequential adsorption (RSA) with isotropic orientations of the k-mers (orientation of the k-mers along horizontal x and vertical y directions are equiprobable). In the RSA model, overlapping of the k-mers is forbidden. During the evaporation, an upper interface falls with a linear velocity of u in the vertical direction and the k-mers undergo translation Brownian motion. The MC simulations were run at different initial concentrations, p_{i}, (p_{i}∈[0,p_{j}], where p_{j} is the jamming concentration), lengths of k-mers (k∈[1,12]), and solvent evaporation rates, u. For completely dried films, the spatial distributions of k-mers and their electrical conductivities in both x and y directions were examined. Significant evaporation-driven self-assembly and orientation stratification of the k-mers oriented along the x and y directions were observed. The extent of stratification increased with increasing value of k. The anisotropy of the electrical conductivity of the film can be finely regulated by changes in the values of p_{i}, k, and u.

A novel alignment-free method for detection of lateral genetic transfer based on TF-IDF.

Lateral genetic transfer (LGT) plays an important role in the evolution of microbes. Existing computational methods for detecting genomic regions of putative lateral origin scale poorly to large data. Here, we propose a novel method based on TF-IDF (Term Frequency-Inverse Document Frequency) statistics to detect not only regions of lateral origin, but also their origin and direction of transfer, in sets of hierarchically structured nucleotide or protein sequences. This approach is based on the frequency distributions of k-mers in the sequences. If a set of contiguous k-mers appears sufficiently more frequently in another phyletic group than in its own, we infer that they have been transferred from the first group to the second. We performed rigorous tests of TF-IDF using simulated and empirical datasets. With the simulated data, we tested our method under different parameter settings for sequence length, substitution rate between and within groups and post-LGT, deletion rate, length of transferred region and k size, and found that we can detect LGT events with high precision and recall. Our method performs better than an established method, ALFY, which has high recall but low precision. Our method is efficient, with runtime increasing approximately linearly with sequence length.

Quality control of microbiota metagenomics by k-mer analysis.

BackgroundThe biological and clinical consequences of the tight interactions between host and microbiota are rapidly being unraveled by next generation sequencing technologies and sophisticated bioinformatics, also referred to as microbiota metagenomics. The recent success of metagenomics has created a demand to rapidly apply the technology to large case–control cohort studies and to studies of microbiota from various habitats, including habitats relatively poor in microbes. It is therefore of foremost importance to enable a robust and rapid quality assessment of metagenomic data from samples that challenge present technological limits (sample numbers and size). Here we demonstrate that the distribution of overlapping k-mers of metagenome sequence data predicts sequence quality as defined by gene distribution and efficiency of sequence mapping to a reference gene catalogue.ResultsWe used serial dilutions of gut microbiota metagenomic datasets to generate well-defined high to low quality metagenomes. We also analyzed a collection of 52 microbiota-derived metagenomes. We demonstrate that k-mer distributions of metagenomic sequence data identify sequence contaminations, such as sequences derived from “empty” ligation products. Of note, k-mer distributions were also able to predict the frequency of sequences mapping to a reference gene catalogue not only for the well-defined serial dilution datasets, but also for 52 human gut microbiota derived metagenomic datasets.ConclusionsWe propose that k-mer analysis of raw metagenome sequence reads should be implemented as a first quality assessment prior to more extensive bioinformatics analysis, such as sequence filtering and gene mapping. With the rising demand for metagenomic analysis of microbiota it is crucial to provide tools for rapid and efficient decision making. This will eventually lead to a faster turn-around time, improved analytical quality including sample quality metrics and a significant cost reduction. Finally, improved quality assessment will have a major impact on the robustness of biological and clinical conclusions drawn from metagenomic studies.Electronic supplementary materialThe online version of this article (doi:10.1186/s12864-015-1406-7) contains supplementary material, which is available to authorized users.

K-mer natural vector and its application to the phylogenetic analysis of genetic sequences.

Based on the well-known k-mer model, we propose a k-mer natural vector model for representing a genetic sequence based on the numbers and distributions of k-mers in the sequence. We show that there exists a one-to-one correspondence between a genetic sequence and its associated k-mer natural vector. The k-mer natural vector method can be easily and quickly used to perform phylogenetic analysis of genetic sequences without requiring evolutionary models or human intervention. Whole or partial genomes can be handled more effective with our proposed method. It is applied to the phylogenetic analysis of genetic sequences, and the obtaining results fully demonstrate that the k-mer natural vector method is a very powerful tool for analysing and annotating genetic sequences and determining evolutionary relationships both in terms of accuracy and efficiency.

Diminishing return for increased Mappability with longer sequencing reads: implications of the k-mer distributions in the human genome

BackgroundThe amount of non-unique sequence (non-singletons) in a genome directly affects the difficulty of read alignment to a reference assembly for high throughput-sequencing data. Although a longer read is more likely to be uniquely mapped to the reference genome, a quantitative analysis of the influence of read lengths on mappability has been lacking. To address this question, we evaluate the k-mer distribution of the human reference genome. The k-mer frequency is determined for k ranging from 20 bp to 1000 bp.ResultsWe observe that the proportion of non-singletons k-mers decreases slowly with increasing k, and can be fitted by piecewise power-law functions with different exponents at different ranges of k. A slower decay at greater values for k indicates more limited gains in mappability for read lengths between 200 bp and 1000 bp. The frequency distributions of k-mers exhibit long tails with a power-law-like trend, and rank frequency plots exhibit a concave Zipf’s curve. The most frequent 1000-mers comprise 172 regions, which include four large stretches on chromosomes 1 and X, containing genes of biomedical relevance. Comparison with other databases indicates that the 172 regions can be broadly classified into two types: those containing LINE transposable elements and those containing segmental duplications.ConclusionRead mappability as measured by the proportion of singletons increases steadily up to the length scale around 200 bp. When read length increases above 200 bp, smaller gains in mappability are expected. Moreover, the proportion of non-singletons decreases with read lengths much slower than linear. Even a read length of 1000 bp would not allow the unique alignment of reads for many coding regions of human genes. A mix of techniques will be needed for efficiently producing high-quality data that cover the complete human genome.

Reference-Free Validation of Short Read Data

BackgroundHigh-throughput DNA sequencing techniques offer the ability to rapidly and cheaply sequence material such as whole genomes. However, the short-read data produced by these techniques can be biased or compromised at several stages in the sequencing process; the sources and properties of some of these biases are not always known. Accurate assessment of bias is required for experimental quality control, genome assembly, and interpretation of coverage results. An additional challenge is that, for new genomes or material from an unidentified source, there may be no reference available against which the reads can be checked.ResultsWe propose analytical methods for identifying biases in a collection of short reads, without recourse to a reference. These, in conjunction with existing approaches, comprise a methodology that can be used to quantify the quality of a set of reads. Our methods involve use of three different measures: analysis of base calls; analysis of k-mers; and analysis of distributions of k-mers. We apply our methodology to wide range of short read data and show that, surprisingly, strong biases appear to be present. These include gross overrepresentation of some poly-base sequences, per-position biases towards some bases, and apparent preferences for some starting positions over others.ConclusionsThe existence of biases in short read data is known, but they appear to be greater and more diverse than identified in previous literature. Statistical analysis of a set of short reads can help identify issues prior to assembly or resequencing, and should help guide chemical or statistical methods for bias rectification.

Unsupervised statistical clustering of environmental shotgun sequences

BackgroundThe development of effective environmental shotgun sequence binning methods remains an ongoing challenge in algorithmic analysis of metagenomic data. While previous methods have focused primarily on supervised learning involving extrinsic data, a first-principles statistical model combined with a self-training fitting method has not yet been developed.ResultsWe derive an unsupervised, maximum-likelihood formalism for clustering short sequences by their taxonomic origin on the basis of their k-mer distributions. The formalism is implemented using a Markov Chain Monte Carlo approach in a k-mer feature space. We introduce a space transformation that reduces the dimensionality of the feature space and a genomic fragment divergence measure that strongly correlates with the method's performance. Pairwise analysis of over 1000 completely sequenced genomes reveals that the vast majority of genomes have sufficient genomic fragment divergence to be amenable for binning using the present formalism. Using a high-performance implementation, the binner is able to classify fragments as short as 400 nt with accuracy over 90% in simulations of low-complexity communities of 2 to 10 species, given sufficient genomic fragment divergence. The method is available as an open source package called LikelyBin.ConclusionAn unsupervised binning method based on statistical signatures of short environmental sequences is a viable stand-alone binning method for low complexity samples. For medium and high complexity samples, we discuss the possibility of combining the current method with other methods as part of an iterative process to enhance the resolving power of sorting reads into taxonomic and/or functional bins.