Exact Word Matches Research Articles

In silico-based Safety Assessment of the EPSPS CP4 single event expressed in genetically modified crops

In silico-based Safety Assessment of the EPSPS CP4 single event expressed in genetically modified crops

Accurate multiple alignment of distantly related genome sequences using filtered spaced word matches as anchor points

MotivationMost methods for pairwise and multiple genome alignment use fast local homology search tools to identify anchor points, i.e. high-scoring local alignments of the input sequences. Sequence segments between those anchor points are then aligned with slower, more sensitive methods. Finding suitable anchor points is therefore crucial for genome sequence comparison; speed and sensitivity of genome alignment depend on the underlying anchoring methods.ResultsIn this article, we use filtered spaced word matches to generate anchor points for genome alignment. For a given binary pattern representing match and don’t-care positions, we first search for spaced-word matches, i.e. ungapped local pairwise alignments with matching nucleotides at the match positions of the pattern and possible mismatches at the don’t-care positions. Those spaced-word matches that have similarity scores above some threshold value are then extended using a standard X-drop algorithm; the resulting local alignments are used as anchor points. To evaluate this approach, we used the popular multiple-genome-alignment pipeline Mugsy and replaced the exact word matches that Mugsy uses as anchor points with our spaced-word-based anchor points. For closely related genome sequences, the two anchoring procedures lead to multiple alignments of similar quality. For distantly related genomes, however, alignments calculated with our filtered-spaced-word matches are superior to alignments produced with the original Mugsy program where exact word matches are used to find anchor points.Availability and implementation http://spacedanchor.gobics.de Supplementary information Supplementary data are available at Bioinformatics online.

Multi-Stream Word-Based Compression Algorithm for Compressed Text Search

In this article, we present a novel word-based lossless compression algorithm for text files using a semi-static model. We named this method the ‘Multi-stream word-based compression algorithm (MWCA)’ because it stores the compressed forms of the words in three individual streams depending on their frequencies in the text and stores two dictionaries and a bit vector as side information. In our experiments, MWCA produces a compression ratio of 3.23 bpc on average and 2.88 bpc for files greater than 50 MB; if a variable length encoder such as Huffman coding is used after MWCA, the given ratios are reduced to 2.65 and 2.44 bpc, respectively. MWCA supports exact word matching without decompression, and its multi-stream approach reduces the search time with respect to single-stream algorithms. Additionally, the MWCA multi-stream structure supplies the reduction in network load by requesting only the necessary streams from the database. With the advantage of its fast compressed search feature and multi-stream structure, we believe that MWCA is a good solution, especially for storing and searching big text data.

Fuzzy Bag-of-Words Model for Document Representation

One key issue in text mining and natural language processing is how to effectively represent documents using numerical vectors. One classical model is the Bag-of-Words (BoW). In a BoW-based vector representation of a document, each element denotes the normalized number of occurrence of a basis term in the document. To count the number of occurrence of a basis term, BoW conducts exact word matching, which can be regarded as a hard mapping from words to the basis term. BoW representation suffers from its intrinsic extreme sparsity, high dimensionality, and inability to capture high-level semantic meanings behind text data. To address the aforementioned issues, we propose a new document representation method named fuzzy Bag-of-Words (FBoW) in this paper. FBoW adopts a fuzzy mapping based on semantic correlation among words quantified by cosine similarity measures between word embeddings. Since word semantic matching instead of exact word string matching is used, the FBoW could encode more semantics into the numerical representation. In addition, we propose to use word clusters instead of individual words as basis terms and develop fuzzy Bag-of-WordClusters (FBoWC) models. Three variants under the framework of FBoWC are proposed based on three different similarity measures between word clusters and words, which are named as $\text{FBoWC}_{\rm mean}$ , $\text{FBoWC}_{\rm max}$ , and $\text{FBoWC}_{\rm min}$ , respectively. Document representations learned by the proposed FBoW and FBoWC are dense and able to encode high-level semantics. The task of document categorization is used to evaluate the performance of learned representation by the proposed FBoW and FBoWC methods. The results on seven real-word document classification datasets in comparison with six document representation learning methods have shown that our methods FBoW and FBoWC achieve the highest classification accuracies.

Phylogeny reconstruction based on the length distribution of k-mismatch common substrings

BackgroundVarious approaches to alignment-free sequence comparison are based on the length of exact or inexact word matches between pairs of input sequences. Haubold et al. (J Comput Biol 16:1487–1500, 2009) showed how the average number of substitutions per position between two DNA sequences can be estimated based on the average length of exact common substrings.ResultsIn this paper, we study the length distribution of k-mismatch common substrings between two sequences. We show that the number of substitutions per position can be accurately estimated from the position of a local maximum in the length distribution of their k-mismatch common substrings.

Analysis and Priorities: Developing a Modern, Effective, Regulatory Framework

Analysis and Priorities: Developing a Modern, Effective, Regulatory Framework

Efficient Parameterized Word Matching Using Bit-Parallelism and Partitioning the Text

Word matching problem is to find all the exact occurrences of a pattern P[0...m-1] in the text T[0...n-1], where P neither contains any white space nor preceded and followed by space. In the parameterized word matching problem, a given word P[0...m-1] is said to match with a sub-word t of the text T[0...n-1], if there exists a one-to-one correspondence between the symbols of P and the symbols of t. Exact Word Matching (EWM) problem has been previously solved by partitioning the text into number of tables in the pre-processing phase and then applying either brute force approach or fast hashing during the searching process. This paper presents an extension of EWM problem for parameterized word matching. It first split the text into number of tables in the pre-processing phase and then applying prev-encoding and bit-parallelism technique, Parameterized Shift-Or (PSO) during the searching phase. Experimental results show that this technique performs better than PSO.

Efficient Hybrid Semantic Text Similarity using Wordnet and a Corpus

Text similarity plays an important role in natural language processing tasks such as answering questions and summarizing text. At present, state-of-the-art text similarity algorithms rely on inefficient word pairings and/or knowledge derived from large corpora such as Wikipedia. This article evaluates previous word similarity measures on benchmark datasets and then uses a hybrid word similarity in a novel text similarity measure (TSM). The proposed TSM is based on information content and WordNet semantic relations. TSM includes exact word match, the length of both sentences in a pair, and the maximum similarity between one word and the compared text. Compared with other well-known measures, results of TSM are surpassing or comparable with the best algorithms in the literature.

Spaced words and kmacs: fast alignment-free sequence comparison based on inexact word matches.

In this article, we present a user-friendly web interface for two alignment-free sequence-comparison methods that we recently developed. Most alignment-free methods rely on exact word matches to estimate pairwise similarities or distances between the input sequences. By contrast, our new algorithms are based on inexact word matches. The first of these approaches uses the relative frequencies of so-called spaced words in the input sequences, i.e. words containing ‘don't care’ or ‘wildcard’ symbols at certain pre-defined positions. Various distance measures can then be defined on sequences based on their different spaced-word composition. Our second approach defines the distance between two sequences by estimating for each position in the first sequence the length of the longest substring at this position that also occurs in the second sequence with up to k mismatches. Both approaches take a set of deoxyribonucleic acid (DNA) or protein sequences as input and return a matrix of pairwise distance values that can be used as a starting point for clustering algorithms or distance-based phylogeny reconstruction. The two alignment-free programmes are accessible through a web interface at ‘Göttingen Bioinformatics Compute Server (GOBICS)’: http://spaced.gobics.de http://kmacs.gobics.de and the source codes can be downloaded.

Kmacs: the k-mismatch average common substring approach to alignment-free sequence comparison.

Motivation: Alignment-based methods for sequence analysis have various limitations if large datasets are to be analysed. Therefore, alignment-free approaches have become popular in recent years. One of the best known alignment-free methods is the average common substring approach that defines a distance measure on sequences based on the average length of longest common words between them. Herein, we generalize this approach by considering longest common substrings with k mismatches. We present a greedy heuristic to approximate the length of such k-mismatch substrings, and we describe kmacs, an efficient implementation of this idea based on generalized enhanced suffix arrays.Results: To evaluate the performance of our approach, we applied it to phylogeny reconstruction using a large number of DNA and protein sequence sets. In most cases, phylogenetic trees calculated with kmacs were more accurate than trees produced with established alignment-free methods that are based on exact word matches. Especially on protein sequences, our method seems to be superior. On simulated protein families, kmacs even outperformed a classical approach to phylogeny reconstruction using multiple alignment and maximum likelihood.Availability and implementation: kmacs is implemented in C++, and the source code is freely available at http://kmacs.gobics.de/Contact: chris.leimeister@stud.uni-goettingen.deSupplementary information: Supplementary data are available at Bioinformatics online.

The Distribution of Word Matches Between Markovian Sequences with Periodic Boundary Conditions

Word match counts have traditionally been proposed as an alignment-free measure of similarity for biological sequences. The D(2) statistic, which simply counts the number of exact word matches between two sequences, is a useful test bed for developing rigorous mathematical results, which can then be extended to more biologically useful measures. The distributional properties of the D(2) statistic under the null hypothesis of identically and independently distributed letters have been studied extensively, but no comprehensive study of the D(2) distribution for biologically more realistic higher-order Markovian sequences exists. Here we derive exact formulas for the mean and variance of the D(2) statistic for Markovian sequences of any order, and demonstrate through Monte Carlo simulations that the entire distribution is accurately characterized by a Pólya-Aeppli distribution for sequence lengths of biological interest. The approach is novel in that Markovian dependency is defined for sequences with periodic boundary conditions, and this enables exact analytic formulas for the mean and variance to be derived. We also carry out a preliminary comparison between the approximate D(2) distribution computed with the theoretical mean and variance under a Markovian hypothesis and an empirical D(2) distribution from the human genome.

Characterizing the D2 Statistic: Word Matches in Biological Sequences

Word matches are often used in sequence comparison methods, either as a measure of sequence similarity or in the first search steps of algorithms such as BLAST or BLAT. The D2 statistic is the number of matches of words of k letters between two sequences. Recent advances have been made in the characterization of this statistic and in the approximation of its distribution. Here, these results are extended to the case of approximate word matches. We compute the exact value of the variance of the D2 statistic for the case of a uniform letter distribution, and introduce a method to provide accurate approximations of the variance in the remaining cases. This enables the distribution of D2 to be approximated for typical situations arising in biological research. We apply these results to the identification of cis-regulatory modules, and show that this method detects such sequences with a high accuracy. The ability to approximate the distribution of D2 for both exact and approximate word matches will enable the use of this statistic in a more precise manner for sequence comparison, database searches, and identification of transcription factor binding sites.

Asymptotic behaviour and optimal word size for exact and approximate word matches between random sequences

AbstractThe present work is concerned with a fast and accurate sequence comparison method: the count of the number of words of length k letters shared by two sequences, also known as the D2 statistic. We link recent theoretical advances in the characterization of D2 asymptotic distributions with applications to biological sequences. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

A corpus analysis approach for automatic query expansion and its extension to multiple databases

Searching online text collections can be both rewarding and frustrating. While valuable information can be found, typically many irrelevant documents are also retrieved, while many relevant ones are missed. Terminology mismatches between the user's query and document contents are a main cause of retrieval failures. Expanding a user's query with related words can improve search performances, but finding and using related words is an open problem. This research uses corpus analysis techniques to automatically discover similar words directly from the contents of the databases which are not tagged with part-of-speech labels. Using these similarities, user queries are automatically expanded, resulting in conceptual retrieval rather than requiring exact word matches between queries and documents. We are able to achieve a 7.6% improvement for TREC 5 queries and up to a 28.5% improvement on the narrow-domain Cystic Fibrosis collection. This work has been extended to multidatabase collections where each subdatabase has a collection-specific similarity matrix associated with it. If the best matrix is selected, substantial search improvements are possible. Various techniques to select the appropriate matrix for a particular query are analyzed, and a 4.8% improvement in the results is validated.

Foreign Accent, Comprehensibility, and Intelligibility in the Speech of Second Language Learners

This study examines the interrelationships among accentedness, perceived comprehensibility, and intelligi bility in the speech of L2 learners. Eighteen native speak ers (NSs) of English listened to excerpts of extemporaneous English speech produced by 10 Mandarin NSs and two English NSs. We asked the listeners to transcribe the utterances in standard orthography and to rate them for degree of foreign‐accentedness and comprehensibility on 9‐ point scales. We assigned the transcriptions intelligibility scores on the basis of exact word matches. Although the utterances tended to be highly intelligible and highly rated for comprehensibility, the accent judgment scores ranged widely, with a noteworthy proportion of scores at the “heavily‐accented” end of the scale. We calculated Pearson correlations for each listener's intelligibility, accentedness, and comprehensibility scores and the phonetic, phonemic, and grammatical errors in the stimuli, as well as goodness of intonation ratings. Most listeners showed significant correlations between accentedness and errors, fewer lis teners showed correlations between accentedness and per ceived comprehensibility, and fewer still showed a rela tionship between accentedness and intelligibility. The findings suggest that although strength of foreign accent is correlated with perceived comprehensibility and intelligibility, a strong foreign accent does not necessarily reduce the comprehensibility or intelligibility of L2 speech.

The perception of rhythm and word boundaries in noise-masked speech.

The present experiment tested the suggestion that human listeners may exploit durational information in speech to parse continuous utterances into words. Listeners were presented with six-syllable unpredictable utterances under noise-masking, and were required to judge between alternative word strings as to which best matched the rhythm of the masked utterances. For each utterance there were four alternative strings: (a) an exact rhythmic and word boundary match, (b) a rhythmic mismatch, and (c) two utterances with the same rhythm as the masked utterance, but different word boundary locations. Listeners were clearly able to perceive the rhythm of the masked utterances: The rhythmic mismatch was chosen significantly less often than any other alternative. Within the three rhythmically matched alternatives, the exact match was chosen significantly more often than either word boundary mismatch. Thus, listeners both perceived speech rhythm and used durational cues effectively to locate the position of word boundaries.

An experiment with semantics and Goffman's indirect method

Goffman's indirect method partitions a group of documents into classes based on the number of attributes each pair of documents have in common. The classes produced depend on the threshold value chosen to generate the classes. The indirect method was used here to classify a set of one hundred and nine medical documents, using the index terms assigned by Index Medicus. The attributes used in the indirect method were defined in three ways. First, only exact matches between index terms were considered as attributes in common, resulting in a very fine partition. Next, MeSH tree structure was used to approximate the relationships between index terms, so that the attributes in common were words within a small semantic distance of each other. This produced a broader partition, but several of the documents related in the exact word match dropped below the threshold. To compensate for this, a third definition was used, to give extra weight to exact match relationships. This produced a reasonable classification with all classes nameable. Graphic representations of the three partitions illustrate the structure of the set of documents.