Approximate String Matching Methods Research Articles

Predictive Quantization and Symbolic Dynamics

Capturing long-term statistics of signals and time series is important for modeling recurrent phenomena, especially when such recurrences are a-periodic and can be characterized by the approximate repetition of variable length motifs, such as patterns in human gestures and trends in financial time series or musical melodies. Regressive and auto-regressive models that are common in such problems, both analytically derived and neural network-based, often suffer from limited memory or tend to accumulate errors, making them sensitive during training. Moreover, such models often assume stationary signal statistics, which makes it difficult to deal with switching regimes or conditional signal dynamics. In this paper, we describe a method for time series modeling that is based on adaptive symbolization that maximizes the predictive information of the resulting sequence. Using approximate string-matching methods, the initial vectorized sequence is quantized into a discrete representation with a variable quantization threshold. Finding an optimal signal embedding is formulated in terms of a predictive bottleneck problem that takes into account the trade-off between representation and prediction accuracy. Several downstream applications based on discrete representation are described in this paper, which includes an analysis of the symbolic dynamics of recurrence statistics, motif extraction, segmentation, query matching, and the estimation of transfer entropy between parallel signals.

Designing a word recommendation application using the Levenshtein Distance algorithm

Good scriptwriting or reporting requires a high level of accuracy. The basic problem is that the level of accuracy of the authors is not the same. The low level of accuracy allows for mistyping of words in a sentence. Typing errors caused the word to become non-standard. Even worse, the word became meaningless. In this case, the recommendation application serves to provide word-writing recommendations in case of a typing error. This application can reduce the error rate of the writer when typing. One method to improve word spelling is Approximate String Matching. This method applies an approach to the string search process. The Levenshtein Distance algorithm is a part of the Approximate String-Matching method. This method, firstly, is necessary to go through the preprocessing stage to correct an incorrectly written word using the Levenshtein Distance algorithm. The application testing phase uses ten texts composed of 100 words, ten texts composed of 100 to 250 words, and ten texts composed of 250 to 500 words. The average accuracy rate of these test results was 95%, 94%, and 90%.

Supporting Uncertain Predicates in DBMS Using Approximate String Matching and Probabilistic Databases

Current relational database systems are deterministic in nature and lack the support for approximate matching. The result of approximate matching would be the tuples annotated with the percentage of similarity but the existing relational database system can not process these similarity scores further. In this paper, we propose a system to support approximate matching in the DBMS field. We introduce a `≈' (uncertain predicate operator) for approximate matching and devise a novel formula to calculate the similarity scores. Instead of returning an empty answer set in case of no match, our system gives ranked results thereby providing a glance at existing tuples closely matching with the queried literals. Two variants of the `≈' operator are also introduced for numeric data: `≈+' for higher-the-better and `≈-' for lower-the-better cases. Efficient approximate string matching methods are proposed for matching string-type data whereas numeric closeness is used for other types of data (date, time, and number). We also provide results of our system taken over several sample queries that illustrate the significance of our system. All experiments are performed using the MySQL database, whereas the IMDb movie database and European Football database are used as sample datasets.

Speeding Up Large-Scale Next Generation Sequencing Data Analysis with pBWA

Newly available DNA sequencing technologies can generate billions of DNA sequences in a single machine run, making it feasible to obtain an individual’s entire genome sequences very quickly and at a very affordable cost. These personal genome sequences can be used to identify genetic variations associated with variable traits, but they must first be aligned to a reference genome sequence using computer algorithms that make use of approximate string matching methods. The process of aligning a very large number of short sequence reads is computationally expensive and has led to the development of many sequence mapping programs. While new software, such as BWA, SOAP2, and Bowtie, efficiently align large numbers of short DNA sequences, we proposed that parallel computing would provide additional speedup in data processing to accommodate the rapidly increasing sequencing throughput. For this purpose, we developed pBWA, an efficient parallel version of BWA, based on the Open MPI library. pBWA retains the multi-threading capability provided by BWA while adding efficient parallelization for its core alignment functions. We have shown that pBWA’s wall-time speedup is bounded only by the size of the parallel system. pBWA can run in both parallel and multi threaded environments simultaneously, allowing it to be tailored to run on parallel systems of all configurations and sizes. By taking the advantage of high performance computing cluster, the use of pBWA can cut down the computing wall-time for the reference alignment step from weeks to hours for extremely large DNA sequence data. We expect that the availability of pBWA should facilitate the analysis of large-scale genome sequencing data generated by the new generations of sequencing technologies. The source code and detailed user manual of pBWA are freely available at http://sourceforge.net/projects/pbwa.

Information retrieval from historical newspaper collections in highly inflectional languages: A query expansion approach

The aim of the study was to test whether query expansion by approximate string matching methods is beneficial in retrieval from historical newspaper collections in a language rich with compounds and inflectional forms (Finnish). First, approximate string matching methods were used to generate lists of index words most similar to contemporary query terms in a digitized newspaper collection from the 1800s. Top index word variants were categorized to estimate the appropriate query expansion ranges in the retrieval test. Second, the effectiveness of approximate string matching methods, automatically generated inflectional forms, and their combinations were measured in a Cranfield‐style test. Finally, a detailed topic‐level analysis of test results was conducted. In the index of historical newspaper collection the occurrences of a word typically spread to many linguistic and historical variants along with optical character recognition (OCR) errors. All query expansion methods improved the baseline results. Extensive expansion of around 30 variants for each query word was required to achieve the highest performance improvement. Query expansion based on approximate string matching was superior to using the inflectional forms of the query words, showing that coverage of the different types of variation is more important than precision in handling one type of variation.

Mapping biological entities using the longest approximately common prefix method.

BackgroundThe significant growth in the volume of electronic biomedical data in recent decades has pointed to the need for approximate string matching algorithms that can expedite tasks such as named entity recognition, duplicate detection, terminology integration, and spelling correction. The task of source integration in the Unified Medical Language System (UMLS) requires considerable expert effort despite the presence of various computational tools. This problem warrants the search for a new method for approximate string matching and its UMLS-based evaluation.ResultsThis paper introduces the Longest Approximately Common Prefix (LACP) method as an algorithm for approximate string matching that runs in linear time. We compare the LACP method for performance, precision and speed to nine other well-known string matching algorithms. As test data, we use two multiple-source samples from the Unified Medical Language System (UMLS) and two SNOMED Clinical Terms-based samples. In addition, we present a spell checker based on the LACP method.ConclusionsThe Longest Approximately Common Prefix method completes its string similarity evaluations in less time than all nine string similarity methods used for comparison. The Longest Approximately Common Prefix outperforms these nine approximate string matching methods in its Maximum F1 measure when evaluated on three out of the four datasets, and in its average precision on two of the four datasets.

Improved candidate drug mining for Alzheimer's disease.

Alzheimer's disease (AD) is the main cause of dementia for older people. Although several antidementia drugs such as donepezil, rivastigmine, galantamine, and memantine have been developed, the effectiveness of AD drug therapy is still far from satisfactory. Recently, the single nucleotide polymorphisms (SNPs) have been chosen as one of the personalized medicine markers. Many pharmacogenomics databases have been developed to provide comprehensive information by associating SNPs with drug responses, disease incidence, and genes that are critical in choosing personalized therapy. However, we found that some information from different sets of pharmacogenomics databases is not sufficient and this may limit the potential functions for pharmacogenomics. To address this problem, we used approximate string matching method and data mining approach to improve the searching of pharmacogenomics database. After computation, we can successfully identify more genes linked to AD and AD-related drugs than previous online searching. These improvements may help to improve the pharmacogenomics of AD for personalized medicine.

Secure Hamming distance based record linkage with malicious adversaries

Record linkage aims at finding the matching records from two or multiple different databases. Many approximate string matching methods in privacy-preserving record linkage have been presented. In this paper, we study the problem of secure record linkage between two data files in the two-party computation setting. We note that two records are linked when all the Hamming distances between their attributes are smaller than some fixed thresholds. We firstly construct two efficient secure protocols for computing the powers of an encrypted value and implementing zero test on an encrypted value, then present an efficient protocol within constant rounds for computing the Hamming distance based record linkage in the presence of malicious adversaries by transferring these two protocols. We also discuss the extension of our protocol for settling the Levenshtein distance based record linkage problem.

A Consensus Algorithm for Approximate String Matching

Approximate string matching (ASM) is a well-known computational problem with important applications in database searching, plagiarism detection, spelling correction, and bioinformatics. The two main issues with most ASM algorithms are (1) computational complexity, and (2) low specificity due to a large amount of false positives being reported. In this paper, a very efficient ASM method is proposed, along with a post -processing stage designed to significantly reduce the amount of false positives. Results with random strings show that the proposed method is capable of performing a search within a large (1M b) string in about 100ms, with a sensitivity and specificity of nearly 100%.

Building Linked Open Data towards integration of biomedical scientific literature with DBpedia

BackgroundThere is a growing need for efficient and integrated access to databases provided by diverse institutions. Using a linked data design pattern allows the diverse data on the Internet to be linked effectively and accessed efficiently by computers. Previously, we developed the Allie database, which stores pairs of abbreviations and long forms (LFs, or expanded forms) used in the life sciences. LFs define the semantics of abbreviations, and Allie provides a Web-based search service for researchers to look up the LF of an unfamiliar abbreviation. This service encounters two problems. First, it does not display each LF’s definition, which could help the user to disambiguate and learn the abbreviations more easily. Furthermore, there are too many LFs for us to prepare a full dictionary from scratch. On the other hand, DBpedia has made the contents of Wikipedia available in the Resource Description Framework (RDF), which is expected to contain a significant number of entries corresponding to LFs. Therefore, linking the Allie LFs to DBpedia entries may present a solution to the Allie’s problems. This requires a method that is capable of matching large numbers of string pairs within a reasonable period of time because Allie and DBpedia are frequently updated.ResultsWe built a Linked Open Data set that links LFs to DBpedia titles by applying key collision methods (i.e., fingerprint and n-gram fingerprint) to their literals, which are simple approximate string-matching methods. In addition, we used UMLS resources to normalise the life science terms. As a result, combining the key collision methods with the domain-specific resources performed best, and 44,027 LFs have links to DBpedia titles. We manually evaluated the accuracy of the string matching by randomly sampling 1200 LFs, and our approach achieved an F-measure of 0.98. In addition, our experiments revealed the following. (1) Performances were similar independently from the frequency of the LFs in MEDLINE. (2) There is a relationship (r2 = 0.96, P < 0.01) between the occurrence frequencies of LFs in MEDLINE and their presence probabilities in DBpedia titles.ConclusionsThe obtained results help Allie users locate the correct LFs. Because the methods are computationally simple and yield a high performance and because the most frequently used LFs in MEDLINE appear more often in DBpedia titles, we can continually and reasonably update the linked dataset to reflect the latest publications and additions to DBpedia. Joining LFs between scientific literature and DBpedia enables cross-resource exploration for mutual benefits.

Performance evaluation of load balancing strategies for approximate string matching application on an MPI cluster of heterogeneous workstations

In this paper, we present three parallel approximate string matching methods on a parallel architecture with heterogeneous workstations to gain supercomputer power at low cost. The first method is the static master–worker with uniform distribution strategy, the second one is the dynamic master–worker with allocation of subtexts and the third one is the dynamic master–worker with allocation of text pointers. Further, we propose a hybrid parallel method that combines the advantages of static and dynamic parallel methods in order to reduce the load imbalance and communication overhead. This hybrid method is based on the following optimal distribution strategy: the text collection is distributed proportional to workstation’s speed. We evaluated and compared the performance of the four methods with clusters one, two, four, six and eight heterogeneous workstations. The experimental results demonstrate that dynamic allocation of text pointers and hybrid methods achieve better performance than the two original ones. We also present an analytical performance model for the four methods that confirms the actual behaviour of the experimental results.

Dictionary organizations for efficient similarity retrieval

Searching dictionaries for an exact match to a given input string can be done very efficiently. Performance degrades, however, when inexact matches have to be located. Approximate string matching methods enable inexact matches to be located by comparing the search string with every dictionary entry. To avoid such exhaustive searches, dictionaries are generally partitioned according to some criteria. This paper examines some methods of dictionary organization that could lead to improved performances. The use of a variant of the n-gram method as a preselection technique is also discussed, along with a method using two levels of indexing. Experimental results show that of all the methods examined, indexing the string on each half provides the best result. A two-level indexing method combining two partitioning criteria also performs reasonably well. The n-gram technique appears to be close to these two in performance.