Knuth-Morris-Pratt Research Articles

A Fast Exact String Matching Algorithm Based on Nested Classification

This paper makes a research on the exact string matching, proposed a nested classification string matching algorithm. This paper based on the KMP algorithm, constructs the multi-level string matching and use the KR algorithm calculates the next function quickly, in order to replace the traditional KMP algorithm which is using the recursive thinking to calculate the next[j] function value in the large number of pattern string under the premise. Through the use of this method we can reduce the number of matches thereby raising the efficiency.

Functional verification of signature detection architectures for high speed network applications

To meet the future internet traffic challenges, enhancement of hardware architectures related to network security has vital role where software security algorithms are incompatible with high speed in terms of Giga bits per second (Gbps). In this paper, we discuss signature detection technique (SDT) used in network intrusion detection system (NIDS). Design of most commonly used hardware based techniques for signature detection such as finite automata, discrete comparators, Knuth-Morris-Pratt (KMP) algorithm, content addressable memory (CAM) and Bloom filter are discussed. Two novel architectures, XOR based pre computation CAM (XPCAM) and multi stage look up technique (MSLT) Bloom filter architectures are proposed and implemented in third party field programmable gate array (FPGA), and area and power consumptions are compared. 10Gbps network traffic generator (TNTG) is used to test the functionality and ensure the reliability of the proposed architectures. Our approach involves a unique combination of algorithmic and architectural techniques that outperform some of the current techniques in terms of performance, speed and powerefficiency.

Parallelization of KMP String Matching Algorithm on Different SIMD Architectures: Multi-Core and GPGPU's

matching is a classical problem in computer science. After the study of the Naive string search, Brute Force and the KMP algorithm, several advantages and disadvantages of the algorithms have been analyzed. Considering KMP in particular concept of parallelization has been introduced to improve the performance of the KMP algorithm. The algorithm is designed to work on SIMD parallel architecture where text is divided for parallel processing and special searching at division point is required for consistent and complete searching. This algorithm reduces the number of comparisons and parallelization improves the time efficiency. This algorithm achieves a better result as compared to the multithreaded version of the algorithm where again by text dividing, the parallelization is achieved.

Fast searching in packed strings

Given strings P and Q the (exact) string matching problem is to find all positions of substrings in Q matching P. The classical Knuth–Morris–Pratt algorithm [SIAM J. Comput. 6 (2) (1977) 323–350] solves the string matching problem in linear time which is optimal if we can only read one character at the time. However, most strings are stored in a computer in a packed representation with several characters in a single word, giving us the opportunity to read multiple characters simultaneously. In this paper we study the worst-case complexity of string matching on strings given in packed representation. Let m ⩽ n be the lengths P and Q, respectively, and let σ denote the size of the alphabet. On a standard unit-cost word-RAM with logarithmic word size we present an algorithm using time O ( n log σ n + m + occ ) . Here occ is the number of occurrences of P in Q. For m = o ( n ) this improves the O ( n ) bound of the Knuth–Morris–Pratt algorithm. Furthermore, if m = O ( n / log σ n ) our algorithm is optimal since any algorithm must spend at least Ω ( ( n + m ) log σ log n + occ ) = Ω ( n log σ n + occ ) time to read the input and report all occurrences. The result is obtained by a novel automaton construction based on the Knuth–Morris–Pratt algorithm combined with a new compact representation of subautomata allowing an optimal tabulation-based simulation.

모바일 환경에서 파일 검색 엔진을 위한 효과적인 방식

This study focuses on the modeling file search engine and suggesting modified file search schema based on weight value using file contents in order to improve the performance in terms of search accuracy and matching time. Most of the file search engines have used string matching algorithms like KMP(Knuth.Morris.Pratt), which may limit portability and fast searching time. However, this kind of algorithms don`t find exactly the files what you want. Hence, the file search engine based on weight value using file contents is proposed here in order to optimize the performance for mobile environments. The Comparison with previous research shows that the proposed schema provides better.

A simple fast hybrid pattern-matching algorithm

The Knuth–Morris–Pratt (KMP) pattern-matching algorithm guarantees both independence from alphabet size and worst-case execution time linear in the pattern length; on the other hand, the Boyer–Moore (BM) algorithm provides near-optimal average-case and best-case behaviour, as well as executing very fast in practice. We describe a simple algorithm that employs the main ideas of KMP and BM (with a little help from Sunday) in an effort to combine these desirable features. Experiments indicate that in practice the new algorithm is among the fastest exact pattern-matching algorithms discovered to date, apparently dominant for alphabet size above 15–20.

A computationally efficient engine for flexible intrusion detection

Pattern matching for network security and intrusion detection demands exceptionally high performance. This paper describes a novel systolic array-based string matching architecture using a buffered, two-comparator variation of the Knuth-Morris-Pratt (KMP) algorithm. The architecture compares favorably with the state-of-the-art hardwired designs while providing on-the-fly reconfiguration, efficient hardware utilization, and high clock rates. KMP is a well-known computationally efficient string-matching technique that uses a single comparator and a precomputed transition table. Through the use of the transition table, the number of redundant comparisons performed is reduced. Through various algorithmic changes, we enable KMP to be used in hardware, providing the computational efficiency of the serial algorithm and the high throughput of a parallel hardware architecture. The efficiency of the system allows for a faster and denser implementation than any other RAM-based exact match system. We add a second comparator and an input buffer and then prove that the modified algorithm can function efficiently implemented as an element of a systolic array. The system can accept at least one character in each cycle while guaranteeing that the stream will never stall. In this paper, we prove the bound on the buffer size and running time of the systolic array, discuss the architectural considerations involved in the FPGA implementation, and provide performance comparisons against other approaches.

Adapting the Knuth–Morris–Pratt algorithm for pattern matching in Huffman encoded texts

In the present work we perform compressed pattern matching in binary Huffman encoded texts [Huffman, D. (1952). A method for the construction of minimum redundancy codes, Proc. of the IRE, 40, 1098-1101]. A modified Knuth-Morris-Pratt algorithm is used in order to overcome the problem of false matches, i.e., an occurrence of the encoded pattern in the encoded text that does not correspond to an occurrence of the pattern itself in the original text. We propose a bitwise KMP algorithm that can move one extra bit in the case of a mismatch since the alphabet is binary. To avoid processing any bit of the encoded text more than once, a preprocessed table is used to determine how far to back up when a mismatch is detected, and is defined so that we are always able to align the start of the encoded pattern with the start of a codeword in the encoded text. We combine our KMP algorithm with two practical Huffman decoding schemes which handle more than a single bit per machine operation; skeleton trees defined by Klein [Klein, S. T. (2000). Skeleton trees for efficient decoding of huffman encoded texts. Information Retrieval, 3, 7-23], and numerical comparisons between special canonical values and portions of a sliding window presented in Moffat and Turpin [Moffat, A., & Turpin, A. (1997). On the implementation of minimum redundancy prefix codes. IEEE Transactions on Communications, 45, 1200-1207]. Experiments show rapid times of our algorithms compared to the decompress then search method, therefore, files can be kept in their compressed form, saving memory space. When compression gain is important, these algorithms are better than cgrep [Ferragina, P., Tommasi, A., & Manzini, G. (2004). C Library to over compressed texts, http://roquefort.di.unipi.it/~ferrax/CompressedSearch], which is only slightly faster than ours.

On maximal suffixes and constant-space linear-time versions of KMP algorithm

Constant-space linear-time string-matching algorithms are usually very sophisticated. Most of them consist of two phases: (very technical) preprocessing phase and searching phase. An exception is one-phase Crochemore's algorithm (Theoret. Comput. Sci. 92 (1992) 33). It is an on-line version of Knuth–Morris–Pratt algorithm (KMP) with “on-the-fly” computation of pattern shifts (as approximate periods). In this paper we explore further Crochemore's approach, and construct alternative algorithms which are differently structured. In Crochemore's algorithm the approximate-period function is restarted from inside, which means that several internal variables of this function are changing globally, also Crochemore's algorithm strongly depends on the concrete implementation of approximate-periods computation. We present a simple modification of KMP algorithm which works in O(1) space, O(n) time for any function which computes periods or approximate periods in O(1)-space and linear time. The approximate-period function can be treated as a black box. We identify class of patterns, self-maximal words, which are especially well suited for Crochemore-style string matching. A new O(1)-space string-matching algorithm, MaxSuffix-Matching, is proposed in the paper, which gives yet another example of applicability of maximal suffixes.

On Obtaining Knuth, Morris, and Pratt's String Matcher by Partial Evaluation

We present the first formal proof that partial evaluation of a quadratic string matcher can yield the precise behaviour of Knuth, Morris, and Pratt's linear string matcher.<br /> <br />Obtaining a KMP-like string matcher is a canonical example of partial evaluation: starting from the naive, quadratic program checking whether a pattern occurs in a text, one ensures that backtracking can be performed at partial-evaluation time (a binding-time shift that yields a staged string matcher); specializing the resulting staged program yields residual programs that do not back up on the text, a la KMP. We are not aware, however, of any formal proof that partial evaluation of a staged string matcher precisely yields the KMP string matcher, or in fact any other specific string matcher.<br /> <br />In this article, we present a staged string matcher and we formally prove that it performs the same sequence of comparisons between pattern and text as the KMP string matcher. To this end, we operationally specify each of the programming languages in which the matchers are written, and we formalize each sequence of comparisons with a trace semantics. We also state the (mild) conditions under which specializing the staged string matcher with respect to a pattern string provably yields a specialized string matcher whose size is proportional to the length of this pattern string and whose time complexity is proportional to the length of the text string. Finally, we show how tabulating one of the functions in this staged string matcher gives rise to the `next' table of the original KMP algorithm.<br /> <br />The method scales for obtaining other linear string matchers, be they known or new.

Window-accumulated subsequence matching problem is linear

Given two strings, text t of length n, and pattern p = p 1…p k of length k, and given a natural number w, the subsequence matching problem consists in finding the number of size w windows of text t which contain pattern p as a subsequence, i.e. the letters p 1,…,p k occur in the window, in the same order as in p, but not necessarily consecutively (they may be interleaved with other letters). Subsequence matching is used for finding frequent patterns and association rules in databases. We generalize the Knuth–Morris–Pratt (KMP) pattern matching algorithm; we define a non-conventional kind of RAM, the MP-RAMs which model more closely the microprocessor operations; we design an O(n) on-line algorithm for solving the subsequence matching problem on MP-RAMs.

Fast and flexible string matching by combining bit-parallelism and suffix automata

The most important features of a string matching algorithm are its efficiency and its flexibility. Efficiency has traditionally received more attention, while flexibility in the search pattern is becoming a more and more important issue. Most classical string matching algorithms are aimed at quickly finding an exact pattern in a text, being Knuth-Morris-Pratt (KMP) and the Boyer-Moore (BM) family the most famous ones. A recent development uses deterministic "suffix automata" to design new optimal string matching algorithms, e.g. BDM and TurboBDM. Flexibility has been addressed quite separately by the use of "bit-parallelism", which simulates automata in their nondeterministic form by using bits and exploiting the intrinsic parallelism inside the computer word, e.g. the Shift-Or algorithm. Those algorithms are extended to handle classes of characters and errors in the pattern and/or in the text, their drawback being their inability to skip text characters. In this paper we merge bit-parallelism and suffix automata, so that a nondeterministic suffix automaton is simulated using bit-parallelism. The resulting algorithm, called BNDM, obtains the best from both worlds. It is much simpler to implement than BDM and nearly as simple as Shift-Or. It inherits from Shift-Or the ability to handle flexible patterns and from BDM the ability to skip characters. BNDM is 30%-40% faster than BDM and up to 7 times faster than Shift-Or. When compared to the fastest existing algorithms on exact patterns (which belong to the BM family), BNDM is from 20% slower to 3 times faster, depending on the alphabet size. With respect to flexible pattern searching, BNDM is by far the fastest technique to deal with classes of characters and is competitive to search allowing errors. In particular, BNDM seems very adequate for computational biology applications, since it is the fastest algorithm to search on DNA sequences and flexible searching is an important problem in that area. As a theoretical development related to flexible pattern matching, we introduce a new automaton to recognize suffixes of patterns with classes of characters. To the best of our knowledge, this automaton has not been studied before.

Simple Optimal String Matching Algorithm

We present a new string matching algorithm optimal on average (with equiprobability and independence of letters, in O(m+nlog|Σ|m/m), where n is the size of the text and m the size of the searched word, both taken on an alphabet Σ) and linear in the worst case (in O(m+n)). Of all the algorithms that verify these two complexities, ours is the simplest since it uses only a single structure, a suffix automaton. Moreover, its preprocessing phase is linearly dynamical; i.e., it is possible to search the words p1, then p1p2, p1p2p3,…,p−1p−2p−3···pi with O(∑|pi|) total preprocessing time. Among the algorithms that verify this property (for instance, the Knuth–Morris–Pratt algorithm) our algorithm is the only one to be optimal on average.

A positive supercompiler

AbstractWe introduce apositive supercompiler, a version of Turchin's supercompiler maintaining only positive information during transformation, and using folding without generalization. The positive supercompiler can also be regarded as a variant of Wadler's deforestation maintaining an increased amount of information. We compare our algorithm to deforestation and, in less detail, to partial evaluation, Turchin's supercompiler, Generalized Partial Computation (GPC), and partial deduction by classifying these transformers by the amount of information they maintain during transformation. This factor is significant, as a differentiating example reveals: positive supercompilation, Turchin's supercompiler, GPC and partial deduction can specialize a general pattern matcher with respect to a fixed pattern to obtain an efficient matcher very similar to the Knuth–Morris–Pratt algorithm. Deforestation and traditional partial evaluation achieve this effect only after a non-trivial hand rewriting of the general matcher.

A Left-to-Right Preprocessing Computation for the Boyer-Moore String Matching Algorithm

The string matching algorithm is generally divided into two phases : the preprocessing phase to obtain the shift tables and the main matching process with the tables. In the off-line version, we computed the tables after the whole pattern has been input, while in the on-line version, we computed the tables as the new character of the pattern is input, that is, on the fly. The shift table for the Knuth-Morris-Pratt (KMP) is computed from left to right, and hence suitable for the on-line version. The d-table and Horspool table in the Boyer-Moore (BM) algorithm are also computed from left to right in linear time. We show in this paper that the dd-table for the BM algorithm can be computed in O(m log m) expected time from left to right for a random pattern, and hence can be incorporated into an on-line version.

The Möbius function of factor order

Intervals in the factor ordering of a free monoid are investigated. It was shown by Farmer (1982) that such intervals (β, α) are contractible or homotopy spheres in case β is the empty word. We observe here that the same is true in general. This implies that the Möbius function of factor order takes values in {0, + 1, −1}. A recursive rule for this Möbius function is given, which allows efficient computation via the Knuth—Morris—Pratt algorithm. The Möbius function of subword order was studied in Björner (1990). We give here a simpler proof (a parity-changing involution) for its combinatorial interpretation.

Formal derivation of a pattern matching algorithm

This paper is devoted to the synthesis of a functional version of the Knuth-MorrisPratt algorithm for pattern matching. This algorithm was first discussed by Knuth [6]; since then formal developments have been given by Dijkstra [4] and Dromey [5], among many others. The novel aspects of the present treatment are: (i) the result is expressed as a (very short) functional program; and (ii) the derivation makes use of the calculus of lists described by Bird [ 11. In order to make this paper as self-contained as possible, we shall indicate the concepts and notations of [l] used in our derivation as we go along. However, we do assume some familiarity with the basic ideas of functional programming. (For an introduction to functional programming, see Bird and Wadler [3].) The calculus of lists is based on the definition of lists as an algebraic structure (actually, a monoid) under the primitive operation of concatenation, denoted by the sign *. Thus

An analytical comparison of two string searching algorithms

Average case analyses of two algorithms to locate the leftmost occurrence of a string P attern in a string T ext are conducted in this paper. One algorithm is based on a straightforward trial-and-error approach, the other one uses a sophisticated stragegy discovered by Knuth, Morris and Pratt (1977). Costs measured are the expected number of comparisons between individual characters. Let N aive and kmp denote the average case complexities of the two algorithms, respectively. We show that 1−(1/c)+(1/c 2) is an accurate approximation for the ratio kmp/N aive, provided both P attern and T ext are random strings over an alphabet of size c. In both cases, the application of Markov chain theory is expedient for performing the analysis. However, in order to get rid of complex conditioning, the Markov chain model for the kmp algorithm is based on some heuristics. This approach is believed to be practically sound. Some indication on the complexity that might be involved in an exact average case analysis of the kmp algorithm can be found in the work by Guibas and Odlyzko (1981).

An alternative for the implementation of the Knuth-Morris-Pratt algorithm

An alternative for the implementation of the Knuth-Morris-Pratt algorithm

Saving Space in Fast String-Matching

The string-matching problem is to find all instances (as contiguous substrings) of a “pattern” character string x in a longer “text" string ”. The naive algorithm, trying the pattern from scratch starting at each successive text position, requires only a fixed number of auxiliary storage locations but time proportional to $|x|\cdot|Y|$ (worst case). On the other hand, the fast algorithm of Knuth, Morris, and Pratt requires only time proportional to $|Y|$ but extra space proportional to $|x|$ (every case). Algorithms described in this paper reduce the extra space used by the Knuth–Morris–Pratt algorithm down to $O(\log |x|)$, and the time for the naive algorithm down to $O(|x|^\varepsilon |y|)$ for any fixed $\varepsilon > 0$. Also described are implementations on two-way multihead finite automata and multitape Turing machines.