Corasick Algorithm Research Articles

Application of the Aho-Korasik Algorithm for the Selection of Primers for Loop Isothermal Amplification

This paper presents a program which allows user to do primer design for identifying DNA target site or a whole genome with a goal of performing loopmediated isothermal amplification. The review of the most popular existing primer design programs for LAMP is carried out. Recommended conditions are presented in the paper. They are required to be taken in consideration during the process of primer design for loop-mediated isothermal amplification. These are the conditions: primer’s length, GC-content, amplicon average size, annealing temperature and distance between primers. A search for primer positions in genome is needed since loop-mediated isothermal amplification requires primer kits that consist of 6 primers in order for primer design to be done. The Aho–Corasick algorithm was proposed for a search implementation. This algorithm is capable of simultaneous search for a number of sample (primer) entries in a longer sequence (a fragment or a whole genome). This software allows the search for primers in genomes of various length and it groups primers by kits, which in turn could be applied in laboratory experiments. These kits are formed according both to the recommended conditions of primer selection for performing loop-mediated isothermal amplification and to the initial conditions, which are determined by the user before the process. After that, the user may choose the best option for their case from a list of primer kits that are being created as a result of performed computer analysis. The test run of the program was done during the search for a specific primer kit that is meant to be used for performing loop-mediated isothermal amplification of genome with a goal of detection of novel coronavirus infection SARS-CoV-2, a virus that triggers a dangerous disease, COVID-19. The software was developed using Python with BioPython and Pyahocorasick libraries and available at the link: https://cloud.mail.ru/public/C7av/QCkSiUomz.

The Diversification and Enhancement of an IDS Scheme for the Cybersecurity Needs of Modern Supply Chains

Despite the tremendous socioeconomic importance of supply chains (SCs), security officers and operators are faced with no easy and integrated way for protecting their critical, and interconnected, infrastructures from cyber-attacks. As a result, solutions and methodologies that support the detection of malicious activity on SCs are constantly researched into and proposed. Hence, this work presents the implementation of a low-cost reconfigurable intrusion detection system (IDS), on the edge, that can be easily integrated into SC networks, thereby elevating the featured levels of security. Specifically, the proposed system offers real-time cybersecurity intrusion detection over high-speed networks and services by offloading elements of the security check workloads on dedicated reconfigurable hardware. Our solution uses a novel framework that implements the Aho–Corasick algorithm on the reconfigurable fabric of a multi-processor system-on-chip (MPSoC), which supports parallel matching for multiple network packet patterns. The initial performance evaluation of this proof-of-concept shows that it holds the potential to outperform existing software-based solutions while unburdening SC nodes from demanding cybersecurity check workloads. The proposed system performance and its efficiency were evaluated using a real-life environment in the context of European Union’s Horizon 2020 research and innovation program, i.e., CYRENE.

TahcoRoll: fast genomic signature profiling via thinned automaton and rolling hash.

ObjectivesGenomic signatures like k-mers have become one of the most prominent approaches to describe genomic data. As a result, myriad real-world applications, such as the construction of de Bruijn graphs in genome assembly, have been benefited by recognizing genomic signatures. In other words, an efficient approach of genomic signature profiling is an essential need for tackling high-throughput sequencing reads. However, most of the existing approaches only recognize fixed-size k-mers while many research studies have shown the importance of considering variable-length k-mers.MethodsIn this paper, we present a novel genomic signature profiling approach, TahcoRoll, by extending the Aho–Corasick algorithm (AC) for the task of profiling variable-length k-mers. We first group nucleotides into two clusters and represent each cluster with a bit. The rolling hash technique is further utilized to encode signatures and read patterns for efficient matching.ResultsIn extensive experiments, TahcoRoll significantly outperforms the most state-of-the-art k-mer counters and has the capability of processing reads across different sequencing platforms on a budget desktop computer.ConclusionsThe single-thread version of TahcoRoll is as efficient as the eight-thread version of the state-of-the-art, JellyFish, while the eight-thread TahcoRoll outperforms the eight-thread JellyFish by at least four times.

Efficient pattern matching algorithm for security and Binary Search Tree (BST) based memory system in Wireless Intrusion Detection System (WIDS)

Wireless Intrusion Detection System (WIDS) has been introduced for providing enhanced security level in WLANs, owing to the numerous and potentially devastating threats against it. WIDS are software- or hardware-based. The hardware-based approaches focus on memory efficiency in pattern matching. This paper proposes a pattern matching algorithm called All-Ready State Traversal pattern matching algorithm. This algorithm constructs the state traversal machine with 1280 bytes size, and enables users to store large sized string patterns in the pattern database. The state traversal machine facilitates the easy retrieval of these patterns through the path vector. Further, the proposed work also follows a number of basic ASCII characters with 128 bytes size; and designs the memory architecture using Binary Search Tree (BST) structure. The hardware generates the addresses of input strings. State traversal machine and bits split algorithms are used to merge together the common addresses of input strings. The merged addresses are encrypted and decrypted by Blowfish algorithm to allow the valid packets and to discard the invalid packets using WIDS. Thus, the proposed algorithm provides a significant reduction in memory usage than that of Aho–Corasick algorithm.

Accelerated preprocessing in task of searching substrings in a string

Introduction.A rapid development of the systems such as Yandex, Google, etc., has predetermined the relevance of the task of searching substrings in a string, and approaches to its solution are actively investigated. This task is used to create database management systems that support associative search. Besides, it is applicable in solving information security issues and creating antivirus programs. Algorithms of searching substring in a string are used in signature-based discovery tasks.Materials and Methods. The solution to the problem is based on the Aho-Corasick algorithm which is a typical technique of searching substrings in a string. At the same time, a new approach regarding preprocessing is employed.Research Results. The possibility of constructing the transition function and suffix references through suffix arrays and special mappings, is shown. The relationship between the prefix tree and suffix arrays was investigated, which provided the development of a fundamentally new method of constructing the transition and error functions. The results obtained enable to substantially shorten the time intervals spent on the preelection processing of a set of pattern strings when using an integer alphabet. The paper lists eight algorithms. The developed algorithms are evaluated. The results obtained are compared to the formerly known. Two theorems and eight lemmas are proved. Two examples illustrating features of the practical application of the developed preprocessing procedure are given.Discussion and Conclusions. The preprocessing procedure proposed in this paper is based on the communication between the suffix array built on the ground of a set of pattern strings and the construction of transition and error functions at the initial stages of the Aho-Corasick algorithm. This approach differs from the traditional one and requires the use of algorithms providing a suffix array in linear time. Thus, the algorithms that enable to significantly reduce the time for preprocessing of a set of pattern strings under the condition of using a certain type of alphabet in comparison to the known approach proposed in the Aho- Corasick algorithm are described. The research results presented in the paper can be used in antivirus programs that apply searching for signatures of malicious data objects in the memory of a computer system. In addition, this approach to solving the problem on searching substrings in a string will significantly speed up the operation of database management systems using associative search.

MultiPLZW: A novel multiple pattern matching search in LZW-compressed data

Searching encrypted or compressed data provides security and privacy without sacrificing efficiency. It has many applications in cloud storage, bioinformatics, IoT, unmanned aerial vehicles and drones. This paper introduces a novel, simple, and efficient algorithm to locate all occurrences of a set of patterns in LZW-compressed data, in a single pass. The algorithm comprises a preprocessing phase and a subsequent search phase. It uses a modified version of the generalized suffix tree, a lookup table, a mapping table, and a history tree. The proposed algorithm is superior in terms of the time complexity, while maintaining a space complexity of the same order as the best of existing algorithms. The time complexity is O(n+m+r), which is proportional to the length of the LZW-compressed data, where n is the length of the compressed data, m is the total size of the patterns, and r is the number of pattern occurrences in the compressed data. The space complexity is O(m2+t+r), where t is the size of the dictionary table that is used during compression. Experimental results show a significant improvement in search time, approximately twice as fast, compared to decompressing and then searching using Aho–Corasick algorithm. Also, results on various dataset sizes, demonstrate the algorithm’s superior scalability, which improves as the size of the dataset increases.

OrfM: a fast open reading frame predictor for metagenomic data.

Summary: Finding and translating stretches of DNA lacking stop codons is a task common in the analysis of sequence data. However, the computational tools for finding open reading frames are sufficiently slow that they are becoming a bottleneck as the volume of sequence data grows. This computational bottleneck is especially problematic in metagenomics when searching unassembled reads, or screening assembled contigs for genes of interest. Here, we present OrfM, a tool to rapidly identify open reading frames (ORFs) in sequence data by applying the Aho–Corasick algorithm to find regions uninterrupted by stop codons. Benchmarking revealed that OrfM finds identical ORFs to similar tools (‘GetOrf’ and ‘Translate’) but is four-five times faster. While OrfM is sequencing platform-agnostic, it is best suited to large, high quality datasets such as those produced by Illumina sequencers.Availability and Implementation: Source code and binaries are freely available for download at http://github.com/wwood/OrfM or through GNU Guix under the LGPL 3+ license. OrfM is implemented in C and supported on GNU/Linux and OSX.Contacts: b.woodcroft@uq.edu.auSupplementary information: Supplementary data are available at Bioinformatics online.

Spam Filtering through Multiple Pattern Bit Parallel String Matching Combining Shift AND & OR

Spam refers to unsolicited, unwanted and inappropriate bulk email. Spam filtering has become conspicuous as they consume a lot of network bandwidth, overloads the email server and drops the productivity of global economy. Content based spam filtering is accomplished with the help of multiple pattern string matching algorithm. Traditionally Aho Corasick algorithm was used to filter spam which constructs a trie of the spam keywords. The performance degrades in the context of time as well as space as the size of trie increases with the growing spam keywords count. To counterbalance time and space loss, bit parallel multiple pattern string matching algorithm using Shift OR method is used. The method acts as filter performing approximate string matching . This implies that there are some false matches detected by the filter which requires verification. The proposed method for filtering spams has been developed using a combination of Shift AND and OR operation. The method directly works on spam keywords of equal size whereas for unequal size keywords, a new proposed equal size grouping method is developed. Both method shows improvement over the Aho Corasick algorithm in context of space complexity and also behaves as an efficient filter and reducing the number of false matches as present in Shift OR method.

GPU-Friendly Parallel Genome Matching with Tiled Access and Reduced State Transition Table

In this paper, we propose a new parallel genome matching algorithm using graphics processing units (GPUs). Our proposed approach is based on the Aho–Corasick algorithm and it was developed based on a consideration of the architectural features of existing GPUs with a hundred or more cores. Thus, we provide an appropriate task partitioning method that runs on multiple threads and we fully utilize the cache memory and the shared memory structures available in GPUs. Especially, we propose a tiled access method for rapid data transfer from the global memory to the shared memory. We also provide new models for cache-friendly state transition table to improve performance of pattern matching operations on GPUs. The maximum throughput we achieved in various experiments was 15.3 Gbps. Moreover, we showed that our proposed design outperformed an earlier approach with a 15.4 % performance improvement.

The Study On The Applicability Of AHO-CORASICK Algorithm In Identifying Tests’ Validity

Aho-Corasick Algorithm (ACA) is a kind of dictionary-matching algorithm that locates elements of finite set of strings within an input text. It matches all patterns “at once”, so the complexity of the algorithm is linear in the length of the patterns plus the length of the searched text plus the number of output matches. This paper discusses the applicability of Aho-Corasick algorithm in identifying test validity using the standard Guidelines in Evaluating Tests. A proposed Quiz-Zone system was developed in order to evaluate and test the applicability of the algorithm used. Quiz-Zone allows the user to create exam that will check the test’s validity. It also allows the user to choose five types of exam namely: Matching Type, Multiple Choice, Essay, True or False and Short-Answer. The researchers revealed that there are some rules in identifying test validity that ACA can’t be applied. Keywords : Aho Corasick Algorithm, string-matching algorithm, test validity

Multipattern string matching with q -grams

We present three algorithms for exact string matching of multiple patterns. Our algorithms are filtering methods, which apply q -grams and bit parallelism. We ran extensive experiments with them and compared them with various versions of earlier algorithms, e.g., different trie implementations of the Aho--Corasick algorithm. All of our algorithms appeared to be substantially faster than earlier solutions for sets of 1,000--10,000 patterns and the good performance of two of them continues to 100,000 patterns. The gain is because of the improved filtering efficiency caused by q -grams.

The shortest path problem with forbidden paths

We consider a variant of the constrained shortest path problem, where the constraints come from a set of forbidden paths (arc sequences) that cannot be part of any feasible solution. Two solution approaches are proposed for this variant. The first uses Aho and Corasick's keyword matching algorithm to filter paths produced by a k-shortest paths algorithm. The second generalizes Martins' deviation path approach for the k-shortest paths problem by merging the original graph with a state graph derived from Aho and Corasick's algorithm. Like Martins' approach, the second method amounts to a polynomial reduction of the shortest path problem with forbidden paths to a classic shortest path problem. Its significant advantage over the first approach is that it allows considering forbidden paths in more general shortest path problems such as the shortest path problem with resource constraints.

A Boyer–Moore-style algorithm for regular expression pattern matching

This paper presents a Boyer–Moore-type algorithm for regular expression pattern matching, answering an open problem posed by Aho in 1980 (Pattern Matching in Strings, Academic Press, New York, 1980, p. 342). The new algorithm handles patterns specified by regular expressions—a generalization of the Boyer–Moore and Commentz-Walter algorithms. Like the Boyer–Moore and Commentz-Walter algorithms, the new algorithm makes use of shift functions which can be precomputed and tabulated. The precomputation algorithms are derived, and it is shown that the required shift functions can be precomputed from Commentz-Walter's d 1 and d 2 shift functions. In certain cases, the Boyer–Moore (respectively Commentz-Walter) algorithm has greatly outperformed the Knuth–Morris–Pratt (respectively Aho–Corasick) algorithm (as discussed by Watson in his Ph.D. Thesis, Eindhoven University of Technology, September 1995, and in: N. Ziviani, R. Baeza-Yates, K. Guimaraes (Eds.), Proc. Third South American Workshop on String Processing, International Informatics Series, vol. 4, Carleton University Press, Recife, Brazil, 1996, pp. 280–294). In testing, the algorithm presented in this paper also frequently outperforms the regular expression generalization of the Aho–Corasick algorithm.

A new regular grammar pattern matching algorithm

This paper presents a Boyer–Moore type algorithm for regular grammar pattern matching, answering a variant of an open problem posed by Aho (Pattern Matching in Strings, Academic Press, New York, 1980, p. 342). The new algorithm handles patterns specified by regular (left linear) grammars—a generalization of the Boyer–Moore (single keyword) and Commentz-Walter (multiple keyword) algorithms.Like the Boyer–Moore and Commentz-Walter algorithms, the new algorithm makes use of shift functions which can be precomputed and tabulated. The precomputation functions are derived, and it is shown that they can be obtained from Commentz-Walter's d1 and d2 shift functions.In most cases, the Boyer–Moore (respectively, Commentz-Walter) algorithm has greatly outperformed the Knuth–Morris–Pratt (respectively, Aho–Corasick) algorithm. In practice, an earlier version of the algorithm presented in this paper also frequently outperforms the regular grammar generalization of the Aho–Corasick algorithm.

TROLL—Tandem Repeat Occurrence Locator

Tandem Repeat Occurrence Locator (TROLL), is a light-weight Simple Sequence Repeat (SSR) finder based on a slight modification of the Aho-Corasick algorithm. It is fast and only requires a standard Personal Computer (PC) to operate. We report running times of 127 s to find all SSRs of length 20 bp or more on the complete Arabdopsis genome--approx. 130 Mbases divided in five chromosomes--using a PC Athlon 650 MHz with 256 MB of RAM. TROLL is an open source project and is available at http://finder.sourceforge.net.

Fast practical multi-pattern matching

The multi-pattern matching problem consists in finding all occurrences of the patterns from a finite set X in a given text T of length n. We present a new and simple algorithm combining the ideas of the Aho–Corasick algorithm and the directed acyclic word graphs. The algorithm has time complexity which is linear in the worst case (it makes at most 2n symbol comparisons) and has good average-case time complexity assuming the shortest pattern is sufficiently long. Denote the length of the shortest pattern by m, and the total length of all patterns by M. Assume that M is polynomial with respect to m, the alphabet contains at least 2 symbols and the text (in which the pattern is to be found) is random, for each position each letter occurs independently with the same probability. Then the average number of comparisons is O((n/m)· logm) , which matches the lower bound of the problem. For sufficiently large values of m the algorithm has a good behavior in practice.