Wu-Manber Algorithm Research Articles

Accelerating Pattern Matching Using a Novel Multi-Pattern-Matching Algorithm on GPU

Nowadays, almost all network traffic is encrypted. Attackers hide themselves using this traffic and attack over encrypted channels. Inspections performed only on packet headers and metadata are insufficient for detecting cyberattacks over encrypted channels. Therefore, it is important to analyze packet contents in applications that require control over payloads, such as content filtering, intrusion detection systems (IDSs), data loss prevention systems (DLPs), and fraud detection. This technology, known as deep packet inspection (DPI), provides full control over the communication between two end stations by keenly analyzing the network traffic. This study proposes a multi-pattern-matching algorithm that reduces the memory space and time required in the DPI pattern matching compared to traditional automaton-based algorithms with its ability to process more than one packet payload character at once. The pattern-matching process in the DPI system created to evaluate the performance of the proposed algorithm (PA) is conducted on the graphics processing unit (GPU), which accelerates the processing of network packets with its parallel computing capability. This study compares the PA with the Aho-Corasick (AC) and Wu–Manber (WM) algorithms, which are widely used in the pattern-matching process, considering the memory space required and throughput obtained. Algorithm tables created with a dataset containing 500 patterns use 425 and 688 times less memory space than those of the AC and WM algorithms, respectively. In the pattern-matching process using these tables, the PA is 3.5 and 1.5 times more efficient than the AC and WM algorithms, respectively.

The WM-q multiple exact string matching algorithm for DNA sequences

The string matching algorithms are among the essential fields in computer science, such as text search, intrusion detection systems, fraud detection, sequence search in bioinformatics. The exact string matching algorithms are divided into two parts: single and multiple. Multiple string matching algorithms involve finding elements of the pattern set P in a given input text T. String matching processes should be done in a time-efficient manner for DNA sequences. As the volume of the text T increases and the number of search patterns increases, the total runtime increases. Efficient algorithms should be selected to perform these search operations as soon as possible. In this study, the Wu-Manber algorithm, one of the multiple exact string matching algorithms, is improved. Although the Wu-Manber algorithm is effective, it has some limitations, such as hash collisions. In this study, the WM-q algorithm, a version of the Wu-Manber algorithm based on the perfect hash function for DNA sequences, is proposed. String matching is performed using different block lengths provided by the perfect hash function instead of using the fixed block length as in the traditional Wu-Manber algorithm. The proposed approach has been compared with E. Coli and Human Chromosome1 datasets, frequently used in the literature, using multiple exact string matching algorithms. The proposed algorithm gives better results for performance metrics such as the average runtime, the average number of characters and hash comparisons.

KPN-based parallelization of Wu–Manber algorithm on multi-core machines

Pattern matching is the most time consuming task in many cybersecurity, bioinformatics and computational biological applications. Speeding up the pattern matching task is an essential step for the success of the aforementioned applications. Wu–Manber algorithm is one of the fastest and most widely used algorithms for multi-pattern matching. Many researchers focused on improving the performance of Wu–Manber algorithm and this work presents a novel attempt parallelize Wu–Manber and make it suitable for multi-core machines. This paper uses Kahn processing network (KPN) model to effectively parallelize data and functional tasks. KPN suggests a parallel programming model that can be utilized in today’s multi-core machines. Hence, we use the KPN model to tailor the execution of Wu–Manber algorithm by breaking down the complexity of data sharing and task processing. The data parallelization is implemented using concurrent executions of multiple KPNs. In addition, task parallelization is achieved within each executing KPN. A single KPN consists of two threads, a producer thread and a consumer thread. The proposed KPN-based parallelization achieves up to 4× speedup over the serial implementation of the algorithm. Finally, the algorithm performance scales well with increasing workloads and the speedup up remains almost constant with increasing number of attack signatures.

XWM: a high-speed matching algorithm for large-scale URL rules in wireless surveillance applications

Large-scale high-speed URL matching is a key operation in many network security systems and surveillance applications in Wireless Sensor Networks. Classic string matching algorithms are unsuitable for large-scale URL filtering due to speed or memory consumption. This paper proposes an extend Wu-Manber algorithm (XWM) which takes advantage of the encoding characteristics of the URL greatly to improve the matching performance of the algorithm. It first adopts the pattern string window selection method to optimize Wu-Manber’s hash process, and then combines hash tables and associative containers to optimize the string comparison process. The experimental results on actual 10 million patterns show that XWM can achieve speeds that are twice as fast as traditional algorithms, especially when the shortest pattern string length is longer, it is more advantageous.

FILTERAPP: An Android-Based Profanity Control Application Using WU-Manber Algorithm with Fault-Tolerant Searching Capability

This study utilized a multi-pattern string matching algorithm to develop an effective android-based mobile profanity control application for children. Initially, three (3) algorithms were subjected to an experiment using a simulation program to determine which algorithm has the highest accuracy rate particularly in providing a fault-tolerant search result. To analyze the performance of each algorithm, pre-defined datasets were used in the experiment for comparative analysis. Three measures were used: Precision, Recall and F-Measure. Based on the conducted experiment, Wu-Manber algorithm has the highest accuracy rate against Aho-Corasick and Set-Horspool algorithms.

Multi-pattern matching with variable-length wildcards using suffix tree

Multi-pattern matching with variable-length wildcards is an interesting and important problem in bioinformatics, information retrieval and other domains. Most of the previously developed multi-pattern matching methods, such as famous Aho–Corasick and Wu–Manber algorithms, aimed to solve some classical string matching problems. However, these algorithms are not efficient for patterns with flexible wildcards or do-not-care characters. In this paper, we propose two efficient algorithms for multi-pattern matching with variable-length wildcards based on suffix tree, called MMST-L and MMST-S, according to the length of exact characters in a pattern. Experimental results show that the two MMST algorithms, in most cases, outperform other various versions of comparing algorithms.

Real-time positioning of a specific object in the big data environment

Real-time positioning of a specific object in the big data environment can improve the monitoring and management capacity for network data. For the real-time positioning of the specific object, it is necessary to quickly search the network data representing a specific object and match its pattern strings and compare the corresponding Internet protocol (IP) address of the matched network data with the IP address library in real time, so as to determine the position of the specific object. When a traditional method is used for pattern string matching, it will occupy a lot of memories and network resources, thereby reducing the positioning effect of the specific object in the big data environment. A positioning method for a specific object of high performance and multi-pattern matching based on three indexes in the big data network environment is proposed in this paper. Firstly, the initialization of Modified Wu-Manber (MWM) algorithm was carried out, and the algorithm was used to match the network data continuously. Secondly, the three indexes were used to improve the MWM algorithm, and the real-time and fast positioning of a specific object in the big data environment was completed by the Third Index Modified Wu-Manber (TMWM). The experimental results show that compared with the traditional method, the proposed algorithm reduces the pattern string matching scope of network data representing the specific object, improves the search speed of the specific object, and locates the specific object in the big data environment in an effective and rapid manner.

Word2vec and dictionary based approach for uyghur text filtering

With emerging of deep learning, the expression of words in computer has made major breakthroughs and the effect of text processing based on word vector has also been significantly improved. This paper maps all patterns into a more abstract vector space by Uyghur-Chinese dictionary and deep learning tool Word2vec, at first. Secondly, a similar pattern is found according the characteristics of the original pattern. Finally, texts are filtered using Wu-Manber algorithm. Experiments show that this method can get obvious filtering accuracy and recall of Uyghur text information improved.

Parallelizing Exact and Approximate String Matching via Inclusive Scan on a GPU

In this study, to substantially improve the runtimes of exact and approximate string matching algorithms, we propose a tribrid parallel method for bit-parallel algorithms such as the Shift-Or and Wu-Manber algorithms. Our underlying idea is to interpret bit-parallel algorithms as inclusive-scan operations, which allow these bit-parallel algorithms to run efficiently on a graphics processing unit (GPU); we achieve this speed-up here because inclusive-scan operations not only eliminate duplicate searches between threads but also realize a GPU-friendly memory access pattern that maximizes memory read/write throughput. To realize our ideas, we first define two binary operators and then present a proof regarding the associativity of these operators, which is necessary for the parallelization of the inclusive-scan operations. Finally, we integrate the inclusive-scan scheme into a previous segmentation-based scheme to maximize search throughput, identifying the best tradeoff point between synchronization cost and duplicate work. Through our experiments, we compared our proposed method with previous segmentation-based methods and indexing-based sequence aligners. For online string matching, our proposed method performed 6.7-16.7 times faster than previous methods, achieving a search throughput of up to 1.88 terabits per second (Tbps) on a GeForce GTX TITAN X GPU. We therefore conclude that our proposed method is quite effective for decreasing the runtimes of online string matching of short patterns.

Efficient Wu Manber String Matching Algorithm for Large Number of Patterns

String matching is one of the most important concept used in computer science in various real life applications like as Intrusion detection system, Data mining, Plagiarism detection system. There are many string matching algorithms which help to find pattern from the text. These algorithms are categorized in single string matching and multiple string matching. The Wu-Manber (WM) algorithm is multiple patterns algorithm which is the finest string matching algorithm. The performance of WM depends on various table build in pre processing phase these are prefix table, shift table and hase table. We introduce a new algorithm namely the Efficient Wu Manber algorithm (EWM) algorithm which is advance version of Wu Manber algorithm with respect to time. Efficient Wu-Manber Algorithm eliminate the prefix table which is unused most of the cases in wu manber, construct two shift table instead of single shift table and uses nonlinear data structure i.e. AVL tree instead of linear data structure i.e. linked list used in WM in Hash table, which reduce the traversed number of nodes to find exact match. The experimental results and analysis show that EWM algorithm has better performance as compare to WM and its existing improved algorithm and also better from various string matching tools.

A Comparative Study of Wu Manber String Matching Algorithm and its Variations

String matching algorithms is become one of the most important topic in the computer science world. These algorithms are used in many real world problems like as scanning the threat in intrusion detection system, finding the pattern in text mining, match the similarity of the document in the plagiarism detection system, recognition in bio informatics and so on. String Matching Algorithms are broadly categorized into single pattern string matching and multiple pattern string matching algorithms. To get the faster solution of the problem most of the cases multiple pattern matching is the best choice. Aho-Corasick, Shift-OR, Robin Karp but Wu Manber Algorithm is better choice because it search the pattern faster(take less time) as well as occupies less space. This paper discus the various pit falls of the wu manber algorithm with their detailed explanation. Also discuss the various improved wu manber algorithm and comparison between these algorithms.

Bit-parallel approximate pattern matching: Kepler GPU versus Xeon Phi

Advanced SIMD features on GPUs and Xeon Phis promote efficient long pattern search.A tiled approach to accelerating the Wu-Manber algorithm on GPUs has been proposed.Both the GPU and Xeon Phi yield two orders-of-magnitude speedup over one CPU core.The GPU-based version with tiling runs up to 2.9 × faster than the Xeon Phi version. Approximate pattern matching (APM) targets to find the occurrences of a pattern inside a subject text allowing a limited number of errors. It has been widely used in many application areas such as bioinformatics and information retrieval. Bit-parallel APM takes advantage of the intrinsic parallelism of bitwise operations inside a machine word. This approach typically encodes non-deterministic finite automaton (NFA) states or value differences between adjacent cells of a dynamic programming matrix in the form of bit arrays. Wu-Manber (WM) is a well-known bit-parallel APM algorithm, which simulates an NFA and gains parallel efficiency by performing multiple state updates within a machine word. An important parameter is the machine word size (e.g. 32 or 64?bits for CPUs). Due to increasing vector capabilities, efficient mapping of bit-parallel APM algorithms onto modern high performance computing architectures is an interesting research topic. Prominent examples are Xeon Phi coprocessors and CUDA-enabled GPUs, which provide words of size 512?bits (by means of vector registers) and 1024?bits (by means of warps), respectively. In this paper, we investigate mappings of the WM algorithm onto these two accelerator types. Both architectures are able to achieve around two orders-of-magnitude speedups compared to a single-threaded CPU implementation. Moreover, our tile-based implementation on a GeForce Titan graphics card runs up to 2.9 × faster than our implementation on an Intel Xeon Phi 5110P. Source code is available at http://xbitpar.sourceforge.net.

A Hybrid Parallel Implementation of the Aho–Corasick and Wu–Manber Algorithms Using NVIDIA CUDA and MPI Evaluated on a Biological Sequence Database

Multiple matching algorithms are used to locate the occurrences of patterns from a finite pattern set in a large input string. Aho–Corasick and Wu–Manber, two of the most well known algorithms for multiple matching require an increased computing power, particularly in cases where large-size datasets must be processed, as is common in computational biology applications. Over the past years, Graphics Processing Units (GPUs) have evolved to powerful parallel processors outperforming Central Processing Units (CPUs) in scientific calculations. Moreover, multiple GPUs can be used in parallel, forming hybrid computer cluster configurations to achieve an even higher processing throughput. This paper evaluates the speedup of the parallel implementation of the Aho–Corasick and Wu–Manber algorithms on a hybrid GPU cluster, when used to process a snapshot of the Expressed Sequence Tags of the human genome and for different problem parameters.

An efficient parallel algorithm for exact multi‐pattern matching

AbstractThis paper presents a parallel algorithm Parallel Extended Bloom Filter (PEBF) for exact multi‐pattern matching based on Bloom filter. To improve the throughput and parallelism of the algorithm, we divided the pattern set into N subsets where the length of patterns is the same, and different subsets would not intersect each other. We construct an EBF for each subset and use N threads to simultaneously process the subsets in parallel. We implement our solution on the graphics processing unit, called G‐PEBF. Experimental results demonstrate that PEBF performs better than the Wu–Manber (WM) algorithm in terms of time and space. And G‐PEBF outperforms the G‐WM (WM algorithm implemented on graphics processing unit). The speedup of G‐PEBF is up to 60 times at peak performance and almost 10 times at worst performance to the PEBF algorithm. Copyright © 2014 John Wiley & Sons, Ltd.

An Improved Wu-Manber Multiple Patterns Matching Algorithm for Mobile Internet Content Security Audit in a Chinese Environment

The rapid progress of mobile Internet has brought about wide range of applications of security audit systems. As a part of security audit system, content security audit is crucial for ensuring the reliability and security for system applications. In this paper, we propose a novel multiple patterns matching algorithm WMMA based on the Chinese context (Wu-Manber Algorithm for Mobile Internet Security Audit) to realize auditing the interactive content in the mobile Internet. Our improved algorithm based on the efficient WM (Wu-Manber) multiple patterns matching algorithm, which can improve the insufficient of the WM when dealing with Chinese context. The simulation results showed that the improved algorithm is more suitable for the mobile Internet content security auditing of Chinese language environment, which has higher operation efficiency.

FPGA Based Firewall using Embedded Processor for Vulnarability Packet Detection

<p>This paper describes the design of high performance packet filtering firewall using embedded system. An FPGA (field programmable gate array) platform has been used for implementation and analysing the network firewall. It is capable of accepting real time changes. This network security application has an ability to perform powerful protection against unwanted data packets such as virus attack, spam in e-mails, hackers, worms, spyware unauthorized contents. However the firewalls don’t address the difficulty of unwanted data packets intrusion. The ultimate aim of this work is to create a systematic way of approach for unwanted packets discard in a network system. We use a specially trained algorithms such as Wu-manber algorithms (high performance, multi-pattern matching), bloom filter algorithm (space efficient data structure for testing an element in the set.Our design is mainly based on machine learning and artificial intelligence. This gives a high efficiency, improved performance and high ability of packet detection with less contribution of time in an effective way.</p>

BLAST: B‐LAyered bad‐character SHIFT tables for high‐speed pattern matching

In this study, the authors propose a new multi-pattern matching algorithm, called BLAST (B-LAyered bad-character Shift Tables with a single-byte search unit), which considers space-time tradeoff in the context of shift values during the search. Here, the term ‘bad character’ is a character that causes a mismatch. While checking multiple bytes in scanning the text at a time, the BLAST algorithm overcomes the reduction of the average shift value in a typical search, which is caused by the dependency on the multi-byte search unit (MBSU) and the large frequency of the last character of the given patterns. From the theoretical analysis, the authors validate the correctness of the BLAST algorithm. Also, from the experimental results across different setups, the authors show that the BLAST algorithm provides the faster search time than the other algorithms. For example, the authors obtain an enhancement by as much as 212.41% on average for various numbers of attack patterns and attack traffic conditions compared with that of the modified Wu-Manber algorithm. In addition, it is shown that the BLAST algorithm drastically reduces the amount of memory required for constructing the shift table based on a MBSU from 64 KB to 1 KB.

Improve Wu-Manber Algorithm for Multiple Pattern Matching Time Efficiency Optimization

Improve Wu-Manber Algorithm for Multiple Pattern Matching Time Efficiency Optimization

Function and Data Parallelization of Wu-Manber Pattern Matching for Intrusion Detection Systems

The safeguarding networks from malicious activities and intrusions continues to be one of the most important aspects in network security. Intrusion Detection Systems (IDSs) play a fundamental role in network protection. Unfortunately, the speeds of existing IDSs are unable to keep up with the rapid increases in network speeds and attack complexities. Fortunately, parallel computing on multi-core systems can lend a helping hand mitigating the performance gap. In this paper, novel and effective parallel implementations of the Wu-Manber (WM) algorithm for signature based detection system are proposed, implemented, and evaluated. The proposed function and data parallel algorithms prove to be effective in terms of execution time reduction and load balancing, thus providing swift intrusion detection at increased network bandwidths. The algorithm achieves an optimal load balance and an average speedup of 2 for four cores.

Multi-Threading and Suffix Grouping on Massive Multiple Pattern Matching Algorithm

The widely used multiple pattern matching algorithms experience severe performance degradation when the number of patterns to match increases. In light of this fact, this paper presents a multi-threaded multiple pattern matching algorithm to overcome the performance degradation; this algorithm presents two additional improvements on the original Wu–Manber algorithm. First, the proposed algorithm employs a multi-threaded execution model to parallelize the pattern matching operation on multi-core processors. Second, the patterns to be searched are distributed over multiple threads according to the pattern similarity. For this purpose, the proposed algorithm groups the target patterns on the basis of their suffixes and distributes the patterns over multiple threads. Through experiments and performance analysis, our algorithm shows a significant performance gain as compared with the original Wu–Manber algorithm and the previously proposed multi-threaded pattern matching on massive pattern sets of size exceeding 5000. The results obtained from the pattern matching operation using eight cores show much improved execution time, which is nearly 14.9 times faster on average than that of the conventional Wu–Manber algorithm. It is demonstrated that the proposed idea improves the overall performance by reducing the amount of workload on a single thread through multi-threading and an efficient data distribution policy.