Pattern Matching Engine Research Articles

MoTion: A new declarative object matching approach in Pharo

Pattern matching is an expressive way of matching data and extracting pieces of information from it. The recent inclusion of pattern matching in the Java and Python languages highlights that such a facility is more and more adopted by developers for everyday development. Other main stream programming languages also offer pattern matching capabilities as part of the language (Rust, Scala, Haskell, and OCaml), with different degrees of expressivity in what can be matched. In the meantime, in graphs, pattern matching takes a slightly different turn; it enhances the expressivity of the patterns that can be defined. Smalltalk currently offers little pattern matching capability to find specific objects inside a large graph of objects using a declarative pattern. In Pharo, the closest library to classical pattern matching that exists is the RBParseTreeSearcher, which allows to express specialized patterns over a Pharo Abstract Syntax Tree to find some inner node. The question arises of what features a flexible pattern matching language should have. In this paper, we review the features found in different existing pattern matching languages, both in General Purpose Languages (like Java) and in declarative graph pattern matching languages. We then describe MoTion, a new pattern matching engine for Pharo smalltalk, combining all these features. We discuss some aspects of MoTion’s implementation and illustrate its use with real case examples.

Deep learning-based feature extraction and optimizing pattern matching for intrusion detection using finite state machine

Deep learning has gained popularity for feature extraction in the field of Network Intrusion Detection and Prevention System (NIDPS) to extract the patterns matching and secure the networks by detecting the unknown and malicious activities. The malicious activities and security attacks are disturbing the normal operations of networks. The new attacks are difficult to monitor due to their new features and pattern types. Different types of methods have been adopted for feature extraction and pattern matching. Deep learning is one of them and subfield of machine learning where it solves the optimization issues layer-wise by looking at the deep structure. On the other hand, pattern matching is another considerable method for intrusion detection due to its variety of applications. However, pattern matching methods are consuming more than 70% of the total running time and cause overhead. In this paper, we propose two methods including Deep Learning-based Feature Extraction (DLFE) and Optimization of Pattern Matching (OPM) for NIDPS systems to optimizes the pattern matching engine in intrusion detection. The experiments are performed by using the snort ruleset for pattern matching and obtained the results. The experiment results show the better performance of proposed methods in terms of time, throughput, and memory.

Multiscale Laplacian graph kernel combined with lexico-syntactic patterns for biomedical event extraction from literature

Bio-event extraction is an extensive research area in the field of biomedical text mining, this focuses on elaborating relationships between biomolecules and can provide various aspects of their nature. Bio-event extraction plays a vital role in biomedical literature mining applications such as biological network construction, pathway curation, and drug repurposing. Extracting biological events automatically is a difficult task because of the uncertainty and assortment of natural language processing such as negations and speculations, which provides further room for the development of feasible methodologies. This paper presents a hybrid approach that integrates an ensemble-learning framework by combining a Multiscale Laplacian Graph kernel and a feature-based linear kernel, using a pattern-matching engine to identify biomedical events with arguments. This graph-based kernel not only captures the topological relationships between the individual event nodes but also identifies the associations among the subgraphs for complex events. In addition, the lexico-syntactic patterns were used to automatically discover the semantic role of each word in the sentence. For performance evaluation, we used the gold standard corpora, namely BioNLP-ST (2009, 2011, and 2013) and GENIA-MK. Experimental results show that our approach achieved better performance than other state-of-the-art systems.

A Dual-Port 8-T CAM-Based Network Intrusion Detection Engine for IoT

This letter presents an energy- and memory-efficient pattern-matching engine for a network intrusion detection system (NIDS) in the Internet of Things. Tightly coupled architecture and circuit co-designs are proposed to fully exploit the statistical behaviors of NIDS pattern matching. The proposed engine performs pattern matching in three phases, where the phase-1 prefix matching employs reconfigurable pipelined automata processing to minimize memory footprint without loss of throughput and efficiency. The processing elements utilize 8-T content-addressable memory (CAM) cells for dual-port search by leveraging proposed fixed-1s encoding. A 65-nm prototype demonstrates best-in-class 1.54-fJ energy per search per pattern byte and 0.9-byte memory usage per pattern byte.

Algorithms to speedup pattern matching for network intrusion detection systems

High-speed network intrusion detection systems (NIDSes) commonly employ TCAMs for fast pattern matching, and parallel TCAM-based pattern matching algorithms have proven promising to achieve even higher line rate. However, two challenges impede parallel TCAM-based pattern matching engines from being truly scalable, namely: (1) how to implement fine-grained parallelism to optimize load balancing and maximize throughput, and (2) how to reconcile between the performance gain and increased power consumption both due to parallelism. In this paper, we propose two techniques to answer the above challenges yielding an ultra-scalable NIDS. We first introduce the concept of negative pattern matching, by which we can splice flows into segments for fine-grained load balancing and optimized parallel speedup while ensuring correctness. negative pattern matching (NPM) also dramatically reduces the number of Ternary Content Addressable Memory (TCAM) lookups thus reducing the power consumption. Then we propose the idea of exclusive pattern matching, which divides the rule sets into subsets; each subset is queried selectively and independently given a certain input without affecting correctness. In concert, these two techniques improve both the pattern matching throughput and scalability in any scenario. Our experimental results show that up to 90% TCAM lookups can be saved, at the cost of merely 10% additional 2-byte index table lookups in the SRAM.

A malicious pattern detection engine for embedded security systems in the Internet of Things.

With the emergence of the Internet of Things (IoT), a large number of physical objects in daily life have been aggressively connected to the Internet. As the number of objects connected to networks increases, the security systems face a critical challenge due to the global connectivity and accessibility of the IoT. However, it is difficult to adapt traditional security systems to the objects in the IoT, because of their limited computing power and memory size. In light of this, we present a lightweight security system that uses a novel malicious pattern-matching engine. We limit the memory usage of the proposed system in order to make it work on resource-constrained devices. To mitigate performance degradation due to limitations of computation power and memory, we propose two novel techniques, auxiliary shifting and early decision. Through both techniques, we can efficiently reduce the number of matching operations on resource-constrained systems. Experiments and performance analyses show that our proposed system achieves a maximum speedup of 2.14 with an IoT object and provides scalable performance for a large number of patterns.

An algorithm for generating model-sensitive search plans for pattern matching on EMF models

In this paper, we propose a new model-sensitive search plan generation algorithm to speed up the process of graph pattern matching. This dynamic-programming-based algorithm, which is able to handle general n-ary constraints in an integrated manner, collects statistical data from the underlying EMF model and uses this information for optimization purposes. Additionally, the search plan generation algorithm itself and its runtime effects on the pattern matching engine have been evaluated by complexity analysis techniques and by quantitative performance measurements, respectively.

Exploring the design space of programmable regular expression matching accelerators

State-of-the-art regular expression (regex) accelerators combine parallel programmable state machines with cascaded, wide-issue instruction processors to improve the storage efficiency and the processing rates, while preserving the programmability. The pattern-matching engine (PME) included on the IBM PowerEN(TM) (Edge-of-Network) processor is one such design, and can be used as an architectural template for a broad design-space exploration. The regex compiler is a key component of such an exploration, involving sophisticated transformations to map large sets of complex regexs to the memory contents and the configuration registers of the accelerator hardware. The design space is explored by varying the main microarchitectural parameters, including the memory size, the number of parallel state machines, and the parameters of the instruction processor. While the design-space exploration confirms the main architectural choices of the PME, it also shows that further optimization is possible by eliminating the bottlenecks in the instruction dispatch mechanisms, which results in an up to 50% reduction in the storage requirements. The design-space exploration utilizes a parameterizable and synthesizable hardware model to evaluate the effects the microarchitectural choices have on the chip area and operating frequency. The synthesis results demonstrate the scalability of the optimization chosen and the need to incorporate these choices into future regex accelerator architectures.

RECONFIGURABLE SELF-ADDRESSABLE MEMORY-BASED FSM A SCALABLE INTRUSION DETECTION ENGINE

One way to detect and thwart a network attack is to compare each incoming packet with predefined patterns, also Called an attack pattern database, and raise an alert upon detecting a match. This article presents a novel pattern-matching Engine that exploits a memory-based, programmable state machine to achieve deterministic processing rates that are Independent of packet and pattern characteristics. Our engine is a self addressable memory based finite state machine (samFsm), whose current state coding exhibits all its possible next states. Moreover, it is fully reconfigurable in that new attack Patterns can be updated easily. A methodology was developed to program the memory and logic. Specifically, we merge “non-equivalent” states by introducing “super characters” on their inputs to further enhance memory efficiency without Adding labels. This is the most high speed self addressable memory based fsm.sam-fsm is one of the most storage-Efficient machines and reduces the memory requirement by 60 times. Experimental results are presented to demonstrate the Validity of sam-fsm.

MATCH: An atom‐typing toolset for molecular mechanics force fields

We introduce a toolset of program libraries collectively titled multipurpose atom-typer for CHARMM (MATCH) for the automated assignment of atom types and force field parameters for molecular mechanics simulation of organic molecules. The toolset includes utilities for the conversion of multiple chemical structure file formats into a molecular graph. A general chemical pattern-matching engine using this graph has been implemented whereby assignment of molecular mechanics atom types, charges, and force field parameters are achieved by comparison against a customizable list of chemical fragments. While initially designed to complement the CHARMM simulation package and force fields by generating the necessary input topology and atom-type data files, MATCH can be expanded to any force field and program, and has core functionality that makes it extendable to other applications such as fragment-based property prediction. In this work, we demonstrate the accurate construction of atomic parameters of molecules within each force field included in CHARMM36 through exhaustive cross validation studies illustrating that bond charge increment rules derived from one force field can be transferred to another. In addition, using leave-one-out substitution it is shown that it is also possible to substitute missing intra and intermolecular parameters with ones included in a force field to complete the parameterization of novel molecules. Finally, to demonstrate the robustness of MATCH and the coverage of chemical space offered by the recent CHARMM general force field (Vanommeslaeghe, et al., J Comput Chem 2010, 31, 671), one million molecules from the PubChem database of small molecules are typed, parameterized, and minimized.

CPM: CUCKOO-BASED PATTERN MATCHING APPLIED FOR NIDS

This paper describes the Cuckoo-based Pattern Matching (CPM) engine which based on a recently developed hashing algorithm called Cuckoo Hashing. We implemented the improved parallel Cuckoo Hashing suitable for hardware-based multi-pattern matching with arbitrary length. CPM is scalable with multi-character per clock cycle to sustain higher throughput rates with lower hardware resources. With the power of massively parallel processing, the speedup of CPM is up to 128X as compared with serial Cuckoo implementation. Compared to other hardware systems, CPM is far better in performance and save 30% of the area compared with the best system.

Pattern matching engine based on multi-dimensional bloom filters

Concerning the defects of traditional rule matching engine,a solution using multi-dimensional bloom filters based on Field Programmable Gate Array(FPGA) was proposed.The rule matching engine was designed to process both packet header and payload in parallel.The suspicious strings were picked up by the multi-dimensional bloom filters engines,and then sent to bit-split state machine for verification.The experimental results demonstrate that the false positive probability of the engine is reduced by using the multi-dimensional bloom filters,which results in a higher throughput.

Deterministic finite automata characterization and optimization for scalable pattern matching

Memory-based Deterministic Finite Automata (DFA) are ideal for pattern matching in network intrusion detection systems due to their deterministic performance and ease of update of new patterns, however severe DFA memory requirements make it impractical to implement thousands of patterns. This article aims to understand the basic relationship between DFA characteristics and memory requirements, and to design a practical memory-based pattern matching engine. We present a methodology that consists of theoretical DFA characterization, encoding optimization, and implementation architecture. Results show the validity of the characterization metrics, effectiveness of the encoding techniques, and efficiency of the memory-based pattern engines.

Hybrid pattern matching for trusted intrusion detection

AbstractIntrusion Detection Systems (IDSs) rely on pattern matching to detect and thwart a network attack by comparing packets with a database of known attack patterns. The key requirements of trusted intrusion detection are accurate pattern matching, adaptive, and reliable reconfiguration for new patterns. To address these requirements, this paper presents a trusted intrusion detection by utilizing hybrid pattern matching engines: FPGA‐based and multicore‐based pattern matching engine. To achieve synchronization of these two pattern matching engines, methodologies including multi‐threading DFA and clustered state coding have been developed. These hybrid pattern matching engines increases the reliability and trustworthy of intrusion detection systems. Copyright © 2009 John Wiley & Sons, Ltd.

Patterns of behaviour [electronics design

SINCE THE end of the 1990s, the semiconductor industry has worked on finding ever more sophisticated ways of drawing tinier and tinier features on the surface of silicon wafers. The wavelengths of light used can now be five times longer than the features themselves because it has proven too difficult and expensive to move from deep ultraviolet wavelengths around 200nm long to the extreme ultraviolet range. Even in the late 1990s, feature sizes rapidly approached the wavelength commonly used in lithography 248nm. As feature sizes get closer to the source wavelength, diffraction around the mask elements gets worse. The blurring caused by this diffraction threatened to distort the on-chip features or even make them disappear in some cases, rendering the integrated circuit (IC) unusable.Pattern matching can let us return to using something more intuitive. Pattern-matchingtechnology makes it possible to identify, isolate and define specific geometric configurations visually. Once recognised and defined, these patterns can be added to a pattern library that can be used by a pattern matching engine to automatically scan designs for matching patterns.

A Memory-Efficient Pattern Matching with Hardware-Based Bit-Split String Matchers for Deep Packet Inspection

This paper proposes a hardware-based parallel pattern matching engine using a memory-based bit-split string matcher architecture. The proposed bit-split string matcher separates the transition table from the state table, so that state transitions towards the initial state are not stored. Therefore, total memory requirements can be minimized.

A memory-efficient heterogeneous parallel pattern matching scheme in deep packet inspection

This paper presents for hardware-based parallel pattern matching scheme that adopts heterogeneous bit-split string matchers for deep packet inspection (DPI) devices. Considering the pattern lengths, a set of target patterns is partitioned into two subsets for short and long patterns. By adopting the appropriate bit-split string matcher types for the two subsets, the memory requirements can be optimized for the bit-split parallel pattern matching engine. Experimental results show that the total memory requirements decrease by 39.40% and 20.52%, in comparison with the existing bit-split pattern matching approaches.

Optimized memory based accelerator for scalable pattern matching

One of the most promising techniques to detect and thwart a network attack in a network intrusion detection system is to compare each incoming packet with pre-defined attack patterns. This comparison can be performed by a pattern matching engine which has several key requirements including scalability to line rates of network traffic and easy updating of new attack patterns. Memory-based deterministic finite automata meet these requirements, however their storage requirement will grow exponentially with the number of patterns which makes it impractical for implementation. In this paper, we propose a customized memory-based pattern matching engine, whose storage requirement linearly increases with the number of patterns. The basic idea is to allocate one memory slot for each state instead of each edge of the deterministic finite automaton. To demonstrate this idea, we have developed two customized memory decoders. We evaluate them by comparing with a traditional approach in terms of programmability and resource requirements. We also examine their effectiveness for different optimized deterministic finite automata. Experimental results are presented to demonstrate the validity of our proposed approach.

Hardware Architecture for High-Performance Regular Expression Matching

This paper presents a bitmap-based hardware architecture for the Glushkov nondeterministic finite automaton (G-NFA), which recognizes a given regular expression. We show that the inductions of the functions needed to construct the G-NFA can be generalized to include other special symbols commonly used in extended regular expressions such as the POSIX 1003.2 format. Our proposed implementation can detect the ending positions of all substrings of an input string T, which start at arbitrary positions of T and belong to the language defined by the given regular expression. To achieve high performance, the implementation is generalized to the NFA, which processes K symbols in each operation cycle. We provide an efficient solution for the boundary condition when the length of the input string is not an integral multiple of K. Compared with previous designs, our proposed architecture is more flexible and programmable because the pattern matching engine uses memory rather than logic.

PAMELA: Pattern Matching Engine with Limited-Time Update for NIDS/NIPS

Several hardware-based pattern matching engines for network intrusion/prevention detection systems (NIDS/NIPSs) can achieve high throughput with less hardware resources. However, their flexibility to update new patterns is limited and still challenging. This paper describes a PAttern Matching Engine with Limited-time updAte (PAMELA) engine using a recently proposed hashing algorithm called Cuckoo Hashing. PAMELA features on-the-fly pattern updates without reconfiguration, more efficient hardware utilization, and higher performance compared with other works. First, we implement the improved parallel exact pattern matching with arbitrary length based on Cuckoo Hashing and linked-list technique. Second, while PAMELA is being updated with new attack patterns, both stack and FIFO are utilized to bound insertion time due to the drawback of Cuckoo Hashing and to avoid interruption of input data stream. Third, we extend the system for multi-character processing to achieve higher throughput. Our engine can accommodate the latest Snort rule-set, an open source NIDS/NIPS, and achieve the throughput up to 8.8 Gigabit per second while consuming the lowest amount of hardware. Compared to other approaches, ours is far more efficient than any other implemented on Xilinx FPGA architectures.