Deep Packet Inspection Systems Research Articles

Dual Cuckoo Filter with a Low False Positive Rate for Deep Packet Inspection

Network traffic control and classification have become increasingly dependent on deep packet inspection (DPI) approaches, which are the most precise techniques for intrusion detection and prevention. However, the increasing traffic volumes and link speed exert considerable pressure on DPI techniques to process packets with high performance in restricted available memory. To overcome this problem, we proposed dual cuckoo filter (DCF) as a data structure based on cuckoo filter (CF). The CF can be extended to the parallel mode called parallel Cuckoo Filter (PCF). The proposed data structure employs an extra hash function to obtain two potential indices of entries. The DCF magnifies the superiority of the CF with no additional memory. Moreover, it can be extended to the parallel mode, resulting in a data structure referred to as parallel Dual Cuckoo filter (PDCF). The implementation results show that using the DCF and PDCF as identification tools in a DPI system results in time improvements of up to 2% and 30% over the CF and PCF, respectively.

Accelerating Pattern Matching Using a Novel Multi-Pattern-Matching Algorithm on GPU

Nowadays, almost all network traffic is encrypted. Attackers hide themselves using this traffic and attack over encrypted channels. Inspections performed only on packet headers and metadata are insufficient for detecting cyberattacks over encrypted channels. Therefore, it is important to analyze packet contents in applications that require control over payloads, such as content filtering, intrusion detection systems (IDSs), data loss prevention systems (DLPs), and fraud detection. This technology, known as deep packet inspection (DPI), provides full control over the communication between two end stations by keenly analyzing the network traffic. This study proposes a multi-pattern-matching algorithm that reduces the memory space and time required in the DPI pattern matching compared to traditional automaton-based algorithms with its ability to process more than one packet payload character at once. The pattern-matching process in the DPI system created to evaluate the performance of the proposed algorithm (PA) is conducted on the graphics processing unit (GPU), which accelerates the processing of network packets with its parallel computing capability. This study compares the PA with the Aho-Corasick (AC) and Wu–Manber (WM) algorithms, which are widely used in the pattern-matching process, considering the memory space required and throughput obtained. Algorithm tables created with a dataset containing 500 patterns use 425 and 688 times less memory space than those of the AC and WM algorithms, respectively. In the pattern-matching process using these tables, the PA is 3.5 and 1.5 times more efficient than the AC and WM algorithms, respectively.

Assessment of Deep Packet Inspection System of Network traffic and Anomaly Detection

Deep packet SSL inspection is a process that involves decrypting and inspecting SSL encrypted network traffic in order to detect and prevent security threats. With the increasing use of SSL encryption, it has become difficult for traditional network security solutions to inspect encrypted traffic for threats. Deep packet SSL inspection addresses this problem by decrypting the SSL traffic, inspecting it for threats, and then re-encrypting it before forwarding it to its destination. This process involves the use of SSL certificates that mimic the real ones used by the servers, as well as SSL inspection rules that specify which traffic should be decrypted and inspected. Deep packet SSL inspection can be a complex and resource- intensive process, and must be performed carefully to avoid legal or ethical issues related to the interception and inspection of encrypted traffic. However, it is a powerful tool for protecting networks from security threats, and can help organizations detect and prevent attacks that would otherwise go unnoticed.

Вызовы в реализации систем глубокого анализа сетевого трафика методом полного протокольного декодирования

This paper presents a summary of experience in developing the deep packet inspection system using full protocol decoding. The paper reviews the challenges encountered during implementation and provides a high-level overview of the solutions to these issues. The challenges can be grouped into two groups. The first group is related to the fundamental tasks which must be addressed when implementing full protocol decoding systems. This includes ensuring correct protocol parsing, which involves identifying and interpreting protocol headers and fields correctly. Moreover, it is necessary to ensure the processing of fragmented packets and the assembly of fragments into the original message. Additionally, the processing and analysis of encrypted traffic is a crucial task that may require the use of specialized algorithms and tools. The second group of problems is related to optimizing the process of full protocol decoding to ensure high-speed traffic processing, as well as supporting new protocols and the ability to add user-defined extensions. While there are open-source systems that address some of the primary issues associated with full protocol decoding, there may be a need for additional effort and specialized solutions to efficiently operate and expand the functionality of such systems. Although implementing deep network traffic analysis tools using full protocol decoding requires the use of advanced hardware and software technologies, the benefits of such analysis are significant. This approach provides a more complete understanding of network traffic patterns and enables more effective detection and prevention of cyber-attacks. It also allows for more accurate monitoring of network performance and the identification of potential bottlenecks or other issues that may impact network efficiency. In this article, we also emphasize the importance of system architecture development and implementation to ensure the successful deployment of deep network traffic analysis tools using full protocol decoding. At last, we conducted an experiment where several advanced optimizations were implemented in the system that had already solved primary issues. These optimizations related to working with memory, based on the features of the traffic processing scheme. By results, we evaluated significant performance improvement in solving secondary tasks, described in this work.

Using Deep Packet Inspection Data to Examine Subscribers on the Network

This article proposes the creation of the deep packet inspection (DPI) dataset to study subscribers’ behavior on the network, applying ensemble learning to this dataset, and comparing it with the UGRansome dataset. The subscriber can be thought of as a person or a group of users using a network service or connectivity. The DPI features represent the subscriber network usage, and the ensemble learning approach is implemented on the DPI dataset to predict the subscriber's service category on the network. The classification and prediction problem addressed on the DPI dataset reached a precision of 100%. The paper predicts that the web and streaming categories with Netflix, Facebook, and YouTube services will be the most utilized in the next few years. This study will lead to a better understanding of the idiosyncratic behavior of active subscribers on the network, exposing novel network anomalies and facilitating the development of novel DPI systems.

Software Methodology for Estimating the Efficiency of the Hardware Composition of Deep Packet Inspection System Using the Modernized Hooke ‒ Jeeves Method

Deep packet inspection systems on communication networks are used to identify the application generating a specific traffic flow. The issues related to modeling and design of deep packet inspection systems remain poorly understood. In this paper, a software technique for evaluating the effectiveness of the hardware composition of the servers of the deep packet inspection system is presented, using a mathematical model of such a system and software search methods. The description of the program search by the maximum element method and the Hook - Jeeves method is given. A modernization of the Hook-Jeeves method for a monotonically decreasing function is proposed. Comparison of the methods by the number of search steps is performed.

A Software Deep Packet Inspection System for Network Traffic Analysis and Anomaly Detection.

In this paper, to solve the problem of detecting network anomalies, a method of forming a set of informative features formalizing the normal and anomalous behavior of the system on the basis of evaluating the Hurst (H) parameter of the network traffic has been proposed. Criteria to detect and prevent various types of network anomalies using the Three Sigma Rule and Hurst parameter have been defined. A rescaled range (RS) method to evaluate the Hurst parameter has been chosen. The practical value of the proposed method is conditioned by a set of the following factors: low time spent on calculations, short time required for monitoring, the possibility of self-training, as well as the possibility of observing a wide range of traffic types. For new DPI (Deep Packet Inspection) system implementation, algorithms for analyzing and captured traffic with protocol detection and determining statistical load parameters have been developed. In addition, algorithms that are responsible for flow regulation to ensure the QoS (Quality of Services) based on the conducted static analysis of flows and the proposed method of detection of anomalies using the parameter Hurst have been developed. We compared the proposed software DPI system with the existing SolarWinds Deep Packet Inspection for the possibility of network traffic anomaly detection and prevention. The created software components of the proposed DPI system increase the efficiency of using standard intrusion detection and prevention systems by identifying and taking into account new non-standard factors and dependencies. The use of the developed system in the IoT communication infrastructure will increase the level of information security and significantly reduce the risks of its loss.

Constructing Bloom filters by reconfigurable means for solving information security tasks

Recently, the string matching task became a performance bottleneck in network intrusion detection, anti-virus, anti-worms and other signature-based information security systems. Unlike the firewall, NIDS examines not only packet headers, but also the packet bodies, that is, performs the deep packet inspection (DPI). The multi-pattern string matching task is a specific type of string matching functionality performed in DPI systems to search an input stream for a set of patterns rather than a single pattern. Due to rising traffic rates, increasing number and sophistication of attacks and the collapse of Moore's law for sequential processing, traditional software solutions can no longer meet the high requirements of today’s security challenges. Therefore, hardware approaches are proposed to accelerate pattern matching. Combining the flexibility of software and the near-ASIC performance, reconfigurable hardware devices based on Field Programmable Gate Arrays (FPGA) have become increasingly popular for this purpose. There are three main approaches to fulfill the computation-intensive multi-pattern string matching task using FPGA. The techniques (and underlying technologies) of these approaches are: content addressable memory (based on digital comparators), Bloom filter (based on hash-functions) and Aho-Corasick Algorithm (based on finite automata). This article is devoted to the investigation of the second approach - Bloom filter. The features (advantages and disadvantages) of this approach in terms of resource costs, speed/throughput parameters, as well as scaling parameters are explored. The basic scheme and its modifications are considered. The performance characteristics, problems and challenges of implementing this approach on reconfigurable accelerators as well as ways to overcome them are analyzed. The obtained results allow improving the technical parameters when designing pattern matching modules, which are speed-critical components of the signature-based information protection systems hardware. The knowledge obtained in this study allows developers to create more effective reconfigurable means for information security.

FPGA-Assisted DPI Systems: 100 Gbit/s and Beyond

Carrying out deep packet inspection (DPI) in aggregated network connections remains a continuous requirement even though the line rate reaches and exceeds 100 Gb/s. The increasing packet-arrival rate necessitates efficient solutions for on-the-fly packet parsing, packet classification, and distribution for parallelized, software-based payload inspection. Inspection complexity and real-time processing are competing requirements. The deep analysis capabilities of software-based approaches can be enhanced by hardware-based support on time-critical packet parsing and classification. Moreover, some payload inspection tasks can be carried out in hardware as well, further reducing the resources spent on software-based solutions. This paper aims at presenting the state-of-the-art and describing a set of best practices in field programmable gate arrays (FPGA)-based packet processing, which can be applied for DPI-related tasks at 100 Gb/s and beyond. Accordingly, we provide an architectural view of the DPI systems throughout the paper. Besides summarizing the limitations of hardware- and software-based solutions for the three processing phases within a DPI system (packet parsing, packet classification, and payload inspection), this paper reveals the possible trade-offs for choosing the different technical approaches. These limitations include operating frequency, bus size, available memory, on-chip physical resources for hardware-based implementations, and CPU time for software-based solutions.

COIN: A fast packet inspection method over compressed traffic

Matching multiple patterns simultaneously is a key technique in Deep Packet Inspection systems, such as firewall, Intrusion Detection Systems, etc. However, most web services nowadays tend to compress their traffic for less data transferring and better user experience, which has challenged the original multi-pattern matching method that work on raw content only. The straightforward solutions directly match decompressed data which multiply the data to be matched. The state-of-the-art works skip scanning some data in compressed segments, but still exist the redundant checking, which are not efficient enough. In this paper, we propose COmpression INspection (COIN) method for multi-pattern matching over compressed traffic. COIN does not recheck the patterns within compressed segment if it has been matched before, so as to further improve the performance of matching, we have collected real traffic data from Alexa top sites and performed the experiments. The evaluation results show that COIN achieves 20.3% and 17.0% in the average of improvement than the state-of-the-art approaches on the string and regular expression matching with real traffic and rule sets.

Improved deep packet inspection in data stream detection

Finite state automata are widely used in firewalls, data detection and content audit systems to match complex sets of regular expressions in network packets. However, with the continuous increase in the types of network contents and network traffics in recent years, the deep packet inspection systems based on finite state automata also require regular engines for less memory consumption and higher operating speed. This paper analyzes the feature and problem of finite state automata and improves non-deterministic finite automata by reducing the conversion edge to reduce the memory usage. The experiment results which are made by real-world dataset show that the memory usage is reduced more than half.

Efficient Memristor-Based Architecture for Intrusion Detection and High-Speed Packet Classification

Deep packet inspection (DPI) is a critical component to prevent intrusion detection. This requires a detailed analysis of each network packet header and body. Although this is often done on dedicated high-power servers in most networked systems, mobile systems could potentially be vulnerable to attack if utilized on an unprotected network. In this case, having DPI hardware on the mobile system would be highly beneficial. Unfortunately, DPI hardware is generally area and power consuming, making its implementation difficult in mobile systems. We developed a memristor crossbar-based approach, inspired by memristor crossbar neuromorphic circuits, for a low-power, low-area, and high-throughput DPI system that examines both the header and body of a packet. Two key types of circuits are presented: static pattern matching and regular expression circuits. This system is able to reduce execution time and power consumption due to its high-density grid and massive parallelism. Independent searches are performed using low-power memristor crossbar arrays giving rise to a throughput of 160Gbps with no loss in the classification accuracy.

QCF for deep packet inspection

Deep packet inspection (DPI) acts as a tool to control and classify incoming network traffic depending on users,content, applications and becomes a very important aspect of every network today. Instead of utilising only incoming packet header information, Internet service providers utilise DPI for security purposes, flow management, and routing. The high-performance challenges that face DPI techniques to manage line-speed packet processing with restricted available memory are the increasing traffic volumes and link rates. Bloom (BF) and quotient (QF) filters are effective randomised data structures for performing approximate membership queries. They have been employed in many networking applications including DPI. Nevertheless, they consume more calculations which degrade the system performance. Consequently, in this paper, the authors propose a new variant of membership query data structure, called quotient-based Cuckoo filter (QCF), that reflects a merging process between QF and CF to minimise the calculation overhead and employing it in a DPI system. QCF employs only two hash functions and consumes less calculation overhead than BF, QF, and CF. The implementation results show that using QCF as an identification tool in a DPI system result in a time improvement up to 77% over CF and up to 98% over BF and QF.

Parallel multiple pattern matching schemes based on cuckoo filter for deep packet inspection on graphics processing units

A large amount of data now being transferred through networks has made deep packet inspection (DPI) an essential part of security activities. Several DPI systems are developed based on Bloom filters to defend against malicious worm attacks through the Internet. These approaches have achieved significant performance. However, they do not permit deletion of items from the set of target patterns. This study proposes two multiple pattern matching schemes for DPI to exploit high parallelism capacity of graphics processing units (GPUs). Firstly, a GPU-based Cuckoo filter scheme is proposed by adopting a new approximate set membership, called Cuckoo filter, for parallel multiple pattern matching. The Cuckoo filter has many advantages over the Bloom filter such as higher insert performance, higher lookup throughput, less memory consumption, less false positive rate, and delete operation support. Secondly, an implementation of the GPU-based Cuckoo filter, called GPUshared-based Cuckoo filter is proposed. This scheme can efficiently distribute input string and pre-processing data in the hierarchical memory of GPUs to optimise the performance of the GPU-based Cuckoo filter scheme. Experiments show that the proposed schemes offer better performance than the previous approaches based on the Bloom filter.

A scalable architecture for reducing power consumption in pipelined deep packet inspection system

A scalable architecture for reducing power consumption in pipelined AC-DFA (Aho-Corasick deterministic finite automaton) tries for deep packet inspection (DPI) system is proposed. A new scheme for deciding the strides of the AC-DFA trie is devised where the stride of each pipeline is decided variably to reduce the power consumption. Scaling down the clock frequency of the rarely-used stages is applied to reduce wasted power consumption. As a result, a DPI system with the proposed schemes shows a reduction of up to 27% in power consumption, compared with the state-of-the-art DPI systems.

BlindBox

Many network middleboxes perform deep packet inspection (DPI), a set of useful tasks which examine packet payloads. These tasks include intrusion detection (IDS), exfiltration detection, and parental filtering. However, a long-standing issue is that once packets are sent over HTTPS, middleboxes can no longer accomplish their tasks because the payloads are encrypted. Hence, one is faced with the choice of only one of two desirable properties: the functionality of middleboxes and the privacy of encryption. We propose BlindBox, the first system that simultaneously provides {\em both} of these properties. The approach of BlindBox is to perform the deep-packet inspection {\em directly on the encrypted traffic. BlindBox realizes this approach through a new protocol and new encryption schemes. We demonstrate that BlindBox enables applications such as IDS, exfiltration detection and parental filtering, and supports real rulesets from both open-source and industrial DPI systems. We implemented BlindBox and showed that it is practical for settings with long-lived HTTPS connections. Moreover, its core encryption scheme is 3-6 orders of magnitude faster than existing relevant cryptographic schemes.

Speculative parallel pattern matching using stride-k DFA for deep packet inspection

Modern deep packet inspection (DPI) systems match network traffic against a large set of patterns which are defined using regular expressions. Deterministic finite automata (DFA) is generally preferred to parse these regular expressions. However, packets are mostly scanned one byte at a time which becomes a bottleneck for the DPI systems as they are unable to cope up with higher line rates. In this paper, we present an approach which allows a packet to be split up into two chunks. Furthermore, we scan the bytes of each chunk in parallel using speculation and multi-stride (stride-k) DFA. Stride-k DFAs results in fast processing of bytes but leads to high memory usage. Therefore, we propose a transition compression algorithm using alphabet compression table to limit the memory usage of multi-stride DFA. Experimental results show that the speculative parallel pattern matching using stride-k DFA leads to improvement in terms of speedup and latency over traditional DFA matching.

Design and optimizations for efficient regular expression matching in DPI systems

Efficient techniques for pattern matching are essential in a number of networked systems and services, such as intrusion detection systems, application identification and classification services, and traffic management. Most pattern matching applications describe patterns using regular expression and the support engine is Deterministic Finite Automaton (FA). Previous research studies address either performance or space requirements issues. From the original DFA formalism we design and evaluate optimizations to its representation and operation to meet Deep Packet Inspection (DPI) systems’ requirements for commodity platforms, such as (i) decreasing the original DFA memory consumption (high compression ratio) and (ii) performing pattern matching as fast as the original DFA. Our approach spans from designing the DFA to developing memory layouts to get the most of the underlying architecture. The contributions of this work are threefold: (i) a new and improved finite automaton model, called Ranged Compressed DFA (RCDFA), (ii) three RCDFA optimizations for achieving more compression and matching speed, and (iii) three advanced layouts for implementing the compressed automaton without memory and performance penalties. The experimental evaluation shows that RCDFA compresses DFA up to 99% without imposing additional memory lookups. The proposed advanced layouts reach memory compression of around 97%. RCDFA together with the advanced layouts outperforms the standard DFA by up to 32 times in terms of processing speed.

Deep Packet Inspection Based on Many-Core Platform

With the development of computer technology, network bandwidth and network traffic continue to increase. Considering the large data flow, it is imperative to perform inspection effectively on network packets. In order to find a solution of deep packet inspection which can appropriate to the current network environment, this paper built a deep packet inspection system based on many-core platform, and in this way, verified the feasibility to implement a deep packet inspection system under many-core platform with both high performance and low consumption. After testing and analysis of the system performance, it has been found that the deep packet inspection based on many-core platform TILE_Gx36 [1] [2] can process network traffic of which the bandwidth reaches up to 4 Gbps. To a certain extent, the performance has improved compared to most deep packet inspection system based on X86 platform at present.

파이프라인형 DPI 시스템에서 효율적인 소비전력 감소를 위한 동작주파수 설계방법

An efficient method for reducing power consumption in pipelined deep packet inspection systems is proposed. It is based on the observation that the number of memory accesses is dominant for the power consumption and the number of accesses drops drastically as the input goes through stages of the pipelined AC-DFA. A DPI system is implemented where the operating frequency of the stages that are not frequently used in the pipeline is reduced to eliminate the waste of power consumption. The power consumption of the proposed DPI system is measured upon various input character set and up to 25% of ∙저 자 : 김한수 ∙투고일 : 2014. 8. 27, 심사일 : 2014. 9. 17, 게재확정일 : 2014. 11. 12. * 국립과학수사연구원 디지털분석과(Digital Technology and Biometry Division, National Forensic Service) 134 Journal of The Korea Society of Computer and Information December 2014 reduction of total power consumption is obtained, compared to those of the recent DPI systems. The method can be easily applied to other pipelined architecture and string searching applications. ▸