Traffic Inspection Research Articles

HybridSA: GPU Acceleration of Multi-pattern Regex Matching using Bit Parallelism

Multi-pattern matching is widely used in modern software for applications requiring high throughput such as protein search, network traffic inspection, virus or spam detection. Graphics Processor Units (GPUs) excel at executing massively parallel workloads. Regular expression (regex) matching is typically performed by simulating the execution of deterministic finite automata (DFAs) or nondeterministic finite automata (NFAs). The natural implementations of these automata simulation algorithms on GPUs are highly inefficient because they give rise to irregular memory access patterns. This paper presents HybridSA, a heterogeneous CPU-GPU parallel engine for multi-pattern matching. HybridSA uses bit parallelism to efficiently simulate NFAs on GPUs, thus reducing the number of memory accesses and increasing the throughput. Our bit-parallel algorithms extend the classical shift-and algorithm for string matching to a large class of regular expressions and reduce automata simulation to a small number of bitwise operations. We have developed a compiler to translate regular expressions into bit masks, perform optimizations, and choose the best algorithms to run on the GPU. The majority of the regular expressions are accelerated on the GPU, while the patterns that exhibit random memory accesses are executed on the CPU in parallel. We evaluate HybridSA against state-of-the-art CPU and GPU engines, as well as a hybrid combination of the two. HybridSA achieves between 4 and 60 times higher throughput than the state-of-the-art CPU engine and between 4 and 233 times better than the state-of-the-art GPU engine across a collection of real-world benchmarks.

JAPPI: An unsupervised endpoint application identification methodology for improved Zero Trust models, risk score calculations and threat detection

The surge in global digitalization triggered by COVID-19 has led to a significant increase in internet traffic and has precipitated a rapid transformation of the network security landscape. Despite being increasingly difficult, accurate traffic inspection is vital for ensuring productivity while reliably protecting internal assets. Endpoint application identification enables high accuracy inspection and detection by providing network security solutions with specific context on individual connections. However, achieving it in real-time with standard fingerprinting methods based only on client-side traffic has proven to be a challenging problem with no comprehensive solution thus far. In this article, we present a new methodology for identifying endpoint applications from network traffic, utilizing machine learning. Our methodology leverages similarities in the pre-hash string of the JA3 algorithm for fingerprinting application specific TLS Client Hello messages. By utilizing well-known clustering algorithms it is possible to identify the underlying TLS libraries and the application from the traffic remarkably better than with simple string-based matching. Our model can categorize 99,5% of the traffic in a controlled network, and 93,8% in an uncontrolled network, compared to 0,1% and 0,2% using simple string matching. Our methodology is especially effective for enhancing Zero Trust models, calculating a risk score for network events, and improving threat detection accuracy in network security solutions.

Encrypted Network Traffic Analysis and Classification Utilizing Machine Learning.

Encryption is a fundamental security measure to safeguard data during transmission to ensure confidentiality while at the same time posing a great challenge for traditional packet and traffic inspection. In response to the proliferation of diverse network traffic patterns from Internet-of-Things devices, websites, and mobile applications, understanding and classifying encrypted traffic are crucial for network administrators, cybersecurity professionals, and policy enforcement entities. This paper presents a comprehensive survey of recent advancements in machine-learning-driven encrypted traffic analysis and classification. The primary goals of our survey are two-fold: First, we present the overall procedure and provide a detailed explanation of utilizing machine learning in analyzing and classifying encrypted network traffic. Second, we review state-of-the-art techniques and methodologies in traffic analysis. Our aim is to provide insights into current practices and future directions in encrypted traffic analysis and classification, especially machine-learning-based analysis.

Determination of Ethanol and Aromatics in Blood by Headspace Portable Gas Chromatography-Mass Spectrometry (HS-PGC-MS)

Ethanol abuse is increasingly becoming a global issue, adversely affecting both society and families. In blood alcohol concentration (BAC) detection, traditional techniques have long experimental cycles and harsh experimental conditions and often fail to produce timely results. Consequently, achieving swift and precise on-site detection of ethanol poisoning in humans poses a formidable challenge in analytical science. We designed a novel assay for swift on-site analysis of ethanol and BTEX (benzene, toluene, ethylbenzene, and xylene) poisoning in humans using a custom portable headspace injector (portable HS) and portable gas chromatography-mass spectrometry (portable GC-MS). This setup, known for its lightweight design and speed, allows on-site sampling and delivers rapid results. We established a calibration curve (R2 > 0.99) for ethanol in blood using portable gas chromatography-mass spectrometry (portable GC-MS). This on-site blood alcohol concentration (BAC) assay proves simpler than traditional methods, enabling rapid analysis of each sample within a short timeframe and requiring only a small blood volume (0.1 ml). In addition to our investigation on ethanol, we developed a portable method for detecting BTEX in blood. This method provides a favorable coefficient of determination (R2 > 0.99) for various substances, thereby broadening the utility of portable mass spectrometry in areas such as traffic inspection and forensic identification.

Network Traffic Monitoring and Real-time Risk Warning based on Static Baseline Algorithm

With the rapid growth of network traffic, in order to monitor network traffic, the author proposes a baseline based traffic inspection method. The main objective is to develop a global system for identifying malicious traffic, rather than a precise method for detecting the types of worms produced by malicious traffic. Although traffic is caused by the causes, network administrators can use this international search technique to detect malicious traffic data. The system based approach mainly includes designing time based on the traditional traffic model, detecting various equipments and network traffic process, and configuring the traffic flow according to each time frame. This method uses Cisco's NetFlow Collector, a NetFlow Collector (NFC), to collect raw NetFlow data transmitted by the device through UDP every 5 minutes. the Then, three-dimensional data such as communication port, communication time, and traffic flow (bytes or packets) is used to filter, remove the different values, calculate the base values, and compare the real-time results with the base values to check the traffic defects in the current network. If there are differences between the monitoring data and the system configuration at the same time, the system will issue an abnormal warning, and as time accumulates, the alarm level will gradually escalate.

Privacy-Preserving Traceable Encrypted Traffic Inspection in Blockchain-Based Industrial IoT

Privacy-Preserving Traceable Encrypted Traffic Inspection in Blockchain-Based Industrial IoT

Detection of Microgrid Cyberattacks Using Network and System Management

Microgrids, as a major domain of smart grids, facilitate the integration of distributed and renewable energy sources. They are capable of operating autonomously when the connection to the main grid is difficult or cost-ineffective. As the presence of microgrids in the smart grid becomes more common, it becomes more urgent to protect the microgrid from cyberattacks. In this paper, we present the first design and implementation of a microgrid security monitoring platform based on IEC 62351-7:2017 Network and System Management (NSM). The capabilities of this platform are demonstrated on a detailed microgrid model that is deployed on a real-time co-simulation testbed. A hybrid rule-based and machine learning anomaly detection approach is developed to detect attacks targeting the microgrid. To investigate the effectiveness of the NSM security monitoring platform, evaluate the proposed detection schemes, and observe the impact of cyberattacks on the microgrid operation, several formulated attacks against the microgrid are simulated. The effectiveness of different parameters and architectures for the anomaly detection schemes are compared against one another, including Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The obtained results demonstrate the strengths and gaps of security monitoring based on NSM, and can be seen as first step towards the design of more comprehensive monitoring platforms that integrate NSM along with complementary mechanisms such as passive network traffic inspection and device log analysis.

DIVERGENCE: Deep Reinforcement Learning-Based Adaptive Traffic Inspection and Moving Target Defense Countermeasure Framework

Reinforcement learning (RL) is a promising approach for intelligent agents to protect a given system under highly hostile environments. RL allows the agent to adaptively make sequential defense decisions based on the perceived current state of system security aiming to achieve the maximum defense performance in terms of fast, efficient, and automated detection, threat analysis, and response to the threat. In this paper, we propose a deep reinforcement learning (DRL)-based adaptive traffic inspection and moving target defense countermeasure framework, called ‘DIVERGENCE,’ for building a secure networked system. The DIVERGENCE provides two main security services: (1) a DRL-based network traffic inspection mechanism to achieve scalable and intensive network traffic visibility for rapid threat detection; and (2) an address shuffling-based moving target defense (MTD) technique to defend against threats as a proactive intrusion prevention mechanism. Through extensive simulations and experiments, we demonstrate that the DIVERGENCE successfully caught malicious traffic flows while significantly reducing the vulnerability of the network through MTD.

Privacy-Preserving Encrypted Traffic Inspection With Symmetric Cryptographic Techniques in IoT

To ensure the security of Internet of Things (IoT) communications, one can use deep packet inspection (DPI) on network middleboxes to detect and mitigate anomalies and suspicious activities in network traffic of IoT, although doing so over encrypted traffic is challenging. Therefore, in this article, an efficient and privacy-preserving encrypted traffic detection scheme is proposed. The scheme uses only lightweight cryptographic operations (i.e., symmetric encryption, hash functions, and pseudorandom functions) to achieve both privacy and security within an inspection round. A dispute resolution mechanism is also designed to address potential disputes between client(s) and server(s). We also present the corresponding security proof and experimental evaluation, which demonstrate that our proposed scheme achieves strong security and privacy preservation and good performance.

Monitoring OpenFlow Virtual Networks via Coordinated Switch-Based Traffic Mirroring

As network virtualization becomes ubiquitous, legacy hardware-based traffic monitoring systems are no longer viable for dynamic traffic inspection at arbitrary locations in virtual networks. In this paper, we present the design and evaluation of Open Virtual Tap (OVT), a software-defined solution to replace hardware taps for traffic monitoring in OpenFlow virtual networks by utilizing mirroring capabilities of OpenFlow switches. The key idea behind OVT is the joint configuration of all switches in the substrate physical network in order to efficiently mirror flows from all virtual networks. We show that such a design avoids inefficiencies that result from existing software-based traffic mirroring solutions in which each virtual network configures its own switches independently of other virtual networks. We evaluate OVT using model-driven simulations as well as Mininet experiments with realistic applications for intrusion detection and video telephony analysis. Specifically, in our experiments, we observe that OVT can achieve up to 20% improvement in flow coverage compared to existing traffic mirroring approaches.

Passenger Surveillance Using Deep Learning in Post-COVID-19 Intelligent Transportation System.

Intelligent Transport System should be renovated in many aspects in post-pandemic situation like COVID-19. The passenger-count inside a car will be restricted based on the vehicle capacity and the COVID-19 hot-spot zone. Traffic rules will be impacted to align with a similar contagious outbreak. The on-road ‘Yellow-Vulture’ cameras need to incorporate such surveillance rules to monitor related anomalies for preventing contamination. To maintain safe-distance, an automatic surveillance system will be preferred by the Government very soon. Moreover, facial mask usage during the journey has become an essential habit to stop the spread of the infection. In this article, we have proposed a deep-Learning based framework that employs an augmented image data set to provide proper surveillance in the transport system to maintain the health protocols. Fast and accurate detection of the number of passengers inside a car and their face masks from the traffic inspection camera feed has been demonstrated. We have exploited the advantages of the popular Transfer Learning approach with novel variations of images while performing the training. To the best of our knowledge, this is the first attempt to watch over in-vehicle social-distancing in post-pandemic circumstances through deep-Learning based image analysis. The superiority of the proposed framework has been established over several state-of-the-art techniques using different numerical metrics and visual comparisons along with a support of statistical hypothesis test. Our technique has achieved 98.5% testing accuracy in various adverse conditions. Zero-shot evaluation has been explored for the Real-Time-Medical-Mask-Detection data set Wang et al. (Real-Time-Medical-Mask-Detection, 2020a https://github.com/TheSSJ2612/Real-Time-Medical-Mask-Detection/, Accessed 14 Nov 2020), where we have attained 96.4% accuracy that manifests the generalization of the network.

Android application for identification of vehicle plates for traffic inspection

Android application for identification of vehicle plates for traffic inspection

Acceleration of Intrusion Detection in Encrypted Network Traffic Using Heterogeneous Hardware.

More than 75% of Internet traffic is now encrypted, and this percentage is constantly increasing. The majority of communications are secured using common encryption protocols such as SSL/TLS and IPsec to ensure security and protect the privacy of Internet users. However, encryption can be exploited to hide malicious activities, camouflaged into normal network traffic. Traditionally, network traffic inspection is based on techniques like deep packet inspection (DPI). Common applications for DPI include but are not limited to firewalls, intrusion detection and prevention systems, L7 filtering, and packet forwarding. With the widespread adoption of network encryption though, DPI tools that rely on packet payload content are becoming less effective, demanding the development of more sophisticated techniques in order to adapt to current network encryption trends. In this work, we present HeaderHunter, a fast signature-based intrusion detection system even for encrypted network traffic. We generate signatures using only network packet metadata extracted from packet headers. In addition, we examine the processing acceleration of the intrusion detection engine using different heterogeneous hardware architectures.

Deep Reinforcement Learning-Based Traffic Sampling for Multiple Traffic Analyzers on Software-Defined Networks

Intrusion detection system (IDS) and deep packet inspection (DPI) are widely used to detect network attacks and anomalies, thereby enhancing cyber-security. Conventional traffic analyzers such as IDS have fixed locations and a limited capacity to perform DPI on large volumes of network traffic. Nowadays, software-defined networking (SDN) technology, which provides flexibility, elasticity, and programmability by decoupling the network control and data planes, makes it possible to capture entire or a certain portion of data traffic flows on SDN-capable switches and steer the captured network traffic to one of the traffic analyzers on the network. Therefore, how to sample network traffic and where to steer the sampled traffic among multiple traffic analyzers are critical problems facing cyber-security. Since there is a possibility that potentially useful information will be lost in not-captured traffic, deciding the sampling points and sampling rates of network traffic remains important. Additionally, after determining the sampling points and rates, sampled traffic must be sent to one of the multiple traffic analyzers for traffic inspection, which may incur additional network delivery overheads. We propose a less-intrusive traffic sampling mechanism for multiple traffic analyzers on an SDN-capable network using a deep deterministic policy gradient (DDPG), which is a representative deep reinforcement learning (DRL) algorithm for continuous action control. The proposed system learns sampling resource allocation policy under the uncertainty of flow distribution according to sampled traffic inspection results obtained from multiple traffic analyzers. Through extensive simulations and the SDN-based testbed experiments, we demonstrate that the proposed approach has a high probability of capturing malicious flows while maintaining a balanced load of multiple traffic analyzers and reducing flow monitoring overheads.

Оценка матрицы корреспонденций методом Пуассона на основе данных онлайн-сервиса поиска автомобильных попутчиков

For effective functioning of transport systems, information on spatial distribution of transport flows is necessary both at the current moment and for future periods. Traditional methods of obtaining observation results require very high costs, In addition, there is a high probability of obtaining low-quality data due to the strong influence of the human factor. The disadvantage of automated methods of traffic flow inspection is the high cost of equipment. The paper proposes a method for obtaining data on spatial distribution of passenger flows using constant monitoring of the online service of joint trips. To illustrate the possibilities of this approach, the online service for searching for car companions was used on the example of Sverdlovsk region. The correspondence matrix is constructed using a gravitational model applying the Poisson method. The Gauss - Seidlitz iterative method is used to find unknown parameters of the model. The assessment of quality and adequacy of the constructed model values of correspondence was carried out. It is shown that the spatial distribution of traffic flows obtained on the basis of data from online services for searching for car companions is well described by a gravitational model with a piecewise constant gravity function. Based on the analysis of the observed correspondence matrix, it was found that traffic flows between Yekaterinburg and all other cities of the Sverdlovsk region significantly exceed traffic flows between these cities.

Real-Time Panoramic Multi-Target Detection Based on Mobile Machine Vision and Deep Learning

Research on computer vision technology based on deep learning has become an important research direction in the field of artificial intelligence. Among them, the target detection task of images and videos has occupied a pivotal position in many intelligent vision research and applications. The purpose of this article is to learn from the existing object recognition and target detection technology foundation, to conduct in-depth research on the current traffic patrol deficiencies, to achieve real-time patrol and monitoring of key roads. This article chooses to use the target detection algorithm of Faster R-CNN in deep learning, applies drones and mobile machine vision to traffic patrol inspection, and detects various targets in the PASCAL VOC data set. The AP of the bus is 68.20%; The AP of the car is 75.50%; The moving object recognition and tracking method based on Faster R-CNN and Hungarian matching method is verified, and traffic flow data can be calculated to realize real-time monitoring of road traffic. The research in this paper helps to overcome the defects of human eye recognition in the traditional monitoring system and improve the efficiency of traffic inspection.

Power Prefixes Prioritization for Smarter BGP Reconvergence

BGP reconvergence events involving a large number of prefixes may result in the loss of large amounts of traffic. Based on the observation that a very small number of prefixes carries the vast majority of traffic, we propose Power Prefixes Prioritization (PPP) to ensure the routes of these popular BGP prefixes converge first. By doing so, we significantly reduce the amount of traffic lost during reconvergence events. To achieve this, PPP obtains an ordered list of popular prefixes through traffic inspection, and configures the resulting prefix rank in the BGP routers to prioritize the processing and advertisement of BGP routes. We model the benefits of PPP over traditional BGP processing in terms of traffic loss for both generic and a Zipf traffic distribution, and we consider the impact of sampling in the process of obtaining the prefix rank. Applying the mechanism to real traffic traces obtained from WIDE, we show that PPP reduces the amount of traffic lost by an order of magnitude, even when we configure it to use conservative sampling rates. We prototype our proposal in Quagga to show the feasibility of its implementation, and we observe similar traffic loss reduction. PPP can be deployed incrementally, as it is implemented purely as a change in the router-internal BGP processing behavior.

The Sorry State of TLS Security in Enterprise Interception Appliances

Network traffic inspection, including TLS traffic, in enterprise environments is widely practiced. Reasons for doing so are primarily related to improving enterprise security (e.g., phishing and malicious traffic detection) and meeting legal requirements (e.g., preventing unauthorized data leakage and copyright violations). To analyze TLS-encrypted data, network appliances implement a Man-in-the-Middle (MITM) TLS proxy by acting as the intended web server to a requesting client (e.g., a browser) and acting as the client to the actual/outside web server. As such, the TLS proxy must implement both a TLS client and a server and handle a large amount of traffic, preferably in real-time. However, as protocol and implementation layer vulnerabilities in TLS/HTTPS are quite frequent, these proxies must be at least as secure as a modern, up-to-date web browser and a properly configured web server (e.g., an A+ rating in SSLlabs.com). As opposed to client-end TLS proxies (e.g., as in several anti-virus products), the proxies in network appliances may serve hundreds to thousands of clients, and any vulnerability in their TLS implementations can significantly downgrade enterprise security. To analyze TLS security of network appliances, we develop a comprehensive framework, combining and extending tests from existing work on client-end and network-based interception studies. We analyze 13 representative network appliances over a period of more than a year (including versions before and after notifying affected vendors, a total of 17 versions) and uncover several security issues. For instance, we found that four appliances perform no certificate validation at all, three use pre-generated certificates, and eleven accept certificates signed using MD5, exposing their clients to MITM attacks. Our goal is to highlight the risks introduced by widely used TLS proxies in enterprise and government environments, potentially affecting many systems hosting security, privacy, and financially sensitive data.

Multiple pattern matching for network security applications: Acceleration through vectorization

As both new network attacks emerge and network traffic increases in volume, the need to perform network traffic inspection at high rates is ever increasing. The core of many security applications that inspect network traffic (such as Network Intrusion Detection) is pattern matching. At the same time, pattern matching is a major performance bottleneck for those applications: indeed, it is shown to contribute to more than 70% of the total running time of Intrusion Detection Systems. Although numerous efficient approaches to this problem have been proposed on custom hardware, it is challenging for pattern matching algorithms to gain benefit from the advances in commodity hardware. This becomes even more relevant with the adoption of Network Function Virtualization, that moves network services, such as Network Intrusion Detection, to the cloud, where scaling on commodity hardware is key for performance.In this paper, we tackle the problem of pattern matching and show how to leverage the architecture features found in commodity platforms. We present efficient algorithmic designs that achieve good cache locality and make use of modern vectorization techniques to utilize data parallelism within each core. We first identify properties of pattern matching that make it fit for vectorization and show how to use them in the algorithmic design. Second, we build on an earlier, cache-aware algorithmic design and show how we apply cache-locality combined with SIMD gather instructions to pattern matching. Third, we complement our algorithms with an analytical model that predicts their performance and that can be used to easily evaluate alternative designs. We evaluate our algorithmic design with open data sets of real-world network traffic: Our results on two different platforms, Haswell and Xeon-Phi, show a speedup of 1.8x and 3.6x, respectively, over Direct Filter Classification (DFC), a recently proposed algorithm by Choi et al. for pattern matching exploiting cache locality, and a speedup of more than 2.3x over Aho–Corasick, a widely used algorithm in today’s Intrusion Detection Systems. Finally, we utilize highly parallel hardware platforms, evaluate the scalability of our algorithms and compare it to parallel implementations of DFC and Aho–Corasick, achieving processing throughput of up to 45Gbps and close to 2 times higher throughput than Aho–Corasick.

The New Threats of Information Hiding: The Road Ahead

Compared to cryptography, steganography is a less discussed domain. However, there is a recent trend of exploiting various information hiding techniques to empower malware, for instance to bypass security frameworks of mobile devices or to exfiltrate sensitive data. This is mostly due to the need to counteract increasingly sophisticated security mechanisms, such as code analysis, runtime countermeasures, or real-time traffic inspection tools. In this perspective, this paper presents malware exploiting information hiding in a broad sense, i.e., it does not focus on classical covert channels, but also discusses other camouflage techniques. Differently from other works, this paper solely focuses on real-world threats observed in the 2011 - 2017 timeframe. The observation indicates a growing number of malware equipped with some form of data hiding capabilities and a lack of effective and universal countermeasures.