Real-time Traffic Classification Research Articles

SCNTA

Real-time network inspection applications face a threat of vulnerability as high-speed networks continue to expand. For companies and ISPs, real-time traffic classification is an issue. The classifier monitor is made up of three modules: Capturing_of_Packets (CoP) and pre-processing, Reconciliation_of_Flow (RoF), and categorization of Machine Learning (ML). Based on parallel processing along with well-defined interfacing of data, the modules are framed, allowing each module to be modified and upgraded separately. The Reconciliation_of_Flow (RoF) mechanism becomes the output bottleneck in this pipeline. In this implementation, an optimal reconciliation process was used, resulting in an average delivery time of 0.62 seconds. In order to verify our method, we equated the results of the AdaBoost Ensemble Learning Algorithm (ABELA), Naive Bayes (NB), Decision Tree (DT), K-Nearest Neighbor (KNN), and Flexible Naive Bayes (FNB) in the classification module. The architectural design of the run time CSNTA categorization (flow-based) scheme is presented in this paper.

Is Encrypted ClientHello a Challenge for Traffic Classification?

Although the widely-used Transport Layer Security (TLS) protocol hides application data, an unencrypted part of the TLS handshake, specifically the server name indication (SNI), is a backdoor for encrypted traffic classification frameworks. The recently developed Encrypted ClientHello (ECH) amendment to the TLS protocol aims to protect the privacy-sensitive content of the ClientHello message, including SNI. Conversely, ECH can be a game-changer in the early detection of encrypted traffic. The paper shows that the performance of the state-of-the-art traffic classification algorithms degrades significantly with the introduction of the ECH. Hence, novel approaches to real-time traffic classification are required. The paper develops two novel traffic classification algorithms to address this challenge. The first one uses unencrypted bytes of the TLS Hello messages as independent features of the Random Forest algorithm. It is extremely lightweight and suits throughput-focused traffic classification. It is faster than state-of-the-art algorithms by three times and achieves higher classification quality. The second algorithm augments the approach of the first one by focusing on the particular metadata of the handshake. This way, it efficiently extracts data from the exchange and achieves the highest classification quality in all the considered scenarios. It has a three times lower error rate than state-of-the-art algorithms and provides a reliable classification of ECH traffic.

A New Method for Flow-Based Network Intrusion Detection Using the Inverse Potts Model

Network Intrusion Detection Systems (NIDS) play an important role as tools for identifying potential network threats. In the context of ever-increasing traffic volume on computer networks, flow-based NIDS arise as good solutions for real-time traffic classification. In recent years, different flow-based classifiers have been proposed using Machine Learning (ML) algorithms. Nevertheless, classical ML-based classifiers have some limitations. For instance, they require large amounts of labeled data for training, which might be difficult to obtain. Additionally, most ML-based classifiers are not capable of domain adaptation, i.e. after being trained on an specific data distribution, they are not general enough to be applied to other related data distributions. And, finally, many of the models inferred by these algorithms are black boxes, which do not provide explainable results. To overcome these limitations, we propose a new algorithm, called Energy-based Flow Classifier (EFC). This anomaly-based classifier uses inverse statistics to infer a statistical model based on labeled benign examples. We show that EFC is capable of accurately performing binary flow classification and is more adaptable to different data distributions than classical ML-based classifiers. Given the positive results obtained on three different datasets (CIDDS-001, CICIDS17 and CICDDoS19), we consider EFC to be a promising algorithm to perform robust flow-based traffic classification.

Analysis of promising approaches and research on traffic flow classification for maintaining QoS by ML methods in SDN networks

One of the most important tasks that exist in modern networks is to maintain the Quality-of-Service QoS at the appropriate level which can be achieved by applying various traffic management mechanisms. In order to maintain the QoS parameters in the proper state, you need to know the types of traffic passing through the network. Given high-tech and high-performance networks such as SDN networks, traffic classification by conventional methods becomes almost impossible. Data mining methods, including Machine Learning methods, come to the rescue. The article analyzes the main promising approaches to real-time traffic classification for maintaining QoS in SDN networks by ML methods as well as provides a comparative overview of the most outstanding works in this field.

Mobile network traffic pattern classification with incomplete a priori information

In complex networks systems like mobile edge infrastructures, real-time traffic classification according to application types is an enabling technique for network resource optimization and advanced security management. State-of-the-art schemes take advantage of machine learning techniques to train classification models based on behavioral characteristics of network traffic flows. Nonetheless, most existing studies assume complete a priori information of the application classes and formulate the task as a standalone multi-class classification problem. Such classification models cannot properly handle the unknown applications that are absent from the training set during the time of training. In this work, we propose a practical mobile network traffic classification scheme that builds robust classifiers based on incomplete a priori information. Specifically, the core idea is to extract the unknown patterns emerging in the network periodically to complement the initial labeled data set that only consists of a limited number of known applications. We propose two algorithms for the unknown pattern extraction step. One is based on iterative asymmetric binary classification and the other is based on constrained clustering. Empirical results based on a public data set show that the proposed scheme can effectively detect both known and unknown applications.

Review of Machine Learning Techniques for Network Traffic Classification

Network traffic classification is an essential tool in cyber security for the recognition and interception of cyber insider threats. Traffic classification is the first step to distinguish various applications and protocols that are available in the network. The core component of network intrusion detection is a network traffic analysis that investigates the network behavior based on traffic characterization. Network traffic classification is the centre segment of the network traffic analysis and particularly for filtering traffic in order to identify any malicious activities within the network. Numerous techniques are proposed by researchers to date varying from traditional Port-based to Machine/Deep Learning techniques. Machine Learning emerges as a prominent solution for encrypted and real-time traffic classification; exploiting the statistical properties of the network flow. In this paper various Machine/Deep Learning techniques for network traffic classification are critically evaluated. The main purpose is to investigate different traffic classification approaches (supervised, semi-supervised, unsupervised) and provide summarization for a set of trends followed by various researchers for classifying network traffic.

FPGA-Based Network Traffic Classification Using Machine Learning

Real-time classification of internet traffic is critical for the efficient management of networks. Classification approaches based on machine learning techniques have shown promising results with high levels of accuracy. In this article, the suitability of packet-level and flow-level features is validated using stepwise regression and random forest feature selection. Moreover, the optimal percentage of packets considered within a flow while extracting flow-level features is determined. Several experiments are conducted using naive Bayes, support vector machine, $k$ -nearest neighbor, random forest, and artificial neural networks on the University of Brescia (UNIBS) and the University of New Brunswick (UNB) datasets, which are both publicly available. The performed experiments show that 60% of flow packets are a good compromise that ensures high performance in the least processing time. The results of the conducted experiments indicate that random forest outperforms other algorithms achieving a maximum accuracy of 98.5% and an F-score of 0.932. Further, and since software-based classifiers cannot meet the anticipated real-time requirements, we propose a Field-Programmable Gate Array (FPGA) based random forest implementation that utilizes a highly pipelined architecture to accelerate such a time-consuming task. The proposed design achieves an average throughput of 163.24 Gbps, exceeding throughputs of reported hardware-based classifiers that use comparable approaches, which in turn ensures the continuity of real-time traffic classification at congested data centers.

Computing aware scheduling in mobile edge computing system

Mobile edge computing (MEC) is an emerging technology recognized as an effective solution to guarantee the Quality of Service for computation-intensive and latency-critical traffics. In MEC system, the mobile computing, network control and storage functions are deployed by the servers at the network edges (e.g., base station and access points). One of the key issue in designing the MEC system is how to allocate finite computational resources to multi-users. In contrast with previous works, in this paper we solve this issue by combining the real-time traffic classification and CPU scheduling. Specifically, a support vector machine based multi-class classifier is adopted, the parameter tunning and cross-validation are designed in the first place. Since the traffic of same class has similar delay budget and characteristics (e.g. inter-arrival time, packet length), the CPU scheduler could efficiently scheduling the traffic based on the classification results. In the second place, with the consideration of both traffic delay budget and signal baseband processing cost, a preemptive earliest deadline first (EDF) algorithm is deployed for the CPU scheduling. Furthermore, an admission control algorithm that could get rid off the domino effect of the EDF is also given. The simulation results show that, by our proposed scheduling algorithm, the classification accuracy for specific traffic class could be over 82 percent, meanwhile the throughput is much higher than the existing scheduling algorithms.

On Internet Traffic Classification: A Two-Phased Machine Learning Approach

Traffic classification utilizing flow measurement enables operators to perform essential network management. Flow accounting methods such as NetFlow are, however, considered inadequate for classification requiring additional packet-level information, host behaviour analysis, and specialized hardware limiting their practical adoption. This paper aims to overcome these challenges by proposing two-phased machine learning classification mechanism with NetFlow as input. The individual flow classes are derived per application throughk-means and are further used to train a C5.0 decision tree classifier. As part of validation, the initial unsupervised phase used flow records of fifteen popular Internet applications that were collected and independently subjected tok-means clustering to determine unique flow classes generated per application. The derived flow classes were afterwards used to train and test a supervised C5.0 based decision tree. The resulting classifier reported an average accuracy of 92.37% on approximately 3.4 million test cases increasing to 96.67% with adaptive boosting. The classifier specificity factor which accounted for differentiating content specific from supplementary flows ranged between 98.37% and 99.57%. Furthermore, the computational performance and accuracy of the proposed methodology in comparison with similar machine learning techniques lead us to recommend its extension to other applications in achieving highly granular real-time traffic classification.

ITCM: A Real Time Internet Traffic Classifier Monitor

The continual growth of high speed networks is a challenge for real-time network analysis systems. The real time traffic classification is an issue for corporations and ISPs (Internet Service Providers). This work presents the design and implementation of a real time flow-based network traffic classification system. The classifier monitor acts as a pipeline consisting of three modules: packet capture and pre-processing, flow reassembly, and classification with Machine Learning (ML). The modules are built as concurrent processes with well defined data interfaces between them so that any module can be improved and updated independently. In this pipeline, the flow reassembly function becomes the bottleneck of the performance. In this implementation, was used a efficient method of reassembly which results in a average delivery delay of 0.49 seconds, approximately. For the classification module, the performances of the K-Nearest Neighbor (KNN), C4.5 Decision Tree, Naive Bayes (NB), Flexible Naive Bayes (FNB) and AdaBoost Ensemble Learning Algorithm are compared in order to validate our approach.

Hierarchical Real-time Network Traffic Classification Based on ECOC

Classification of network traffic is basic and essential for manynetwork researches and managements. With the rapid development ofpeer-to-peer (P2P) application using dynamic port disguisingtechniques and encryption to avoid detection, port-based and simplepayload-based network traffic classification methods were diminished.An alternative method based on statistics and machine learning hadattracted researchers' attention in recent years. However, most ofthe proposed algorithms were off-line and usually used a single classifier.In this paper a new hierarchical real-time model was proposed which comprised of a three tuple (source ip, destination ip and destination port)look up table(TT-LUT) part and layered milestone part. TT-LUT was used to quickly classify short flows whichneed not to pass the layered milestone part, and milestones in layered milestone partcould classify the other flows in real-time with the real-time feature selection and statistics.Every milestone was a ECOC(Error-Correcting Output Codes) based model which was usedto improve classification performance. Experiments showed that the proposedmodel can improve the efficiency of real-time to 80%, and themulti-class classification accuracy encouragingly to 91.4% on the datasets which had been captured from the backbone router in our campus through a week. DOI : http://dx.doi.org/10.11591/telkomnika.v12i2.3877

Application traffic classification at the early stage by characterizing application rounds

This article proposes a machine learning-based high-accuracy algorithm called “APPlication Round method (APPR)” to identify network application traffic at the early stage. For each TCP/UDP flow, discriminators available at the early stage are determined to support high-accuracy and real-time traffic classification. Such discriminators characterize the possible negotiation behaviors of each flow from an application layer perspective. The ability of flow attributes is tested using several machine learning algorithms. By contrast, this study also compares the level of accuracy of the proposed method with those reported by machine learning-based application traffic classification methods that have addressed real-time application traffic classification problems based on identical sample traffic sets. By applying a pruned C4.5 tree machine learning algorithm to real traffic traces, the proposed method offers a maximal 99.21%, with an average overall accuracy of 92.88% for all traffic samples. Compared to other machine learning algorithms, the proposed algorithm not only provides a minimal accuracy improvement of approximately 7–8% for normal ratio data sets and more than 15–30% improvement of overall accuracy for fixed ratio data samples, but is also suitable for on-line identification because of the low-flow test duration. Furthermore, the proposed method is also appropriate for identifying encrypted protocols because it demonstrates high accuracy, classifies encryption-based protocols, and supports real-time classification.

A Traffic Classification Agent for Virtual Networks Based on QoS Classes

Internet growth drives the emergence and usage of new applications, which have specific requirements for Quality of Service (QoS). Thus, differentiation of packets which belong to certain QoS classes becomes a crucial factor to ensure the desirable quality for the applications. Within this context, this paper proposes an agent for real-time traffic classification based on QoS classes for virtual network environments, which aims at forwarding the flow to the adequate virtual network according to the QoS class. The study was made using the Mininet emulator and the Openflow Protocol. And the study showed that the agent using the Naive Bayes method performs the more suitable classification in real time.

고정 IP-port 기반 응용 레벨 인터넷 트래픽 분석에 관한 연구

인터넷의 대중화로 인해 네트워크 트래픽은 나날이 증가되고 있다. 따라서 네트워크 자원의 효과적인 사용을 위한 응용 트래픽 분석의 중요성은 나날이 강조되고 있다. 본 논문에서는 고정 IP-port기반의 응용 트래픽 분석 방법론을 제안한다. 고정 IP-port는 오직 한 응용에서 고정적으로 사용하는 {IP address, port number, transport protocol}의 쌍으로써 각각의 응용을 분석해서 자동적으로 수집할 수 있다. 본 논문에서는 고정 IP-port를 사용하여 기존 연구에서 제안된 방법들 보다 매우 가볍고, 빠르며 정확한 실시간 트래픽 분석 시스템을 설계하였다. 또한, 기존의 연구에서 제안된 통일성 없는 검증 방법을 보완하여 객관적 검증 시스템을 설계하고 분석결과를 정확하게 검증하였다. 본 논문은 고정 IP-port를 추출하는 매우 효과적인 방법과 시스템 구조, 그리고 분석 결과의 객관적 검증 시스템을 제안한다. 그리고 실험과 검증 시스템을 통하여 고정 IP-port 기반 응용 레벨 인터넷 트래픽 분석 방법론의 타당성을 증명한다. As network traffic is dramatically increasing due to the popularization of Internet, the need for application traffic classification becomes important for the effective use of network resources. In this paper, we present an application traffic classification method based on fixed IP-port information. A fixed IP-port is a {IP address, port number, transport protocol}triple dedicated to only one application, which is automatically collected from the behavior analysis of individual applications. We can classify the Internet traffic more accurately and quickly by simple packet header matching to the collected fixed IP-port information. Therefore, we can construct a lightweight, fast, and accurate real-time traffic classification system than other classification method. In this paper we propose a novel algorithm to extract the fixed IP-port information and the system architecture. Also we prove the feasibility and applicability of our proposed method by an acceptable experimental result.

Supervised Learning Real-time Traffic Classifiers

Network traffic classification plays an important role in various network activities. Due to the ineffectiveness of traditional port-based and payload-based methods, recent works proposed using machine learning methods to classify flows based on statistical characteristics. In this study, we present a comprehensive evaluation of the effectiveness of these statistical methods for real-time traffic classification problem. We evaluate three different flow feature sets that are used to capture distinct properties of each application, two of them consist of features generated from full flows and the third is made up of early sub-flow statistics derived from the first few packets of each flow. We compare various supervised machine learning schemes to identify the one most suitable for traffic classification. We also apply feature selection to identify the most significant features. The results indicate that statistical characteristics of traffic flows are capable to distinguish applications; in particular it’s feasible to use the features derived from early sub-flows for realtime traffic classification. The results also indicate that classifiers based on decision tree outperform others, as they obtain highest accuracy and fastest classifying speed.

Hybrid internet traffic classification technique

Accurate and real-time classification of network traffic is significant to network operation and management such as QoS differentiation, traffic shaping and security surveillance. However, with many newly emerged P2P applications using dynamic port numbers, masquerading techniques, and payload encryption to avoid detection, traditional classification approaches turn to be ineffective. In this paper, we present a layered hybrid system to classify current Internet traffic, motivated by variety of network activities and their requirements of traffic classification. The proposed method could achieve fast and accurate traffic classification with low overheads and robustness to accommodate both known and unknown/encrypted applications. Furthermore, it is feasible to be used in the context of real-time traffic classification. Our experimental results show the distinct advantages of the proposed classification system, compared with the one-step Machine Learning (ML) approach.