Flow Statistical Features Research Articles

FSDC: Flow Samples and Dimensions Compression for Efficient Detection of DNS-over-HTTPS Tunnels

This paper proposes an innovative approach capitalized on the distinctive characteristics of command and control (C&C) beacons, namely, time intervals and frequency between consecutive unique connections, to compress the network flow dataset. While previous studies on the same matter used single technique, we propose a multi-technique approach for efficient detection of DoH tunnels. We use a baseline public dataset, CIRA-CIC-DoHBrw-2020, containing over a million network flow properties and statistical features of DoH, tunnels, benign DoH and normal browsing (HTTPS) traffic. Each sample is represented by 33 features with a timestamp. Our methodology combines star graph and bar plot visualizations with supervised and unsupervised learning techniques. The approach underscores the importance of C&C beacon characteristic features in compressing a dataset and reducing a flow dimension while enabling efficient detection of DoH tunnels. Through compression, the original dataset size and dimensions are reduced by approximately 95% and 94% respectively. For supervised learning, RF emerges as the top-performing algorithm, attaining precision and recall scores of 100% each, with speed increase of ≈6796 times faster in training and ≈55 in testing. For anomaly detection models, OCSVM emerges as the most suitable choice for this purpose, with precision (88.89) and recall (100). Star graph and bar graph models also show a clear difference between normal traffic and DoH tunnels. The reduction in flow sample size and dimension, while maintaining accuracy, holds promise for edge networks with constrained resources and aids security analysts in interpreting complex ML models to identify Indicators of Compromise (IoC).

TITAN: Combining a bidirectional forwarding graph and GCN to detect saturation attack targeted at SDN.

The decoupling of control and forwarding layers brings Software-Defined Networking (SDN) the network programmability and global control capability, but it also poses SDN security risks. The adversaries can use the forwarding and control decoupling character of SDN to forge legitimate traffic, launching saturation attacks targeted at SDN switches. These attacks can cause the overflow of switch flow tables, thus making the switch cannot forward benign network traffic. How to effectively detect saturation attack is a research hotspot. There are only a few graph-based saturation attack detection methods. Meanwhile, the current graph generation methods may take useless or misleading information to the attack detection, thus decreasing the attack detection accuracy. To solve the above problems, this paper proposes TITAN, a bidirecTional forwardIng graph-based saturaTion Attack detectioN method. TITAN defines flow forwarding rules and topology information, and designs flow statistical features. Based on these definitions, TITAN generates nodes of the bi-forwarding graph based on the flow statistics features and edges of the bi-forwarding graph based on the network traffic routing paths. In this way, each traffic flow in the network is transformed into a bi-directional forwarding graph. Then TITAN feeds the above bidirectional forwarding graph into a Graph Convolutional Network (GCN) to detect whether the flow is a saturation attack flow. The experimental results show that TITAN can effectively detect saturation attacks in SDNs with a detection accuracy of more than 97%.

DeepMetricCorr: Fast flow correlation for data center networks with deep metric learning

Flow correlation is a crucial task for operators to efficiently manage data center networks, as it provides a holistic perspective of the data center network by correlating ingress flow at each network node with the corresponding egress flow on a hop-by-hop basis. Existing attempts to solve the flow correlation problem involve traditional and feature-based methods, which have major limitations in application scenarios, processing speed and accuracy in the dynamic data center network environment, especially at the presence of chains of Virtual Network Functions (VNFs). Addressing this issue, this paper proposes a novel deep neural network based flow correlation method, called DeepMetricCorr. DeepMetricCorr composes multidimensional flow statistical features, metric learning, and a channel attention mechanism to solve flow correlation problems accurately. It is featured with a lightweight design which reduces computational overhead. The experiments on real-world datasets demonstrate that DeepMetricCorr outperforms other state-of-the-art methods in correlation accuracy, especially on load balancers with an over 2× improvement. Furthermore, the model maintains a low latency (¡0.5 s) as the number of candidate flows increases over 8000.

Bi-level Flow Based Anomalous Activity Identification System for IoT Devices

With the advanced technologies, IoT has widely emerged with data collection, processing, and communication as well in smart applications. The wireless medium in the IoT devices would broadcast the data, which makes them easily targeted by the attacks. In the local network, the normal communication attack is restricted to small local domain or local nodes. However, the attack present in IoT devices gets expanded to a large area that would cause destructive effects. The heterogeneity and distribution of IoT services/applications make the security of IoT a more challenging and complex one. This paper aims to propose a bi-level flow based anomalous activity identification system in IoT. Initially, the flow based features get extracted along with the statistical features like mean, median, variance, correlation, and correntropy. Subsequently, Bi-level classification is carried out in this work. In level 1, the presence of attack is detected and the level 2 classification classifies the type of attack. A decision tree is used for detecting the attacks by checking whether the network traffic is anomalous traffic or normal traffic. In level 2, an Optimized Neural network (NN) is used for categorizing the attacks in IoT with the knowledge of flow features and statistical features. To make the detection and classification more accurate, the weight of NN will be optimally tuned by a new Combined Whale SeaLion Algorithm (CWSA) that hybridizes the concepts of both SLnO and WOA. At last, the performance of the adopted method is computed over other traditional models in terms of accuracy, sensitivity, specificity, precision, FPR, FDR, FNR, NPV, F1-score, and MCC.

A Neuro-Fuzzy System to Detect IPv6 Router Alert Option DoS Packets

Detecting the denial of service attacks that solely target the router is a maximum security imperative in deploying IPv6 networks. The state-of-the-art Denial of Service detection methods aim at leveraging the advantages of flow statistical features and machine learning techniques. However, the detection performance is highly affected by the quality of the feature selector and the reliability of datasets of IPv6 flow information. This paper proposes a new neuro-fuzzy inference system to tackle the problem of classifying the packets in IPv6 networks in crucial situation of small-supervised training dataset. The proposed system is capable of classifying the IPv6 router alert option packets into denial of service and normal by utilizing the neuro-fuzzy strengths to boost the classification accuracy. A mathematical analysis from the fuzzy sets theory perspective is provided to express performance benefit of the proposed system. An empirical performance test is conducted on comprehensive dataset of IPv6 packets produced in a supervised environment. The result shows that the proposed system overcomes robustly some state-of-the-art systems.

An efficient feature generation approach based on deep learning and feature selection techniques for traffic classification

Substantial recent efforts have been made on the application of Machine Learning (ML) techniques to flow statistical features for traffic classification. However, the classification performance of ML techniques is severely degraded due to the high dimensionality and redundancy of flow statistical features, the imbalance in the number of traffic flows and concept drift of Internet traffic. With the aim of comprehensively solving these problems, this paper proposes a new feature optimization approach based on deep learning and Feature Selection (FS) techniques to provide the optimal and robust features for traffic classification. Firstly, symmetric uncertainty is exploited to remove the irrelevant features in network traffic data sets, then a feature generation model based on deep learning is applied to these relevant features for dimensionality reduction and feature generation, finally Weighted Symmetric Uncertainty (WSU) is exploited to select the optimal features by removing the redundant ones. Based on real traffic traces, experimental results show that the proposed approach can not only efficiently reduce the dimension of feature space, but also overcome the negative impacts of multi-class imbalance and concept drift problems on ML techniques. Furthermore, compared with the approaches used in the previous works, the proposed approach achieves the best classification performance and relatively higher runtime performance.

A no-reference optical flow-based quality evaluator for stereoscopic videos in curvelet domain

Most of the existing 3D video quality assessment (3D-VQA/SVQA) methods only consider spatial information by directly using an image quality evaluation method. In addition, a few take the motion information of adjacent frames into consideration. In practice, one may assume that a single data-view is unlikely to be sufficient for effectively learning the video quality. Therefore, integration of multi-view information is both valuable and necessary. In this paper, we propose an effective multi-view feature learning metric for blind stereoscopic video quality assessment (BSVQA), which jointly focuses on spatial information, temporal information and inter-frame spatio-temporal information. In our study, a set of local binary patterns (LBP) statistical features extracted from a computed frame curvelet representation are used as spatial and spatio-temporal description, and the local flow statistical features based on the estimation of optical flow are used to describe the temporal distortion. Subsequently, a support vector regression (SVR) is utilized to map the feature vectors of each single view to subjective quality scores. Finally, the scores of multiple views are pooled into the final score according to their contribution rate. Experimental results demonstrate that the proposed metric significantly outperforms the existing metrics and can achieve higher consistency with subjective quality assessment.

Novel feature selection and classification of Internet video traffic based on a hierarchical scheme

Accurate traffic classification is critical for efficient network management and resources utilization. Different video traffics have different QoS (Quality of Service) requirements. To provide Internet video services with better QoS support, a fine grained classification scheme for network video traffic is proposed in this paper. Through extensive statistical analysis of typical video traffic flows with a consistency-based method, several new flow statistical features are extracted. They are found to be more effective in discriminating different video traffics, especially from the QoS perspective, than commonly used features available in the literature. A hierarchical k-Nearest Neighbor (kNN) classification algorithm is then developed based on the combinations of these statistical features. Experiments are performed to evaluate the effectiveness of the proposed method on a large scale real network video traffic data. The experimental results show that the proposed method outperforms existing methods applying commonly used flow statistical features.

Numerical investigation of particle saltation in the bed-load regime

This paper numerically investigates particle saltation in a turbulent channel flow having a rough bed consisting of 2–3 layers of densely packed spheres. In this study, we combined three the state-of-the-art technologies, i.e., the direct numerical simulation of turbulent flow, the combined finite-discrete element modelling of the deformation, movement and collision of the particles, and the immersed boundary method for the fluid-solid interaction. Here we verify our code by comparing the flow and particle statistical features with the published data and then present the hydrodynamic forces acting on a particle together with the particle coordinates and velocities, during a typical saltation. We found strong correlation between the abruptly decreasing particle stream-wise velocity and the increasing vertical velocity at collision, which indicates that the continuous saltation of large grain-size particles is controlled by collision parameters such as particle incident angle, local rough bed packing arrangement, and particle density, etc. This physical process is different from that of particle entrainment in which turbulence coherence structures play an important role. Probability distribution functions of several important saltation parameters and the relationships between them are presented. The results show that the saltating particles hitting the windward side of the bed particles are more likely to bounce off the rough bed than those hitting the leeside. Based on the above findings, saltation mechanisms of large grain-size particles in turbulent channel flow are presented.

English

Traffic classification technique is an important tool for network and system security in the environments such as cloud computing based environment. Modern traffic classification methods plans to take the gain of flow statistical features and machine learning methods, but the classification performance is affected by reduced supervised information, and unfamiliar applications. In addition detection of anomalies in the flow level is not considered in earlier approaches. Current work proposes Flow-level anomaly detection with the framework of Unknown Flow Detection approaches. Flow-level anomaly can be detected by using Synthetic flow-level traffic trace generation approach(SG -FLT). The two major challenges with such an approach are to characterize normal and anomalous network behavior, and to discover realistic models defining normal and anomalous traffic at the flow level. Unknown flow detection approach has been performed by Flow level propagation and finding the correlated flows to boost the classification accuracy. Performance evaluation is conducted on real-world network traffic datasets which demonstrates that the proposed scheme provides efficient performance than existing methods in the complex network environment.

Rans modeling of intermittent turbulence in a thermally stable stratified boundary layer

The flow structure and statistical features of a turbulent stable stratified boundary layer are studied by using the Reynolds-averaged Navier-Stokes (RANS) scheme of turbulence, which takes into account the influence of internal gravity waves. The possibility of the RANS description of intermittent turbulence both near the surface and above the surface in the vicinity of the low-level jet flow formed above the boundary layer is analyzed. The role of turbulent diffusion (third-order statistical moments) in intermittent turbulence generation is discussed. Numerical results are demonstrated to agree with results of LES modeling and actual observations. Intermittency of turbulent kinetic energy both near the surface and above this surface in the vicinity of the low-level jet flow is revealed.

An Effective Network Traffic Classification Method with Unknown Flow Detection

Traffic classification technique is an essential tool for network and system security in the complex environments such as cloud computing based environment. The state-of-the-art traffic classification methods aim to take the advantages of flow statistical features and machine learning techniques, however the classification performance is severely affected by limited supervised information and unknown applications. To achieve effective network traffic classification, we propose a new method to tackle the problem of unknown applications in the crucial situation of a small supervised training set. The proposed method possesses the superior capability of detecting unknown flows generated by unknown applications and utilizing the correlation information among real-world network traffic to boost the classification performance. A theoretical analysis is provided to confirm performance benefit of the proposed method. Moreover, the comprehensive performance evaluation conducted on two real-world network traffic datasets shows that the proposed scheme outperforms the existing methods in the critical network environment.

Traffic classification combining flow correlation and ensemble classifier

Traffic classification has wide applications in network measurements, network security and quality of service. Recent research tends to apply machine learning methods based on flow statistical features to improve traffic classification performance. In this paper, we propose a novel non-parametric approach for traffic classification, which can improve the classification performance effectively by incorporating correlated information into the classification process. Meanwhile, our system employs a lightweight modular architecture, which combines a series of simple linear binary classifiers, each of which can be efficiently implemented and trained on vast amounts of flow data in parallel, to achieve scalability while attaining high accuracy. A large number of experiments are carried out on real traffic data to validate the proposed approach. The results show that the traffic classification performance can be improved significantly while meeting the scalability and stability requirements of large networks.

Network Traffic Classification Using Correlation Information

Traffic classification has wide applications in network management, from security monitoring to quality of service measurements. Recent research tends to apply machine learning techniques to flow statistical feature based classification methods. The nearest neighbor (NN)-based method has exhibited superior classification performance. It also has several important advantages, such as no requirements of training procedure, no risk of overfitting of parameters, and naturally being able to handle a huge number of classes. However, the performance of NN classifier can be severely affected if the size of training data is small. In this paper, we propose a novel nonparametric approach for traffic classification, which can improve the classification performance effectively by incorporating correlated information into the classification process. We analyze the new classification approach and its performance benefit from both theoretical and empirical perspectives. A large number of experiments are carried out on two real-world traffic data sets to validate the proposed approach. The results show the traffic classification performance can be improved significantly even under the extreme difficult circumstance of very few training samples.

Unsupervised traffic classification using flow statistical properties and IP packet payload

In network traffic classification, “unknown applications” is a difficult problem unsolved. Conventional supervised classification methods classify any traffic flow into predefined classes, while cannot handle unknown applications without corresponding supervised data. Some unsupervised clustering algorithms, such as k-means, have been applied to group traffic flows automatically, but a large number of resulting clusters are unable to correctly represent a small number of real applications. To address the problem of unknown applications, we propose a novel unsupervised approach which has the capability to discover application-based traffic classes and classify traffic flows according to their generation applications. In the proposed approach, flow statistical properties and IP packet payload are used in combination to discover traffic classes in the training stage. We introduce a bag-of-words (BoW) model to represent the content of clusters constructed by using flow statistical features, and apply the latent semantic analysis (LSA) to aggregate similar traffic clusters based on their payload content. In the testing stage, only flow statistical features are used to classify traffic flows, that can protect user privacy and deal with known encrypted applications without inspecting IP packets. A number of experiments are carried out on a real-world traffic dataset to demonstrate the effectiveness and robustness of the proposed approach.

Semi-supervised and compound classification of network traffic

This paper presents a new semi-supervised method to effectively improve traffic classification performance when very few supervised training data are available. Existing semi-supervised methods label a large proportion of testing flows as unknown flows due to limited supervised information, which severely affects the classification performance. To address this problem, we propose to incorporate flow correlation into both training and testing stages. At the training stage, we make use of flow correlation to extend the supervised data set by automatically labelling unlabelled flows according to their correlation to the pre-labelled flows. Consequently, a traffic classifier achieves excellent performance because of the enhanced training data set. At the testing stage, the correlated flows are identified and classified jointly by combining their individual predictions, so as to further boost the classification accuracy. The empirical study on the real-world network traffic shows that the proposed method significantly outperforms the state-of-the-art flow statistical feature based classification methods.

Anomaly traffic detection of database network based on flow statistical feature

The traditional intrusion detection system has the problem of high false positive rate and false negative rate. This paper deeply analyzes the differences of statistical features between single-flow and multi-flow on the database network, and presents a group of features that are easy to acquire and can be used to detect the anomaly in database network efficiently. By applying this group of features in Fisher algorithm for anomaly detection, the false positive rate and false negative rate are dramatically reduced. Simultaneously, the model made by using the group of features has the advantages of low algorithm complexity, good detection result and strong generalization ability. Experimental results show that there is higher accuracy when using the features of single-flow and multi-flow to construct the anomaly detection model than only using single-flow features.