Network Traffic Analysis Research Articles

A comprehensive survey on deep packet inspection for advanced network traffic analysis: issues and challenges

Deep Packet Inspection (DPI) provides full visibility into network traffic by performing detailed analysis on both packet header and packet payload. Accordingly, DPI has critical importance as it can be used in applications i.e network security or government surveillance. In this paper, we provide an extensive survey on DPI. Different from the previous studies, we try to efficiently integrate DPI techniques into network analysis mechanisms by identifying performance-limiting parameters in the analysis of modern network traffic. Analysis of the network traffic model with complex behaviors is carried out with powerful hybrid systems by combining more than one technique. Therefore, DPI methods are studied together with other techniques used in the analysis of network traffic. Security applications of DPI on Internet of Things (IoT) and Software-Defined Networking (SDN) architectures are discussed and Intrusion Detection Systems (IDS) mechanisms, in which the DPI is applied as a component of the hybrid system, are examined. In addition, methods that perform inspection of encrypted network traffic are emphasized and these methods are evaluated from the point of security, performance and functionality. Future research issues are also discussed taking into account the implementation challenges for all DPI processes.

Graph-Based Multi-Label Classification for WiFi Network Traffic Analysis

Network traffic analysis, and specifically anomaly and attack detection, call for sophisticated tools relying on a large number of features. Mathematical modeling is extremely difficult, given the ample variety of traffic patterns and the subtle and varied ways that malicious activity can be carried out in a network. We address this problem by exploiting data-driven modeling and computational intelligence techniques. Sequences of packets captured on the communication medium are considered, along with multi-label metadata. Graph-based modeling of the data are introduced, thus resorting to the powerful GRALG approach based on feature information granulation, identification of a representative alphabet, embedding and genetic optimization. The obtained classifier is evaluated both under accuracy and complexity for two different supervised problems and compared with state-of-the-art algorithms. We show that the proposed preprocessing strategy is able to describe higher level relations between data instances in the input domain, thus allowing the algorithms to suitably reconstruct the structure of the input domain itself. Furthermore, the considered Granular Computing approach is able to extract knowledge on multiple semantic levels, thus effectively describing anomalies as subgraphs-based symbols of the whole network graph, in a specific time interval. Interesting performances can thus be achieved in identifying network traffic patterns, in spite of the complexity of the considered traffic classes.

Detection of Malicious Network Flows with Low Preprocessing Overhead

Machine learning (ML) is frequently used to identify malicious traffic flows on a network. However, the requirement of complex preprocessing of network data to extract features or attributes of interest before applying the ML models restricts their use to offline analysis of previously captured network traffic to identify attacks that have already occurred. This paper applies machine learning analysis for network security with low preprocessing overhead. Raw network data are converted directly into bitmap files and processed through a Two-Dimensional Convolutional Neural Network (2D-CNN) model to identify malicious traffic. The model has high accuracy in detecting various malicious traffic flows, even zero-day attacks, based on testing with three open-source network traffic datasets. The overhead of preprocessing the network data before applying the 2D-CNN model is very low, making it suitable for on-the-fly network traffic analysis for malicious traffic flows.

Invalidation of tunnelling attacks in ubiquitous IoT & wireless sensor enviroment using ML methods

Wireless sensors are interconnected are is ubiquitous in nature and provides great advantage are normally constituting perfect communication on wireless networks like infrastructure and Ad Hoc. The wireless connected devices environment can also be created using Internet of Things and also it can be adhoc. These wireless sensors are installed in the military campus to collect information movement of people, sound, capturing pictures of their activities. These kinds of sensors can be used in most secured places, where there is a need of remote monitoring of the security. It can be either used in atomic power stations or military camps. Based on the same, Ad Hoc networks attract great attention because data obstruction is made in communication interval. The communication interval helps to improve the mobile node convenience, node mobility, scalability and network cost. Perhaps, due the overload in data packet transmission, the routing makes the packet blockage between the nodes. Mobile nodes are always roaming around the particular region for supervise the external conditions or surroundings characteristics in order to accumulates the selective information about alive nodes and its communication route. On certain occasions these communication routes are vulnerable to various attacks due to functionality and deployment scenario, especially wormhole attack. This is an Ad Hoc assembling of mobile nodes that can randomly alter their route with respect to the constrained resources such as bandwidth, energy, security and resource discovery. Based on the above resources wormhole nodes create the fraudulent route within the network; the fraudulent route can confuse routing mechanisms which creates the tunnel between nodes. In order to invalidate the wormhole attacks in the route within the network, this research article proposed the “Train and Avoiding” process. The train and avoid process is the machine learning algorithm to find the optimal route where there are no attacks in the route. The process trains the routing to perform network traffic analysis. In addition to this, avoiding process helps to restrict the packets modification as well as fake route creation. Proposed Train and Avoiding based route Selection is used to achieve more prominent reliable routing. It improves PDR, average lifetime of network and minimizes communication issues, and average delay.

Developing machine learning based framework for the network traffic prediction

Network traffic analysis is a crucial step in developing efficient congestion control systems and identifying valid and malicious packets. Because network resources are apportioned based on predicted usage, these solutions reduce network congestion. For a variety of reasons, including dynamic bandwidth allocation, network security, and network planning, the ability to forecast network traffic is critical. Machine learning (ML) techniques to network traffic analysis have received a lot of interest. This article outlines an approach for analyzing network traffic. Three machine learning-based methodologies make up the methodology. The experimental investigation employed the NSL KDD data set. On the basis of accuracy and other criteria, KNN, Support vector machine, and nave bayes are compared.

Early web application attack detection using network traffic analysis

Early web application attack detection using network traffic analysis

A deep learning‐based framework to identify and characterise heterogeneous secure network traffic

AbstractThe evergrowing diversity of encrypted and anonymous network traffic makes network management more formidable to manage the network traffic. An intelligent system is essential to analyse and identify network traffic accurately. Network management needs such techniques to improve the Quality of Service and ensure the flow of secure network traffic. However, due to the usage of non‐standard ports and encryption of data payloads, the classical port‐based and payload‐based classification techniques fail to classify the secured network traffic. To solve the above‐mentioned problems, this paper proposed an effective deep learning‐based framework employed with flow‐time‐based features to predict heterogeneous secure network traffic best. The state‐of‐the‐art machine learning strategies (C4.5, random forest, and K‐nearest neighbour) are investigated for comparison. The proposed 1D‐CNN model achieved higher accuracy in classifying the heterogeneous secure network traffic. In the next step, the proposed deep learning model characterises the major categories (virtual private network traffic, the onion router network traffic, and plain encrypted network traffic) into several application types. The experimental results show the effectiveness and feasibility of the proposed deep learning framework, which yields improved predictive power compared to the state‐of‐the‐art machine learning techniques employed for secure network traffic analysis.

Network intrusion detection based on DNA spatial information

There is an ever-increasing risk of illegal access-induced Network Intrusion (NI), which calls for prompt detection of illegal network behavior through profound Network Traffic (NT) analyses. However, current intrusion detection methods are limited in accuracy due to insufficient data standardization. This paper puts forward a deoxyribonucleic acid (DNA)-Spatial Information (SI) method to overcome these limitations. A DNA encoding model is formed, which defines a mapping relationship between NT attributes and nucleobases to reconstruct NT samples expressed as DNA sequences. Then, a feature extraction algorithm is constructed that deduces a Spatial Information Feature Matrix (SIFM) to represent sequence statistical features. A Random Forest (RF) algorithm is adopted as a matching process to determine NI behaviors considering the detection efficiency. Following experiments evaluate its method performance on two datasets, NSL-KDD and UNSW-NB15. Results demonstrate that DNA-SI obtains better results than state-of-the-art works, where the accuracy, F1-score, recall, far are 95.75%, 94.41%, 94.12%, 3.26% and 92.30%, 92.78%, 89.82%, 4.66% respectively. The fact that it is insusceptible to minority intrusion samples is another point worth attention. In sum, this quick and accurate network intrusion detection points to a new orientation for safeguarding network security.

User‐centered road network traffic analysis with MobilityDB

AbstractPerformance indicators of road networks are a long‐lasting topic of research. Existing schemes assess network properties such as the average speed on road segments and the queuing time at intersections. The increasing availability of user trajectories, collected mainly using mobile phones with a variety of applications, creates opportunities for developing user‐centered performance indicators. Performing such an analysis on big trajectory data sets remains a challenge for the existing data management systems, because they lack support for spatiotemporal trajectory data. This article presents an end‐to‐end solution, based on MobilityDB, a novel moving object database system that extends PostgreSQL with spatiotemporal data types and functions. A new class of indicators is proposed, focused on the users' experience. The indicators address the network design, the traffic flow, and the driving comfort of the motorists. Furthermore, these indicators are expressed as analytical MobilityDB queries over a big set of real vehicle trajectories.

Urban Traffic Pattern Analysis and Applications Based on Spatio-Temporal Non-Negative Matrix Factorization

Analyzing the traffic state of large citywide networks is an inherently difficult task. Various data issues, traffic signals, stops signs and other flow inhibitors of the network-level traffic state make the analysis more difficult than that under the small-scale local traffic state. To address this challenge, we propose a method based on spatio-temporal non-negative matrix factorization (ST-NMF), which is used for road network traffic pattern analysis. The method can be further extended to traffic data reconstruction and traffic prediction. In order to analyze traffic patterns, the proposed spatio-temporal non-negative matrix factorization model represents the network traffic as a linear combination of several basic patterns, which is also interpreted as the dynamics of spatial traffic characteristics over time in low-dimensional space. By the visual display of the spatial and temporal patterns and the assistance of clustering methods, the traffic pattern features are extracted. In the extended applications, data reconstruction relies on the sampling representation of missing data by ST-NMF, and data prediction is based on the prediction of the temporal patterns by ST-NMF. Through our method, we can not only obtain a high-quality data foundation, but also explore typical spatio-temporal patterns and general predictions of the future traffic state. The analysis results have important guiding significance on the management of intelligent transportation systems. Experiments on real-world traffic data are provided to verify the validity of our proposed approach.

IoT Multi-Vector Cyberattack Detection Based on Machine Learning Algorithms: Traffic Features Analysis, Experiments, and Efficiency

Cybersecurity is a common Internet of Things security challenge. The lack of security in IoT devices has led to a great number of devices being compromised, with threats from both inside and outside the IoT infrastructure. Attacks on the IoT infrastructure result in device hacking, data theft, financial loss, instability, or even physical damage to devices. This requires the development of new approaches to ensure high-security levels in IoT infrastructure. To solve this problem, we propose a new approach for IoT cyberattack detection based on machine learning algorithms. The core of the method involves network traffic analyses that IoT devices generate during communication. The proposed approach deals with the set of network traffic features that may indicate the presence of cyberattacks in the IoT infrastructure and compromised IoT devices. Based on the obtained features for each IoT device, the feature vectors are formed. To conclude the possible attack presence, machine learning algorithms were employed. We assessed the complexity and time of machine learning algorithm implementation considering multi-vector cyberattacks on IoT infrastructure. Experiments were conducted to approve the method’s efficiency. The results demonstrated that the network traffic feature-based approach allows the detection of multi-vector cyberattacks with high efficiency.

Robust stacking ensemble model for darknet traffic classification under adversarial settings

Encrypted traffic tunnelled by Tor or VPN is referred to as darknet traffic. The ability to detect, identify, and characterize darknet traffic is critical for detecting network traffic generated by a cyber-attack. Darknet classification models based on Machine Learning / Deep Learning (ML/DL) usually demonstrate high False Positive Rate (FPR) and lower F1-score which are essential metrics for network traffic analysis. Additionally, ML/DL models used in such tasks are susceptible to adversarial perturbed samples that can cause the network security solution to malfunction. This work proposes a Stacking Ensemble (SE) model to combine the predictions of three base learners, 1) Random Forest, 2) K-Nearest Neighbors, and 3) Decision Tree in the most efficient way for improving the overall performance of darknet characterization. A two-layered autoencoder-based defence mechanism consisting of a detector and denoiser is devised to increase the robustness of the network security system against adversarial attacks. The performance of the proposed model is demonstrated through extensive experiments using the CIC-Darknet-2020 dataset. The robustness of the model is tested against three highly transferable adversarial attacks, 1) Fast Gradient Sign Method (FGSM), 2) Basic Iterative Method (BIM), 3) DeepFool and a realistic 4) Boundary attack. The experimental results show that the SE model outperforms the baseline Deep Image and other competitor models by achieving an accuracy score of 98.89% and an FPR of 0.43% in the case of darknet traffic identification. Furthermore, an accuracy of 97.88% is achieved in the case of darknet traffic flow characterization. In addition, the statistical properties of the packet flow are identified as the most commonly affected features by adversarial attacks through the Mean Adversarial Perturbation (MAP) metric.

A Novel Method for Improved Network Traffic Prediction Using Enhanced Deep Reinforcement Learning Algorithm.

Network data traffic is increasing with expanded networks for various applications, with text, image, audio, and video for inevitable needs. Network traffic pattern identification and analysis of traffic of data content are essential for different needs and different scenarios. Many approaches have been followed, both before and after the introduction of machine and deep learning algorithms as intelligence computation. The network traffic analysis is the process of incarcerating traffic of a network and observing it deeply to predict what the manifestation in traffic of the network is. To enhance the quality of service (QoS) of a network, it is important to estimate the network traffic and analyze its accuracy and precision, as well as the false positive and negative rates, with suitable algorithms. This proposed work is coining a new method using an enhanced deep reinforcement learning (EDRL) algorithm to improve network traffic analysis and prediction. The importance of this proposed work is to contribute towards intelligence-based network traffic prediction and solve network management issues. An experiment was carried out to check the accuracy and precision, as well as the false positive and negative parameters with EDRL. Also, convolutional neural network (CNN) machines and deep learning algorithms have been used to predict the different types of network traffic, which are labeled text-based, video-based, and unencrypted and encrypted data traffic. The EDRL algorithm has outperformed with mean Accuracy (97.20%), mean Precision (97.343%), mean false positive (2.657%) and mean false negative (2.527%) than the CNN algorithm.

Task adaptive siamese neural networks for open-set recognition of encrypted network traffic with bidirectional dropout

• Create highly distinguishable similarity scores by a bidirectional dropout method. • Handle task adaptive open-set recognition by a non-parametric Bayesian based metric. • Improve the confidence of similarity scores by a hierarchical cross entropy loss. • Experiments demonstrate our method achieves high performance and generalizability . Existing deep learning approaches have achieved high performance in encrypted network traffic analysis tasks. However, practical requirements such as open-set recognition on dynamically changing tasks (e.g., changes in the target website list), challenge existing methods. While few-shot learning and open-set recognition methods have been proposed for domains such as computer vision , few-shot open-set recognition for encrypted network traffic remains an unexplored area. This paper proposes a task adaptive siamese neural network for open-set recognition of encrypted network traffic with bidirectional dropout data augmentation. Our contributions are three-fold: First, we introduce generated positive and negative pairs into the siamese neural network training process to shape a more precise similarity boundary through bidirectional dropout data augmentation. Second, we utilize Dirichlet Process Gaussian Mixture Model (DPGMM) distribution to fit the similarity scores of the negative pairs constructed by the support set of each query task, and create a new open-set recognition metric. Third, by leveraging the extracted features at coarse and fine granular levels , we construct a hierarchical cross entropy loss to improve the confidence of the similarity score. Extensive experiments on a network traffic dataset and the Omniglot dataset demonstrate the superiority and generalizability of our proposed approach.

A Network Traffic Analysis System based on Big Data

With the rapid development of network technology, network applications are becoming more and more popular. In order to ensure the availability of network and the stable operation of key business, the methodology of network traffic analysis appeared accordingly. Compared with the normal non-big data traffic analysis, using big data for traffic analysis and monitoring can solve the problems of single data source, difficulty in chart expansion horizontally and chart customization, etc. Based on big data technology to build a unified platform on traffic analysis and monitoring, it is able to integrate with the network analysis of data and other dimensions of the associated monitoring information to achieve efficient service on fault warning and navigating location.

Automatic Intersection Extraction Method for Urban Road Networks Based on Trajectory Intersection Points

Automatic intersection identification and extraction are an important foundation for urban road network updates and traffic network analysis and modeling. Existing intersection extraction methods based on steering angles and stopping points suffer from inadequate sampling amounts and threshold settings. To address this problem, we propose a road network intersection automatic extraction method based on vehicle trajectory intersection clustering. First, the continuous trajectory segments are extracted from trajectory data based on the sampling interval. Second, the maximum reconstruction error method is developed to extract straight-line trajectory segments from continuous trajectory segments. The overlapped straight-line trajectory segments belonging to the same direction are merged to reduce the number of segments and enhance road network patterns. To further improve the calculation efficiency of the intersection points of straight-line segments, bounding box filtering and orthogonal filtering are used to filter the straight-line trajectory segments that do not have an intersection relationship. Finally, the obtained straight-line segment intersection points are clustered using a density peak clustering algorithm. The road intersections are automatically extracted using the clustering center. The experimental results on real vehicle trajectories in Lianyungang City show that the proposed method performs well on intersection recognition and calculation efficiency.

Android malware detection using network traffic based on sequential deep learning models

AbstractThe increasing trend of smartphone capabilities has caught the attention of many users. This has led to the emergence of malware that threatening the users' privacy and security. Many malware detection methods have been proposed to deal with emerging threats. One of the most effective ones is to use network traffic analysis. This article proposed a method based on LSTM (Long Short‐term Memory) for malware detection which is capable of not only distinguishing malware and benign samples, but also detecting and identify the new and unseen families of malware. As far as we know, this is the first time that traffic data has been modeled as a sequence of flows and a sequential based deep learning model is employed. In this article, we have performed several case studies to exhibit the capabilities of the proposed method including malware detection, malware family identification, new (not seen before) malware family detection, as well as evaluating the minimum time required to detect malware. The case studies show that the model is even capable of detecting new families of malware with more than 90% accuracy, although these results can only be verified on existing families in this dataset and such a claim cannot be generalized to other examples of malware. Moreover, it is shown the model is able to detect the malware through capturing 50 connection flows (about 1600 packets in average) with the AUC of more than 99.9%.

Transportation systems management considering dynamic wireless charging electric vehicles: Review and prospects

This paper reviews and prospects the research in transportation systems management with the presence of dynamic wireless charging (DWC) electric vehicles (EVs). DWC is based on inductive coupling that allows EVs to get charged wirelessly by driving over the coils installed in the ground. Thus, DWC is also known as ‘charging-while-driving’ or ‘in-motion charging’, which is regarded as one of the most promising charging solutions for EVs. The road that is equipped with wireless chargers underneath its surface is called a wireless charging lane (WCL). The deployment of WCLs will lead to a profound change in the field of transportation systems management. We give a review of recent studies on related topics including WCL allocation and related decisions, energy consumption of EVs considering WCLs, and pricing of EVs for the use of DWC facilities. Based on the analysis of the existing studies, we identify the needs for more flexible WCL design and allocation models, as well as novel ways that incorporate the complexities in energy consumption by EVs and routing behaviour into traffic network analysis. In addition, we emphasise the research opportunities of dynamic traffic modelling and real-time control considering DWC and EVs with state-of-charge (SOC) constraints. A high-level dynamic system model and a general model-based control problem are formulated in this regard. Finally, we foresee a demand for finer traffic management in the WCL environment enabled by emerging technologies such as the internet of vehicles (IoV) and autonomous driving.

The Robustness of Detecting Known and Unknown DDoS Saturation Attacks in SDN via the Integration of Supervised and Semi-Supervised Classifiers

The design of existing machine-learning-based DoS detection systems in software-defined networking (SDN) suffers from two major problems. First, the proper time window for conducting network traffic analysis is unknown and has proven challenging to determine. Second, it is unable to detect unknown types of DoS saturation attacks. An unknown saturation attack is an attack that is not represented in the training data. In this paper, we evaluate three supervised classifiers for detecting a family of DDoS flooding attacks (UDP, TCP-SYN, IP-Spoofing, TCP-SARFU, and ICMP) and their combinations using different time windows. This work represents an extension of the runner-up best-paper award entitled ‘Detecting Saturation Attacks in SDN via Machine Learning’ published in the 2019 4th International Conference on Computing, Communications and Security (ICCCS). The results in this paper show that the trained supervised models fail in detecting unknown saturation attacks, and their overall detection performance decreases when the time window of the network traffic increases. Moreover, we investigate the performance of four semi-supervised classifiers in detecting unknown flooding attacks. The results indicate that semi-supervised classifiers outperform the supervised classifiers in the detection of unknown flooding attacks. Furthermore, to further increase the possibility of detecting the known and unknown flooding attacks, we propose an enhanced hybrid approach that combines two supervised and semi-supervised classifiers. The results demonstrate that the hybrid approach has outperformed individually supervised or semi-supervised classifiers in detecting the known and unknown flooding DoS attacks in SDN.

Hybrid model for bot group activity detection using similarity and correlation approaches based on network traffic flows analysis

In this cyber era, botnets have been a serious threat to computer network security in that they can infect computers connected to a network through malicious applications known as malware. Unlike their previous behavior, botnets have evolved from being centralized to decentralized. Thus, detecting and handling bots' activity is challenging. On the other hand, botnets can actively infect and attack the target concurrently, called bot group activities. Existing detection approaches cannot recognize the activity relation between bots in their group, called activity correlation. This correlation is crucial in obtaining the activity causality between bots because it can identify which bot activity affects the other bot activities during the attack. It is the causality of bot activities that helps prevent bot group attacks. This paper proposes a new model for detecting bot group activity using a hybrid analysis approach, which includes extracting activity patterns using a sliding window segmentation technique, analyzing activity similarities between bots, and analyzing their correlation. The experiment uses two public datasets to evaluate the proposed method. The results show that it can detect bot group activity with as high as 99.73% accuracy, which is better than others, with less than 1% of the false-positive rate.