Network Traffic Samples Research Articles

AI-based security functions co optimization in the SFC

Dealing with large-scale attack traffic and complex attack scenarios can be challenging for a single attack detection system in the 6G era. The ubiquitous artificial intelligence security services enabled by the AI-based Security Functions (AISF) and Service Function Chain (SFC) become strong candidate to solve this problem. However, supervised learning-based AISF requires a significant amount of manually labeled data and is unable to adapt to changing attack scenarios once it is deployed. To this end, we propose an AISF co-optimization method in the SFC. The goal is to use unlabeled network traffic samples to update the model based on pseudo-labeling and co-training. First, we model the AISF chain composed of AISFs with various detection targets and feature subspaces. We also define its detection capability evaluation metrics and final detection result. Then, we design an AISF co-optimization flow including online detection and online optimization workflows. In online optimization, we design the dynamic threshold of normalized comprehensive confidence to generate pseudo-labels for network traffic samples and combine labeled and pseudo-labeled samples to train the new models of AISFs. These models replace the previous ones and continue to detect and optimize. Experimental results in a prototype system show that compared with a single AISF, the AISF chain has higher detection capabilities that can even detect unknown attacks to a certain extent, and co-optimization can make better use of unlabeled network traffic samples than individual optimization.

Deep Q-network-based heuristic intrusion detection against edge-based SIoT zero-day attacks

How to process and classify zero-day attacks due to their huge damage to social Internet of Things (SIoT) systems has become a hot research issue. To solve this issue, we propose a heuristic learning intrusion detection system with Deep Q-Networks (DQN) for edge-based SIoT networks under the scenario of insufficient training samples, which is named DQN-HIDS. It is composed of an SIoT network traffic processing module and a DQN-based heuristic learning network. The SIoT network traffic processing module generates SIoT traffic samples, selects samples entering a classifier and a cybersecurity examiner center, and outputs similarity. We integrate DQN into a heuristic learning network to gradually improve its ability to identify malicious traffic. Specially, reward functions are designed according to the selected actions of the network, in order to punish the behavior of incorrectly labeling malicious samples and make variable reward functions adapt to different execution actions. The LSTM-based DQN then maximizes the cumulative expected reward to find the optimal strategy for the heuristic learning network. Consequently, DQN-HIDS gradually improves the behavior frequency of its labeling, reduces resource workloads, and increases the ability to label SIoT network traffic. Experiments show the performance of DQN-HIDS in terms of the workload of the examiner center and the queue workload of delayed samples, the rewards obtained by the DQN-based heuristic learning network, and the accuracy of the classifier. Comparisons with a state-of-the-art deep learning model and typical machine learning methods are also made, demonstrating the advantages of DQN-HIDS with fewer SIoT network traffic samples.

Artificial intelligence enabled Luong Attention and Hosmer Lemeshow Regression Window‐based attack detection in 6G

SummaryRegardless of the developments of networking and communication technologies, security is without exception a predominant feature to ensure network reliability. The future sixth‐generation (6G) network is anticipated to be carried out with artificial intelligence (AI) powered communication via machine learning (ML), post‐quantum cryptography, and so on. AI‐powered communication has been in recent years utilized in enhancing network traffic performance with respect to resource management, optimal frequency spectrum design, security, and latency. The studies of modern wireless communications and anticipated features of 6G networks revealed a prerequisite for designing a trustworthy attack detection mechanism. In this work, a method called, Luong Attention and Hosmer Lemeshow Regression Window‐based (LA‐HLRW) attack detection in 6G is proposed. Initially, with the raw Botnet Attack dataset obtained as input, preprocessing is performed to normalize network traffic features. Next, the dimensionality of network traffic feature of large‐scale network traffic data is reduced using the Luong Attention integrated with Long Short Term Memory (LSTM)‐based Feature extraction model. Finally, with the objective of classifying network traffic samples for attack detection in 6G, we analyze the low dimensional network traffic feature set produced by Luong Attention integrated with LSTM using the Hosmer Lemeshow Logistic Regression Window‐based Attack Detection model. Extensive experiments are performed with the Botnet Attack dataset to validate the efficiency of the proposed LA‐HLRW method by using different parameters such as attack detection accuracy, attack detection time, precision, and recall. The overall analysis of proposed LA‐HLRW results significantly reduced the attack detection time by 24%, and additionally improved attack detection accuracy, precision, and recall by 5%, 5%, and 6% as compared to existing attack detection methods respectively.

IoT botnet detection with feature reconstruction and interval optimization

The existing botnet detection methods have the problems of uneven sampling, poor feature selection, and weak generalization ability, resulting in low detection and classification results and poor adaptability to the internet of things (IoT) environment with limited computing and storage resources. This paper proposes an IoT botnet detection method using feature reconstruction and interval optimization to solve the above problems. Through the designed address triple and time window-based IP aggregation and feature reconstruction method (ATTW-IP-FR), the network traffic samples obtained from the IoT gateway are integrated, and the flow features are reconstructed to attain the reconstructed sample set. The proposed self-corrected hybrid weighted sampling algorithm balances the normal and botnet flow samples in the reconstructed sample set to get the resampling sample set. The introduced multiattribute decision-making and adjacency relation chain-based sequential forward selection algorithm is applied to eliminate the redundant features in the resampling sample set, and the optimal feature subset is obtained. The resampling sample set filtered by the optimal feature subset is detected and classified through the designed two-stage hybrid heterogeneous model optimized by the intermittent chaos and bald eagle search algorithm-based interval optimization algorithm. The experimental results show that the proposed method effectively detects the botnet in two real IoT scenarios. The detection accuracy is 99.17 % $ \% $ , the Matthews correlation coefficient is 98.35 % $ \% $ , the false positive rate is 0.25 % $ \% $ , and the false negative rate is 1.27 % $ \% $ , which are better than the existing methods. This method can effectively reduce sampling and feature selection time and space overhead and better adapt to the resource-constrained IoT environment.

Network intrusion detection based on DNA spatial information

There is an ever-increasing risk of illegal access-induced Network Intrusion (NI), which calls for prompt detection of illegal network behavior through profound Network Traffic (NT) analyses. However, current intrusion detection methods are limited in accuracy due to insufficient data standardization. This paper puts forward a deoxyribonucleic acid (DNA)-Spatial Information (SI) method to overcome these limitations. A DNA encoding model is formed, which defines a mapping relationship between NT attributes and nucleobases to reconstruct NT samples expressed as DNA sequences. Then, a feature extraction algorithm is constructed that deduces a Spatial Information Feature Matrix (SIFM) to represent sequence statistical features. A Random Forest (RF) algorithm is adopted as a matching process to determine NI behaviors considering the detection efficiency. Following experiments evaluate its method performance on two datasets, NSL-KDD and UNSW-NB15. Results demonstrate that DNA-SI obtains better results than state-of-the-art works, where the accuracy, F1-score, recall, far are 95.75%, 94.41%, 94.12%, 3.26% and 92.30%, 92.78%, 89.82%, 4.66% respectively. The fact that it is insusceptible to minority intrusion samples is another point worth attention. In sum, this quick and accurate network intrusion detection points to a new orientation for safeguarding network security.

PIoT Malicious Traffic Detection Method Based on GAN Sample Enhancement

To solve the problem of network traffic data imbalance under the background of power Internet of things and improve the poor generalization ability of the model, a PIoT malicious traffic detection method based on GAN sample enhancement is developed. Firstly, network traffic samples are preprocessed. Aiming at the imbalance of network traffic, malicious samples generation based on GAN is adopted, which uses the advantages of confrontation training in GAN to generate a small amount of malicious traffic to balance the PIoT malicious traffic. Secondly, 33 features are selected serially to construct a malicious traffic feature set by using analysis of variance and correlation analysis. Finally, the PIoT malicious traffic detection algorithm is implemented based on CatBoost and grid search. The effectiveness of the proposed method is verified on the public dataset CICIDS2017. The experimental results show that the recall rate of the proposed method on CatBoost reaches 96.60%, which is 21.16% higher than that before unbalancing, and the detection accuracy rate reaches 97.96%, which increases 8% compared to that of the other balanced methods, which significantly improves the detection performance of PIoT malicious traffic.

Evaluation of Synthetic Data Generation Techniques in the Domain of Anonymous Traffic Classification

Anonymous network traffic is more pervasive than ever due to the accessibility of services such as virtual private networks (VPN) and The Onion Router (Tor). To address the need to identify and classify this traffic, machine and deep learning solutions have become the standard. However, high-performing classifiers often scale poorly when applied to real-world traffic classification due to the heavily skewed nature of network traffic data. Prior research has found synthetic data generation to be effective at alleviating concerns surrounding class imbalance, though a limited number of these techniques have been applied to the domain of anonymous network traffic detection. This work compares the ability of a Conditional Tabular Generative Adversarial Network (CTGAN), Copula Generative Adversarial Network (CopulaGAN), Variational Autoencoder (VAE), and Synthetic Minority Over-sampling Technique (SMOTE) to create viable synthetic anonymous network traffic samples. Moreover, we evaluate the performance of several shallow boosting and bagging classifiers as well as deep learning models on the synthetic data. Ultimately, we amalgamate the data generated by the GANs, VAE, and SMOTE into a comprehensive dataset dubbed CMU-SynTraffic-2022 for future research on this topic. Our findings show that SMOTE consistently outperformed the other upsampling techniques, improving classifiers’ F1-scores over the control by ~7.5% for application type characterization. Among the tested classifiers, Light Gradient Boosting Machine achieved the highest F1-score of 90.3% on eight application types.

Android Adware Detection Using Machine Learning

Adware, an advertising-supported software, becomes a type of malware when it automatically delivers unwanted advertisements to an infected device, steals user information, and opens other vulnerabilities that allow other malware and adware to be installed. With the rise of more and complex evasive malware, specifically adware, better methods of detecting adware are required. Though a lot of work has been done on malware detection in general, very little focus has been put on the adware family. The novelty of this paper lies in analyzing the individual adware families. To date, no work has been done on analyzing the individual adware families. In this paper, using the CICAndMal2017 dataset, feature selection is performed using information gain, and classification is performed using machine learning. The best attributes for classification of each of the individual adware families using network traffic samples are presented. The results present an average classification rate that is an improvement over previous works for classification of individual adware families.

Stacked recurrent neural network for botnet detection in smart homes

Internet of Things (IoT) devices in Smart Home Network (SHN) are highly vulnerable to complex botnet attacks. In this paper, we investigate the effectiveness of Recurrent Neural Network (RNN) to correctly classify network traffic samples in the minority classes of highly imbalanced network traffic data. Multiple layers of RNN are stacked to learn the hierarchical representations of highly imbalanced network traffic data with different levels of abstraction. We evaluate the performance of Stacked RNN (SRNN) model with Bot-IoT dataset. Results show that SRNN outperformed RNN in all classification scenarios. Specifically, SRNN model learned the discriminating features of highly imbalanced network traffic samples in the training set with better representations than RNN model. Also, SRNN model is more robust and it demonstrated better capability to effectively handle over-fitting problem than RNN model. Furthermore, SRNN model achieved better generalization ability in detecting network traffic samples of the minority classes.

A Novel Way to Generate Adversarial Network Traffic Samples against Network Traffic Classification

Network traffic classification technologies could be used by attackers to implement network monitoring and then launch traffic analysis attacks or website fingerprint attacks. In order to prevent such attacks, a novel way to generate adversarial samples of network traffic from the perspective of the defender is proposed. By adding perturbation to the normal network traffic, a kind of adversarial network traffic is formed, which will cause misclassification when the attackers are implementing network traffic classification with deep convolutional neural networks (CNN) as a classification model. The paper uses the concept of adversarial samples in image recognition for reference to the field of network traffic classification and chooses several different methods to generate adversarial samples of network traffic. The experiment, in which the LeNet‐5 CNN is selected as a classification model used by attackers and Vgg16 CNN is selected as the model to test the transferability of the adversarial network traffic generated, shows the effect of the adversarial network traffic samples.

Hybrid Deep Learning for Botnet Attack Detection in the Internet-of-Things Networks

Deep learning (DL) is an efficient method for botnet attack detection. However, the volume of network traffic data and memory space required is usually large. It is, therefore, almost impossible to implement the DL method in memory-constrained Internet-of-Things (IoT) devices. In this article, we reduce the feature dimensionality of large-scale IoT network traffic data using the encoding phase of long short-term memory autoencoder (LAE). In order to classify network traffic samples correctly, we analyze the long-term inter-related changes in the low-dimensional feature set produced by LAE using deep bidirectional long short-term memory (BLSTM). Extensive experiments are performed with the BoT-IoT data set to validate the effectiveness of the proposed hybrid DL method. Results show that LAE significantly reduced the memory space required for large-scale network traffic data storage by 91.89%, and it outperformed state-of-the-art feature dimensionality reduction methods by 18.92–27.03%. Despite the significant reduction in feature size, the deep BLSTM model demonstrates robustness against model underfitting and overfitting. It also achieves good generalisation ability in binary and multiclass classification scenarios.

A Method of Few-Shot Network Intrusion Detection Based on Meta-Learning Framework

Conventional intrusion detection systems based on supervised learning techniques require a large number of samples for training, while in some scenarios, such as zero-day attacks, security agencies can only intercept a limited number of shots of malicious samples. Therefore, there is a need for few-shot detection. In this paper, a detection method based on a meta-learning framework is proposed for this purpose. The proposed method can be used to distinguish and compare a pair of network traffic samples as a basic task of learning, including a normal unaffected sample and a malicious one. To accomplish this task, we design a deep neural network (DNN) named FC-Net, which mainly comprises two parts: feature extraction network and comparison network. FC-Net learns a pair of feature maps for classification from a pair of network traffic samples, then compares the obtained feature maps, and finally determines whether the pair of samples belongs to the same type. To evaluate the proposed detection method, we construct two datasets for few-shot network intrusion detection based on real network traffic data sources, using a specifically developed approach. The experimental results indicate that the proposed detection method is universal and is not limited to specific datasets or attack types. Training and testing on the same datasets demonstrate that the proposed method can achieve the average detection rate up to 98.88%. The outcome of training on one dataset and testing on the other one confirms that the proposed method can achieve better performance. In a few-shot scenario, malicious samples in an untrained dataset can be detected successfully, and the average detection rate is up to 99.62%.

A Network Traffic Classification Model Based on Metric Learning

Attacks on websites and network servers are among the most critical threats in network security. Network behavior identification is one of the most effective ways to identify malicious network intrusions. Analyzing abnormal network traffic patterns and traffic classification based on labeled network traffic data are among the most effective approaches for network behavior identification. Traditional methods for network traffic classification utilize algorithms such as Naive Bayes, Decision Tree and XGBoost. However, network traffic classification, which is required for network behavior identification, generally suffers from the problem of low accuracy even with the recently proposed deep learning models. To improve network traffic classification accuracy thus improving network intrusion detection rate, this paper proposes a new network traffic classification model, called ArcMargin, which incorporates metric learning into a convolutional neural network (CNN) to make the CNN model more discriminative. ArcMargin maps network traffic samples from the same category more closely while samples from different categories are mapped as far apart as possible. The metric learning regularization feature is called additive angular margin loss, and it is embedded in the object function of traditional CNN models. The proposed ArcMargin model is validated with three datasets and is compared with several other related algorithms. According to a set of classification indicators, the ArcMargin model is proofed to have better performances in both network traffic classification tasks and open-set tasks. Moreover, in open-set tasks, the ArcMargin model can cluster unknown data classes that do not exist in the previous training dataset.

Network Traffic Sampling System Based on Storage Compression for Application Classification Detection

With the development of the Internet, numerous new applications have emerged, the features of which are constantly changing. It is necessary to perform application classification detection on the network traffic to monitor the changes in the applications. Using RelSamp to sample traffic can provide the sampled traffic with sufficient application features to support application classification. RelSamp separately assigns counters for each flow to record the statistical features and introduces a collision chain into the hash flow table to resolve hash conflicts in the table entries. However, in high-speed networks, owing to the number of concurrent flows and heavy-tailed nature of the traffic, the storage allocation method of RelSamp results in a significant waste of storage on the traffic sampling device. Moreover, the hash conflict resolution of RelSamp causes the collision chains of several hash table entries to be excessively deep, thereby reducing the search efficiency of the flow nodes. To overcome the shortcomings of RelSamp, this study presents a sampling model known as MiniSamp. Based on the RelSamp sampling mechanism, MiniSamp introduces shared counter trees to compress the storage space of the counters during the sampling process and integrates an efficient search tree into the hash table. The search tree structure is adjusted according to the network environment to improve the search efficiency of the flow nodes. The experimental results demonstrate that MiniSamp can effectively aid network operators to classify traffic in the high-speed network.

Hadoop‐based analytic framework for cyber forensics

SummaryWith an exponential increase in the data size and complexity of various documents to be investigated, existing methods of network forensics are found not much efficient with respect to accuracy and detection ratio. The existing techniques for network forensic analysis exhibit inherent limitations while processing a huge volume, variety, and velocity of data. It makes network forensic a time‐consuming and resource‐consuming task. To balance time taken and output delivered, these existing techniques put a limit on the amount of data under analysis, which results in a polynomial time complexity of these solutions. So to mitigate these issues, in this paper, we propose an effective framework to overcome the limitation to handle large volume, variety, and velocity of data. An architectural setup that consists of MapReduce framework on top of Hadoop Distributed File System environment is proposed in this paper. The proposed framework demonstrates its capability to handle issues of storage and processing of big data using cloud computing. Also, in the proposed framework, supervised machine learning (random forest‐based decision tree) algorithm has been implemented to demonstrate better sensitivity. To train and validate the model, online available data set from CAIDA is taken and university network traffic samples, with increasing size, has been taken for experiment. Results thus obtained confirm the superiority of the proposed framework in network forensics, with an average accuracy of 99.34% (malicious and nonmalicious traffic).

A network traffic prediction method based on IFS algorithm optimised LSSVM

How to predict network traffic accurately is an important issue in the network congestion control and network management. A network traffic prediction method based on improved free search algorithm optimised least squares support vector machines is proposed. Firstly, the Hurst exponent calculation shows that the network traffic time series has predictability, nonlinear and long-related characteristics, so least squares support vector machines is chosen as prediction model. Then, an improved free search algorithm is introduced so that it can be applied into the parameters optimisation of prediction model based on least squares support vector machines. Finally, the actual network traffic samples data of LAN and WAN are chosen as the simulation object, the simulation results show that the improved free search algorithm has faster convergence speed and better fitness value. Compared with other prediction methods, the proposed prediction method has better predictive effect and smaller predictive error. At the same time, the complexity of computation time shows that the proposed prediction method not only improves the prediction performance, but also does not increase the complexity of the algorithm.

A network traffic prediction method based on IFS algorithm optimised LSSVM

How to predict network traffic accurately is an important issue in the network congestion control and network management. A network traffic prediction method based on improved free search algorithm optimised least squares support vector machines is proposed. Firstly, the Hurst exponent calculation shows that the network traffic time series has predictability, nonlinear and long-related characteristics, so least squares support vector machines is chosen as prediction model. Then, an improved free search algorithm is introduced so that it can be applied into the parameters optimisation of prediction model based on least squares support vector machines. Finally, the actual network traffic samples data of LAN and WAN are chosen as the simulation object, the simulation results show that the improved free search algorithm has faster convergence speed and better fitness value. Compared with other prediction methods, the proposed prediction method has better predictive effect and smaller predictive error. At the same time, the complexity of computation time shows that the proposed prediction method not only improves the prediction performance, but also does not increase the complexity of the algorithm.

Testing the capacity of off-the-shelf systems to store 10GbE traffic

The maturity of the telecommunications market and the fact that user demands increase every day leaves network operators no option but to deploy high-speed infrastructures and test them in an efficient and economical manner. A common approach to this problem has been the storage of network traffic samples for analysis and replay using different versions of what we have named NTSS. This type of task is particularly demanding in 10 Gb Ethernet links and has traditionally been addressed by closed solutions or NTSS built on top of high-end hardware. However, these approaches lack flexibility and extensibility, which typically translates into higher cost. This work studies how NTSS can be built using COTS: a combination of commodity hardware and open source software. To this end, we present the current limitations of COTS systems and focus on low-level optimization techniques at several levels: the NIC driver, hard drives, and the software interaction between them. The application of these techniques has proven crucial for reaching 10 Gb/s rates, as different state-of-the-art systems have shown after an extensive performance test.

Network intrusion detection in covariance feature space

Detecting multiple and various network intrusions is essential to maintain the reliability of network services. The problem of network intrusion detection can be regarded as a pattern recognition problem. Traditional detection approaches neglect the correlation information contained in groups of network traffic samples which leads to their failure to improve the detection effectiveness. This paper directly utilizes the covariance matrices of sequential samples to detect multiple network attacks. It constructs a covariance feature space where the correlation differences among sequential samples are evaluated. Two statistical supervised learning approaches are compared: a proposed threshold based detection approach and a traditional decision tree approach. Experimental results show that both achieve high performance in distinguishing multiple known attacks while the threshold based detection approach offers an advantage of identifying unknown attacks. It is also pointed out that utilizing statistical information in groups of samples, especially utilizing the covariance information, will benefit the detection effectiveness.

A multipurpose, distributed lan traffic monitoring tool

A description is given of a distributed monitoring system that comprises one or more high-speed hardware monitoring engines, which perform the network traffic sampling and do initial data reduction, and a set of software functions, which perform further data reduction and traffic analysis. The hardware monitoring engine monitors the wire itself, facilitating the detection of all traffic, not just traffic to or from known entities, The monitor communicates with a remote system, or host, running an integrated set of software functions and providing a user interface to these functions. Monitoring the wire itself allows the monitoring complexity to be off-loaded from all of the network entities being monitored to the hardware monitoring engines, thereby decreasing the cost of data collection software having to be built into all of the entities being monitored. This is especially advantageous in multivendor, multiprotocol environments, where it is not usually possible to require that each entity in the network implements the necessary data collection function.