Network Traffic Data Research Articles

IoT Traffic Analyzer Tool with Automated and Holistic Feature Extraction Capability.

The Internet of Things (IoT) is an emerging technology that attracted considerable attention in the last decade to become one of the most researched topics in computer science studies. This research aims to develop a benchmark framework for a public multi-task IoT traffic analyzer tool that holistically extracts network traffic features from an IoT device in a smart home environment that researchers in various IoT industries can implement to collect information about IoT network behavior. A custom testbed with four IoT devices is created to collect real-time network traffic data based on seventeen comprehensive scenarios of these devices' possible interactions. The output data is fed into the IoT traffic analyzer tool for both flow and packet levels analysis to extract all possible features. Such features are ultimately classified into five categories: IoT device type, IoT device behavior, Human interaction type, IoT behavior within the network, and Abnormal behavior. The tool is then evaluated by 20 users considering three variables: usefulness, accuracy of information being extracted, performance and usability. Users in three groups were highly satisfied with the interface and ease of use of the tool, with scores ranging from 90.5% to 93.8% and with an average score between 4.52 and 4.69 with a low standard deviation range, indicating that most of the data revolve around the mean.

A parallel low rank matrix optimization method for recovering internet traffic network data via link flow measurement

Internet traffic network data recovery via link flow measurement is an important problem in wireless communication network, but as the problem scale grows rapidly, it is quite challenging to give consideration to both recovery accuracy and computational cost. We construct a novel parallel low-rank matrix optimization model to accurately and quickly recover internet traffic network data via link flow measurement. This model takes full advantage of the low-rankness of traffic network data. Then, an inexact symmetric Gauss–Seidel-based majorized semi-proximal alternating direction method of multipliers is proposed to solve the model. Our method is proved to be globally convergent, and the numerical experiments on the classical Abilene and GÉANT datasets indicate that the performance of our method for fast and accurate recovery of traffic network data is better than that of the state-of-the-art methods. Specially, the numerical results on the large-scale HOD dataset demonstrates that our method is quite suitable for traffic network data recovery problem from realistic scenarios.

Securing a Smart Home with a Transformer-Based IoT Intrusion Detection System

Machine learning (ML)-based Network Intrusion Detection Systems (NIDSs) can classify each network’s flow behavior as benign or malicious by detecting heterogeneous features, including both categorical and numerical features. However, the present ML-based NIDSs are deemed insufficient in terms of their ability to generalize, particularly in changing network environments such as the Internet of Things (IoT)-based smart home. Although IoT devices add so much to home comforts, they also introduce potential risks and vulnerabilities. Recently, many NIDS studies on other IoT scenarios, such as the Internet of Vehicles (IoV) and smart cities, focus on utilizing the telemetry data of IoT devices for IoT intrusion detection. Because when IoT devices are under attack, their abnormal telemetry data values can reflect the anomaly state of those devices. Those telemetry data-based IoT NIDS methods detect intrusion events from a different view, focusing on the attack impact, from the traditional network traffic-based NIDS, which focuses on analyzing attack behavior. The telemetry data-based NIDS is more suitable for IoT devices without built-in security mechanisms. Considering the smart home IoT scenario, which has a smaller scope and a limited number of IoT devices compared to other IoT scenarios, both NIDS views can work independently. This motivated us to propose a novel ML-based NIDS to combine the network traffic-based and telemetry data-based NIDS together. In this paper, we propose a Transformer-based IoT NIDS method to learn the behaviors and effects of attacks from different types of data that are generated in the heterogeneous IoT environment. The proposed method utilizes a self-attention mechanism to learn contextual embeddings for input network features. Based on the contextual embeddings, our method can solve the feature set challenge, including both continuous and categorical features. Our method is the first to utilize both network traffic data and IoT sensors’ telemetry data at the same time for intrusion detection. Experiments reveal the effectiveness of our method on a realistic network traffic intrusion detection dataset named ToN_IoT, with an accuracy of 97.95% for binary classification and 95.78% for multiple classifications on pure network data. With the extra IoT information, the performance of our method has been improved to 98.39% and 97.06%, respectively. A comparative study with existing works shows that our method can achieve state-of-the-art performance on the ToN_IoT dataset.

Deep metric learning based approach for network intrusion detection

Today, intrusion detection systems are the way to defend network intrusion flows. In this paper, to categorize network traffic data, we proposed a novel method for detecting network intrusions. It builds intrusion detection models using a deep metric learning (DML) strategy that incorporates two multi-scale convolutional neural networks (MSCNN) and a Triplet network. During the phase of training MSCNN networks, the network traffic data are divided into attack network traffic data and normal data, and we train two distinct MSCNN networks on the basis of these two datasets. To determine the distance between the network traffic data, a Triplet network is trained to learn a mapping space which preserves the relationships of attacks and normal network flows. Soft-margin triplet loss is used as a loss function to train the Triplet network. In the prediction stage, each new flow passes through two MSCNN networks, which reconstruct the network flows, and then the Triplet network measures the distance between the network flow and the reconstructed flow. To verify the model’s advantages in intrusion detection, we compare the performance of the MSCNN+Triplet network, the CNN+LSTM, and the shallow neural network. Experimental results reveal the proposed scheme has superior detection performance compared to other intrusion detection schemes. The code will be available at https://gitee.com/wbsk/mscnn_triplet/tree/master/.

Research on hybrid intrusion detection based on improved Harris Hawk optimization algorithm

Aiming at the problem of low detection accuracy of network traffic data types by traditional intrusion detection methods, we propose an improved Harris Hawk hybrid intrusion detection method to enhance the detection capability. The improved Harris Hawk optimization algorithm is used as a feature selection scheme to reduce the impact of redundant and noisy features on the performance of the classification model. The algorithm introduces the singer map to initialise the population, uses multi-information fusion to obtain the best prey position, and applies the sine function-based escape energy to execute a prey search strategy to obtain the optimal subset of features. In addition, the original data is preprocessed by the k-nearest neighbour and deep denoising autoencoder (KNN-DDAE) to relieve the imbalance problem of the network traffic data. Finally, a deep neural network (DNN) is used to complete the classification. Simulation experiments are conducted on the dataset NSL-KDD, KDD CUP99, and UNSW-NB15. The results show that our feature selection and data balancing scheme greatly improves the detection accuracy. In addition, the detection performance of this method is better than the current popular intrusion detection schemes.

From Replay to Regeneration: Recovery of UDP Flood Network Attack Scenario Based on SDN

In recent years, various network attacks have emerged. These attacks are often recorded in the form of Pcap data, which contains many attack details and characteristics that cannot be analyzed through traditional methods alone. Therefore, restoring the network attack scenario through scene reconstruction to achieve data regeneration has become an important entry point for detecting and defending against network attacks. However, current network attack scenarios mainly reproduce the attacker’s attack steps by building a sequence collection of attack scenarios, constructing an attack behavior diagram, or simply replaying the captured network traffic. These methods still have shortcomings in terms of traffic regeneration. To address this limitation, this paper proposes an SDN-based network attack scenario recovery method. By parsing Pcap data and utilizing network topology reconstruction, probability, and packet sequence models, network traffic data can be regenerated. The experimental results show that the proposed method is closer to the real network, with a higher similarity between the reconstructed and actual attack scenarios. Additionally, this method allows for adjusting the intensity of the network attack and the generated topology nodes, which helps network defenders better understand the attackers’ posture and analyze and formulate corresponding security strategies.

Monitoring Network Traffic for Suspicious Activity

An intrusion detection system investigates hostile behavior within a network or an approach. Software or a gadget called intrusion detection scans a network or system for an untrustworthy action. As computer connectivity increases, intrusion detection becomes increasingly important for network security. Many Intrusion Detection Systems have been built to defend the networks using statistical and machine learning technologies. Accuracy is a crucial factor in how well an intrusion detection system performs. To decrease false detections and boost detection rates, the accuracy of intrusion detection needs to be improved. In recent works, many strategies have been employed to enhance performance. The Intrusion detection system’s primary task is to analyze network traffic data. To solve this problem, a structured classification system is needed. This problem is approached in the suggested manner. Classification methods are often used to address related issues. NSL-KDD knowledge discovery Dataset is used to evaluate the results of these systems. This research aims to find an efficient classifier that detects anomaly traffic with a high accuracy level and minimal error rate by experimenting with possible machine-learning techniques.

Identification of encrypted and malicious network traffic based on one-dimensional convolutional neural network

The rapid advancement of the Internet has brought a exponential growth in network traffic. At present, devices deployed at edge nodes process huge amount of data, extract key features of network traffic and then forward them to the cloud server/data center. However, since the efficiency of mobile terminal devices in identifying and classifying encrypted and malicious traffic lags behind, how to identify network traffic more efficiently and accurately remains a challenging problem. We design a convolutional neural network model: One-dimensional convolutional neural network with hexadecimal data (HexCNN-1D) that combines normalized processing and attention mechanisms. By adding the attention mechanism modules Global Attention Block (GAB) and Category Attention Block (CAB), network traffic is classified and identified. By extracting effective load information from hexadecimal network traffic, our model can identify most categories of network traffic including encrypted and malicious traffic data. The experimental results show that the average accuracy is 98.8%. Our model can greatly improve the accuracy of network traffic data recognition.

Deep Learning Network Traffic Prediction based on Bayesian Algorithm Optimization

In order to solve the problem of low prediction accuracy caused by artificial determination of hyperparameters of network traffic prediction model, this paper applies Bayesian optimization algorithm (BOA) to the determination of hyperparameters of neural network, and proposes a CNN-LSTM network traffic prediction model based on Bayesian optimization algorithm. The local features and long-term development trends of the data are mined by combining the Convolutional Neural Network (CNN) and the Long Short-Term Memory Neural Network (LSTM), and the Bayesian optimization algorithm is used to calibrate the hyperparameters. This paper uses real network traffic data to test and compare with LSTM, GRU and other models. Experimental results demonstrate that the proposed BOA-CNN-LSTM model reduces the RMSE and MAE by 21.17% and 27.52%, compared to the traditional network traffic prediction model. The results presented in this paper show that the Bayesian optimization algorithm can more quickly and accurately calibrate the hyperparameters of the model, which provides a new method for predicting network traffic time series data in the future.

Implementation of Machine Learning-Based Data Mining Techniques for IDS

The internet is essential for ongoing contact in the modern world, yet its effectiveness might lessen the effect known as intrusions. Any action that negatively affects the targeted system is considered an intrusion. Network security has grown to be a major issue as a result of the Internet's rapid expansion. The Network Intrusion Detection System (IDS), which is widely used, is the primary security defensive mechanism against such hostile assaults. Data mining and machine learning technologies have been extensively employed in network intrusion detection and prevention systems to extract user behaviour patterns from network traffic data. Association rules and sequence rules are the main foundations of data mining used for intrusion detection. Given the Auto encoder algorithm's traditional method's bottleneck of frequent itemsets mining, we provide a Length-Decreasing Support to Identify Intrusion based on Data Mining, which is an upgraded Data Mining Techniques based on Machine Learning for IDS. Based on test results, it appears that the suggested strategy is successful

Generative adversarial networks for network traffic feature generation

Generative Adversarial Networks (GANs) have remained an active area of research, particularly due to their increased and advanced evolving application capabilities. In several domains such as images, facial synthesis, character generation, language processing and multimedia, they have been implemented for advanced tasks. However, there has been more limited progress in network traffic data generation due to the complexities associated with data formats and distributions. This research implements two GAN architectures that include data transforms to simultaneously train and generate categorical and continuous network traffic features. These architectures demonstrate superior performance to the original ‘Vanilla’ GAN approach, which is included as a baseline comparator. Close matches are obtained between logarithms of the means and standard deviations of the fake data and the corresponding quantities from the real data. Moreover, similar principal components are exhibited by the fake and real data streams. Furthermore, some 85% of the features from the fake data could replace those in the real data without detection.

Anomaly Detection Method for Unknown Protocols in a Power Plant ICS Network with Decision Tree

This study aimed to enhance the stability and security of power plant control network systems by developing detectable models using artificial intelligence machine learning techniques. Due to the closed system operation policy of facility manufacturers, it is challenging to detect and respond to security threats using standard security systems. With the increasing digitization of control systems, the risk of external malware penetration is also on the rise. To address this, machine learning techniques were applied to extract patterns from network traffic data produced at an average of 6.5 TB per month, and fingerprinting was used to detect unregistered terminals accessing the control network. By setting a threshold between transmission amounts and attempts using one month of data, an anomaly judgment model was learned to define patterns of data communication between the origin and destination. The hypothesis was tested using machine learning techniques if a new pattern occurred and no traffic occurred. The study confirmed that this method can be applied to not only plant control systems but also closed-structured control networks, where availability is critical, and other industries that use large amounts of traffic data. Experimental results showed that the proposed model outperformed existing models in terms of detection efficiency and processing time.

Multi-Resolution Analysis with Visualization to Determine Network Attack Patterns

Analyzing network traffic activities is imperative in network security to detect attack patterns. Due to the complex nature of network traffic event activities caused by continuously changing computing environments and software applications, identifying the patterns is one of the challenging research topics. This study focuses on analyzing the effectiveness of integrating Multi-Resolution Analysis (MRA) and visualization in identifying the attack patterns of network traffic activities. In detail, a Discrete Wavelet Transform (DWT) is utilized to extract features from network traffic data and investigate their capability of identifying attacks. For extracting features, various sliding windows and step sizes are tested. Then, visualizations are generated to help users conduct interactive visual analyses to identify abnormal network traffic events. To determine optimal solutions for generating visualizations, an extensive evaluation with multiple intrusion detection datasets has been performed. In addition, classification analysis with three different classification algorithms is managed to understand the effectiveness of using the MRA with visualization. From the study, we generated multiple visualizations associated with various window and step sizes to emphasize the effectiveness of the proposed approach in differentiating normal and attack events by forming distinctive clusters. We also found that utilizing MRA with visualization advances network intrusion detection by generating clearly separated visual clusters.

A Network Traffic Anomaly Detection Method Based on Gaussian Mixture Model

How can we learn the normal behavior of some communication processes and predict whether a single communication is under attack, with massive network traffic data representing the time costs of each stage in a single communication process? This paper introduces a statistical method for detecting network traffic anomalies using the Gaussian mixture model. There are two aspects to our contributions. First, we show how to learn the normal behavior of a communication process under the assumption that its time costs are generated from the Gaussian mixture model. Secondly, we show that with the learned Gaussian mixture model, we can predict whether a data point is under attack by computing the likelihood that the data point is drawn from the learned Gaussian distribution. The experimental results show that our method reached high accuracy in some cases, while in some other cases that are more complicated, the data point may have more factors and cannot be represented simply by only one Gaussian mixture model.

Deep learning techniques for securing cyber-physical systems in supply chain 4.0

The fourth industrial revolution's transformation utilizes a Cyber-Physical System (CPS) to secure Supply Chain 4.0. It integrates manufacturing information with internet communication technology to create a smart CPS that tracks products from manufacture to customer delivery using the Internet of Things (IoT). This research uses a Machine Learning (ML) approach for network anomaly detection and constructing data-driven models to detect DDoS attacks on Industry 4.0 CPSs. Limitations of existing techniques, such as artificial data and small datasets, are addressed by capturing network traffic data from a real-world semiconductor production factory. 45 bidirectional network flow features are extracted, and labeled datasets are constructed for training and testing ML models. The proposed PCA-BSO algorithm is employed to select the most relevant features based on their eigenvalues, as the feature with the highest eigenvalues may not always improve classification accuracy. Supervised ML algorithms are evaluated through simulations to assess their performance.

Flow-based intrusion detection on software-defined networks: a multivariate time series anomaly detection approach

In this study, we present and implement the SAnDet (SDN anomaly detector) architecture, an anomaly-based intrusion detection system designed to take advantage of the capabilities offered by software-defined networking (SDN) architecture, as a controller application. The SAnDet system is composed of three modules: statistics collection, anomaly detection, and anomaly prevention. In particular, we utilize replicator neural networks (RNN), which is a specialized variant of the autoencoder, and the LSTM-based encoder–decoder (EncDecAD) method, which is a special type of long short-term memory (LSTM) network that has demonstrated a strong performance on data series particularly, to identify unknown attacks using flow features collected from OpenFlow switches. In our experiments, we utilize flow-based features extracted from network traffic data containing various types of attacks as input to our models in the form of time series. We evaluate the performance of our methods using the accuracy and area under the receiver operating characteristic curve (AUC) metrics. Our experimental results demonstrate that EncDecAD outperforms RNN and that our approach offers several benefits over previously conducted research.

Design and Development of a Data Structure Visualisation System Using the Ant Colony Algorithm

Aim: A data structure visualisation system uses object-oriented thinking and COM technology to dynamically and interactively simulate and track data structure algorithms and realize the dynamic synchronisation and visualisation of abstract data structures and algorithms using the data modelling function and self-test function. Background: Teaching data structures and algorithms is difficult because of their abstraction and dynamics; the use of icons in classroom teaching can be partially abstracted into intuition, but analysing the instantaneous dynamic characteristics of the object and the dynamic execution process of the algorithm is difficult. Objective: A data structure visualisation system employs a data modelling function, visual data structures, a user-friendly and flexible interface, and multipath features for multilevel users. Such systems can be designed more effectively by using the ant colony algorithm. Methods: The more the ants pass by a certain path, the higher the concentration of residual pheromone, and the higher the probability of the subsequent ant selecting that path. Therefore, the individuals in an ant colony communicate messages and cooperate with each other for foraging. Results: The resulting speedup ratio indicates that the speedup is smaller when the number of nodes is 100 or more; the acceleration is higher when the node reaches a certain scale, and the speedup ratio does not change considerably. In this study, the traffic simulation software VISSIM was used to generate road network data; the generated traffic data were analysed and used to design a traffic network data structure. Conclusion: The traffic network data model oriented to the ant colony algorithm was established through abstraction. Accordingly, a parallel ant colony algorithm based on cloud computing was implemented. Finally, a Hadoop cloud computing platform was established and used to run and test the parallel ant colony algorithm program; several experiments were conducted, and the experimental results were analysed concurrently.

Video Stream Recognition Using Bitstream Shape for Mobile Network QoE.

Video streaming service delivery is a challenging task for mobile network operators. Knowing which services clients are using could help ensure a specific quality of service and manage the users' experience. Additionally, mobile network operators could apply throttle, traffic prioritization, or differentiated pricing. However, due to the growth of encrypted Internet traffic, it has become difficult for network operators to recognize the type of service used by their clients. In this article, we propose and evaluate a method for recognizing video streams solely based on the shape of the bitstream on a cellular network communication channel. To classify bitstreams, we used a convolutional neural network that was trained on a dataset of download and upload bitstreams collected by the authors. We demonstrate that our proposed method achieves an accuracy of over 90% in recognizing video streams from real-world mobile network traffic data.

Research on Key Method of Cyber Security Situation Awareness Based on ResMLP and LSTM Network

Cyber security situation awareness, has become a hotpot of research. However, the existing cyber security situation awareness methods are difficult to extract high-order features from network traffic data. In this work, we present an improved cyber security situation awareness method based on ResMLP and LSTM network from a new perspective. Our work focus on cyber attack behavior analysis, that is a key research content of cyber security situation awareness. It introduces the Residual Multi-Layer Perceptrons in deep learning into the network structure of long-short term memory. It can effectively extract the spatial and temporal characteristics of network traffic data, reduce the computational complexity, and improve the accuracy of cyber security situation awareness. Firstly, we extract the spatial features using the ResMLP network. Secondly, we extract the temporal characteristics using the LSTM network. The architecture of the ResMLP network replaces the self-noticing layer with a linear interaction layer, and this design architecture allows the model to guarantee accurate cyber attack behavior analysis performance while balancing the computational cost of the model, which can improve the detection efficiency of the model. Considering that the network data are fed into the model in the form of time series after processing, the model incorporates LSTM networks to avoid the gradient problem while better bringing up the temporal characteristics in the data.The experimental results show that the proposed method can model the future cyber security situation in a network environment more accurately than other similar methods.

Urban Area Characterization and Structure Analysis: A Combined Data-Driven Approach by Remote Sensing Information and Spatial–Temporal Wireless Data

Analysis of urban area function is crucial for urban development. Urban area function features can help to conduct better urban planning and transportation planning. With development of urbanization, urban area function becomes complex. In order to accurately extract function features, researchers have proposed multisource data mining methods that combine urban remote sensing and other data. Therefore, the research of efficient multisource data analysis tools has become a new hot topic. In this paper, a novel urban data analysis method combining spatiotemporal wireless network data and remote sensing data was proposed. First, a Voronoi-diagram-based method was used to divide the urban remote sensing images into zones. Second, we combined period and trend components of wireless network traffic data to mine urban function structure. Third, for multisource supported urban simulation, we designed a novel spatiotemporal city computing method combining graph attention network (GAT) and gated recurrent unit (GRU) to analyze spatiotemporal urban data. The final results prove that our method performs better than other commonly used methods. In addition, we calculated the commuting index of each zone by wireless network data. Combined with the urban simulation conducted in this paper, the dynamic changes of urban area features can be sensed in advance for a better sustainable urban development.