Network Traffic Analysis Research Articles

Android-IoT Malware Classification and Detection Approach Using Deep URL Features Analysis

Currently, malware attacks pose a high risk to compromise the security of Android-IoT apps. These threats have the potential to steal critical information, causing economic, social, and financial harm. Because of their constant availability on the network, Android apps are easily attacked by URL-based traffic. In this paper, an Android malware classification and detection approach using deep and broad URL feature mining is proposed. This study entails the development of a novel traffic data preprocessing and transformation method that can detect malicious apps using network traffic analysis. The encrypted URL-based traffic is mined to decrypt the transmitted data. To extract the sequenced features, the N-gram analysis method is used, and afterward, the singular value decomposition (SVD) method is utilized to reduce the features while preserving the actual semantics. The latent features are extracted using the latent semantic analysis tool. Finally, CNN-LSTM, a multi-view deep learning approach, is designed for effective malware classification and detection.

Batch classifier with adaptive update for backbone traffic classification

Network traffic analysis is an important method for ISPs to know the status of the network. However, the high speed and rapid evolution of backbone network traffic have brought new challenges to traffic analysis. Most existing traffic classifiers are based on full traffic, while processing all the large-scale backbone network traffic is time-intensive and resource-consuming. Moreover, the relationship between feature vectors and classification labels drifts with the rapid evolution of backbone network traffic, which leads to the degradation of the performance of the classification model. This paper presents a model that can implement real-time classification of sampled backbone network traffic and adaptively update the classification model when concept drift occurs. To accurately classify the backbone network traffic in real time, we propose the Multiple Counter Sketch (MC Sketch) to quickly extract feature vectors from the sampled traffic and design an Adaptive Batch Classifier based on Agglomerative Clustering (A-BCAC) to perform unsupervised batch clustering of feature vectors. Using the labels obtained from A-BCAC, we trained the classifier to classify the backbone traffic. In addition, we design an adaptive batch update model based on concept drift detection to solve the concept drift problem. The experimental results on sampled traffic collected on a 10Gbps link and the comparisons with state-of-the-art studies demonstrate the performance and efficiency of our method.

A novel approach for flow analysis in software-based networks using L-moments theory

The continuous increase in the number of devices connected to the Internet, together with the growth of applications and services, has made the tasks of network traffic analysis and classification essential in any environment. The deployment of 5G networks has prompted the research community to establish the pillars of Next-Generation Networks. These include intelligent systems, providing the network with intelligence in management and security tasks. In addition, these tasks require mechanisms capable of characterizing traffic in order to make network decisions. In this context, this paper proposes a novel methodology for processing network traffic using the L-moments theory and Machine Learning algorithms. This methodology is robust to outliers, requires few data to characterize the flows and subsequently fit the classification models. The results show that L-moments are particularly useful for processing network flows, and the classification algorithms obtain very high-quality results. Moreover, we show that the considered statistical tools also allow for a better understanding of the attack behaviour, leading the way to the improvement of the feature selection in similar problems.

Accurate Encrypted Malicious Traffic Identification via Traffic Interaction Pattern Using Graph Convolutional Network

Telecommuting and telelearning have gradually become mainstream lifestyles in the post-epidemic era. The extensive interconnection of massive terminals gives attackers more opportunities, which brings more significant challenges to network traffic security analysis. The existing attacks, often using encryption technology and distributed attack methods, increase the number and complexity of attacks. However, the traditional methods need more analysis of encrypted malicious traffic interaction patterns and cannot explore the potential correlations of interaction patterns in a macroscopic and comprehensive manner. Anyway, the changes in interaction patterns caused by attacks also need further study. Therefore, to achieve accurate and effective identification of attacks, it is essential to comprehensively describe the interaction patterns of malicious traffic and portray the relations of interaction patterns with the appearance of attacks. We propose a method for classifying attacks based on the traffic interaction attribute graph, named G-TIAG. At first, the G-TIAG studies interaction patterns of traffic describes the construction rule of the graphs and selects the attributive features of nodes in each graph. Then, it uses a convolutional graph network with a GRU and self-attention to classify benign data and different attacks. Our approach achieved the best classification results, with 89% accuracy and F1-Score, 88% recall, respectively, on publicly available datasets. The improvement is about 7% compared to traditional machine learning classification results and about 6% compared to deep learning classification results, which finally successfully achieved the classification of attacks.

Analysis of Network Traffic Measurements using Machine Learning

In recent times, an exponential increase in internet traffic has been observed as a result of advancing development of the Internet of Things, mobile networks with sensors, and communication functions within various devices. Further, the COVID-19 pandemic has inevitably led to an explosion of social network traffic. Within this context, considerable attention has been drawn to research on network traffic analysis based on machine learning.In this paper, we design and develop a new machine learning framework for network traffic analysis whereby normal and abnormal traffic is distinguished from one another. To achieve this, we combine together well-known machine learning algorithms and network traffic analysis techniques. Using one of the most widely used datasets KDD CUP'99 in the Weka and Apache Spark environments, we compare and investigate results obtained from time series type analysis of various aspects including malicious codes, feature extraction, data formalization, network traffic measurement tool implementation. Experimental analysis showed that while both the logistic regression and the support vector machine algorithm were excellent for performance evaluation, among these, the logistic regression algorithm performs better. The quantitative analysis results of our proposed machine learning framework show that this approach is reliable and practical. In addition, we determined that the framework developed in the Apache Spark environment exhibits a much faster processing speed in the Spark environment than in Weka as there are more datasets used to create and classify machine learning models

Design and Analysis of a Network Traffic Analysis Tool: NetFlow Analyzer

A network analyzer can often comprehend many protocols, which enables it to display talks taking place between hosts over a network. A network analyzer analyzes the device or network response and measures for the operator to keep an eye on the network’s or object’s performance in an RF circuit. The purpose of the following research includes analyzing the capabilities of NetFlow analyzer to measure various parts, including filters, mixers, frequency sensitive networks, transistors, and other RF-based instruments. NetFlow Analyzer is a network traffic analyzer that measures the network parameters of electrical networks. Although there are other types of network parameter sets including Y, Z, & H-parameters, these instruments are typically employed to measure S-parameters since transmission & reflection of electrical networks are simple to calculate at high frequencies. These analyzers are widely employed to distinguish between two-port networks, including filters and amplifiers. By allowing the user to view the actual data that is sent over a network, packet by packet, a network analyzer informs you of what is happening there. Also, this research will contain the design model of NetFlow Analyzer that Measurements involving transmission and reflection use. Gain, insertion loss, and transmission coefficient are measured in transmission measurements, whereas return loss, reflection coefficient, impedance, and other variables are measured in reflection measurements. These analyzers’ operational frequencies vary from 1 Hz to 1.5 THz. These analyzers can also be used to examine stability in measurements of open loops, audio components, and ultrasonics.

Balancing Information Preservation and Data Volume Reduction: Adaptive Flow Aggregation in Flow Metering Systems

The critical role of network traffic measurement and analysis extends across a range of network operations, ensuring quality of service, security, and efficient resource management. Despite the ubiquity of flow-level measurement, the escalating size of flow entries presents significant scalability issues. This study explores the implications of adaptive gradual flow aggre- gation, a solution devised to mitigate these challenges, on flow information distortion. The investigation maintains flow records in buffers of varying aggregation levels, iteratively adjusted based on the changing traffic load mirrored in CPU and memory utilization. Findings underscore the efficiency of adaptive gradual flow aggregation, particularly when applied to a specific buffer, yielding an optimal balance between information preservation and memory utilization. The paper highlights the particular significance of this approach in Internet of Things (IoT) and contrasted environments, characterized by stringent resource constraints. Consequently, it casts light on the imperative for adaptability in flow aggregation methods, the impact of these techniques on information distortion, and their influence on network operations. This research offers a foundation for future studies targeting the development of more adaptive and effective flow measurement techniques in diverse and resource-limited network environments.

Вызовы в реализации систем глубокого анализа сетевого трафика методом полного протокольного декодирования

This paper presents a summary of experience in developing the deep packet inspection system using full protocol decoding. The paper reviews the challenges encountered during implementation and provides a high-level overview of the solutions to these issues. The challenges can be grouped into two groups. The first group is related to the fundamental tasks which must be addressed when implementing full protocol decoding systems. This includes ensuring correct protocol parsing, which involves identifying and interpreting protocol headers and fields correctly. Moreover, it is necessary to ensure the processing of fragmented packets and the assembly of fragments into the original message. Additionally, the processing and analysis of encrypted traffic is a crucial task that may require the use of specialized algorithms and tools. The second group of problems is related to optimizing the process of full protocol decoding to ensure high-speed traffic processing, as well as supporting new protocols and the ability to add user-defined extensions. While there are open-source systems that address some of the primary issues associated with full protocol decoding, there may be a need for additional effort and specialized solutions to efficiently operate and expand the functionality of such systems. Although implementing deep network traffic analysis tools using full protocol decoding requires the use of advanced hardware and software technologies, the benefits of such analysis are significant. This approach provides a more complete understanding of network traffic patterns and enables more effective detection and prevention of cyber-attacks. It also allows for more accurate monitoring of network performance and the identification of potential bottlenecks or other issues that may impact network efficiency. In this article, we also emphasize the importance of system architecture development and implementation to ensure the successful deployment of deep network traffic analysis tools using full protocol decoding. At last, we conducted an experiment where several advanced optimizations were implemented in the system that had already solved primary issues. These optimizations related to working with memory, based on the features of the traffic processing scheme. By results, we evaluated significant performance improvement in solving secondary tasks, described in this work.

ANALYSIS OF NETWORK TRAFFIC THREATS ACROSS OSI MODEL LAYERS FOR DYNAMIC RTO CALCULATION IN THE CONTEXT OF COMBATING DDoS ATTACKS

This document provides an examination of current threats to network security, viewed through the lens of network traffic analysis at various OSI model layers. It delves into the different forms of Distributed Denial of Service (DDoS) attacks and their ramifications on the Transmission Control Protocol (TCP), with a specific focus on a critical parameter - the Retransmission Timeout (RTO). The text also divulges fundamental algorithms and techniques for calculating RTO, encompassing adaptive methodologies that harness machine learning and artificial intelligence for optimizing the TCP/IP stack. In particular, it offers insights into the functioning of the RTO calculation algorithm, a pivotal element ensuring the reliability of data transmission via TCP. The document elaborates on how this algorithm dynamically adjusts the RTO value based on network conditions and measured Round Trip Time (RTT) values. Furthermore, it furnishes formulas for computing RTO with diverse parameters. Moreover, the document explores the potential of employing machine learning and data analysis methodologies to detect and preempt DDoS attacks. It elucidates how contemporary technologies empower the use of these approaches to minimize false positives in identifying malicious traffic packets, thereby enhancing the effectiveness of safeguarding information systems. Additionally, it provides an illustration of software and hardware tools employed for the practical implementation of these algorithms in devices facilitating data transmission via Ethernet connections. In summary, this work offers insights into contemporary challenges and issues in the realm of network security, especially in the context of the escalating frequency of DDoS attacks. This information proves valuable for students and professionals engaged in the study of network security and the development of measures to fortify networks and systems.

Dark-Forest: Analysis on the Behavior of Dark Web Traffic via DeepForest and PSO Algorithm

The dark web is a shadow area hidden in the depths of the Internet, which is difficult to access through common search engines. Because of its anonymity, the dark web has gradually become a hotbed for a variety of cyber-crimes. Although some research based on machine learning or deep learning has been shown to be effective in the task of analyzing dark web traffic in recent years, there are still pain points such as low accuracy, insufficient real-time performance, and limited application scenarios. Aiming at the difficulties faced by the existing automated dark web traffic analysis methods, a novel method named Dark-Forest to analyze the behavior of dark web traffic is proposed. In this method, firstly, particle swarm optimization algorithm is used to filter the redundant features of dark web traffic data, which can effectively shorten the training and inference time of the model to meet the real-time requirements of dark web detection task. Then, the selected features of traffic are analyzed and classified using the DeepForest model as a backbone classifier. The comparison experiment with the current mainstream methods shows that Dark-Forest takes into account the advantages of statistical machine learning and deep learning, and achieves an accuracy rate of 87.84%. This method not only outperforms baseline methods such as Random Forest, MLP, CNN, and the original DeepForest in both large-scale and small-scale dataset based learning tasks, but also can detect normal network traffic, tunnel network traffic and anonymous network traffic, which may close the gap between different network traffic analysis tasks. Thus, it has a wider application scenario and higher practical value.

Machine Learning-Powered Encrypted Network Traffic Analysis: A Comprehensive Survey

Traffic analysis is the process of monitoring network activities, discovering specific patterns, and gleaning valuable information from network traffic. It can be applied in various fields such as network assert probing and anomaly detection. With the advent of network traffic encryption, however, traffic analysis becomes an arduous task. Due to the invisibility of packet payload, traditional traffic analysis methods relying on capturing valuable information from plaintext payload are likely to lose efficacy. Machine learning has been emerging as a powerful tool to extract informative features without getting access to payload, and thus is widely employed in encrypted traffic analysis. In this paper, we present a comprehensive survey on recent achievements in machine learning-powered encrypted traffic analysis. To begin with, we review the literature in this area and summarize the analysis goals that serve as the basis for literature classification. Then, we abstract the workflow of encrypted traffic analysis with machine learning tools, including traffic collection, traffic representation, traffic analysis method, and performance evaluation. For the surveyed studies, the requirements of classification granularity and information timeliness may vary a lot for different analysis goals. Hence, in terms of the goal of traffic analysis, we present a comprehensive review on existing studies according to four categories: network asset identification, network characterization, privacy leakage detection, and anomaly detection. Finally, we discuss the challenges and directions for future research on encrypted traffic analysis.

Android Adware Detection using Soot and CFG

Adware is the most common type of malware. While considered not harmful in nature, it disrupts the user experience and generates unwanted revenue. Adware is also difficult to analyze and detect, therefore it is actively distributed through the Google Play Store. In the case of known adware detection, analysis takes a long time due to prolonged dynamic analysis to reveal Ads, and the result of network traffic analysis is difficult to determine traffic such as multimedia streaming, which results in a high probability of false positives. In this paper, we introduce a method to efficiently detect adware using static analysis. CFG(Control Flow Graph) analysis was performed using Soot to identify the adware signatures. 52 signatures were obtained through analysis of 1000 samples, and the optimized detection efficiency was measured while changing the depth (level) of the CFG. In addition, it achieves 91.92% of accuracy when analyzing 1380 normal and 4490 adware samples. This approach has similar detection rate in a shorter time compared to the existing dynamic analysis, and it does not require operation check as in previous dynamic analysis approaches.

Internet Threat Detection in Smart Grids Based on Network Traffic Analysis Using LSTM, IF, and SVM

The protection of users of ICT networks, including smart grids, is a challenge whose importance is constantly growing. Internet of Things (IoT) or Internet of Energy (IoE) devices, as well as network resources, store more and more information about users. Large institutions use extensive security systems requiring large and expensive resources. For smart grid users, this becomes difficult. Efficient methods are needed to take advantage of limited sets of traffic features. In this paper, machine learning techniques to verify network events for recognition of Internet threats were analyzed, intentionally using a limited number of parameters. The authors considered three machine learning techniques: Long Short-Term Memory, Isolation Forest, and Support Vector Machine. The analysis is based on two datasets. In the paper, the data preparation process is also described. Eight series of results were collected and compared with other studies. The results showed significant differences between the techniques, the size of the datasets, and the balance of the datasets. We also showed that a more accurate classification could be achieved by increasing the number of analyzed features. Unfortunately, each increase in the number of elements requires more extensive analysis. The work ends with a description of the steps that can be taken in the future to improve the operation of the models and enable the implementation of the described methods of analysis in practice.

Interactive Web-Based Visual Analysis on Network Traffic Data

Network traffic data analysis is important for securing our computing environment and data. However, analyzing network traffic data requires tremendous effort because of the complexity of continuously changing network traffic patterns. To assist the user in better understanding and analyzing the network traffic data, an interactive web-based visualization system is designed using multiple coordinated views, supporting a rich set of user interactions. For advancing the capability of analyzing network traffic data, feature extraction is considered along with uncertainty quantification to help the user make precise analyses. The system allows the user to perform a continuous visual analysis by requesting incrementally new subsets of data with updated visual representation. Case studies have been performed to determine the effectiveness of the system. The results from the case studies support that the system is well designed to understand network traffic data by identifying abnormal network traffic patterns.

D-Score: An expert-based method for assessing the detectability of IoT-related cyber-attacks

IoT devices are known to be vulnerable to various cyber-attacks, such as data exfiltration and the execution of flooding attacks as part of a DDoS attack. When it comes to detecting such attacks using network traffic analysis, it has been shown that some attack scenarios are not always equally easy to detect if they involve different IoT models. That is, when targeted at some IoT models, a given attack can be detected rather accurately, while when targeted at others the same attack may result in too many false alarms. In this research, we attempt to explain this variability of IoT attack detectability and devise a risk assessment method capable of addressing a key question: how easy is it for an anomaly-based network intrusion detection system to detect a given cyber-attack involving a specific IoT model? In the process of addressing this question we (a) investigate the predictability of IoT network traffic, (b) present a novel taxonomy for IoT attack detection which also encapsulates traffic predictability aspects, (c) propose an expert-based attack detectability estimation method which uses this taxonomy to derive a detectability score (termed ‘D-Score’) for a given combination of IoT model and attack scenario, and (d) empirically evaluate our method while comparing it with a data-driven method.

Defending against adversarial machine learning attacks using hierarchical learning: A case study on network traffic attack classification

Machine learning is key for automated detection of malicious network activity to ensure that computer networks and organizations are protected against cyber security attacks. Recently, there has been growing interest in the domain of adversarial machine learning, which explores how a machine learning model can be compromised by an adversary, resulting in misclassified output. Whilst to date, most focus has been given to visual domains, the challenge is present in all applications of machine learning where a malicious attacker would want to cause unintended functionality, including cyber security and network traffic analysis. We first present a study on conducting adversarial attacks against a well-trained network traffic classification model. We show how well-crafted adversarial examples can be constructed so that known attack types are misclassified by the model as benign activity. To combat this, we present a novel defensive strategy based on hierarchical learning to help reduce the attack surface that an adversarial example can exploit within the constraints of the parameter space of the intended attack. Our results show that our defensive learning model can withstand crafted adversarial attacks and can achieve classification accuracy in line with our original model when not under attack.

Smart Approach for Botnet Detection Based on Network Traffic Analysis

Today, botnets are the most common threat on the Internet and are used as the main attack vector against individuals and businesses. Cybercriminals have exploited botnets for many illegal activities, including click fraud, DDOS attacks, and spam production. In this article, we suggest a method for identifying the behavior of data traffic using machine learning classifiers including genetic algorithm to detect botnet activities. By categorizing behavior based on time slots, we investigate the viability of detecting botnet behavior without seeing a whole network data flow. We also evaluate the efficacy of two well-known classification methods with reference to this data. We demonstrate experimentally, using existing datasets, that it is possible to detect botnet activities with high precision.

K-Means clustering-based semi-supervised for DDoS attacks classification

Network attacks of the distributed denial of service (DDoS) form are used to disrupt server replies and services. It is popular because it is easy to set up and challenging to detect. We can identify DDoS attacks on network traffic in a variety of ways. However, the most effective methods for detecting and identifying a DDoS attack are machine learning approaches. This attack is considered to be among the most dangerous internet threats. In order for supervised machine learning algorithms to function, there needs to be tagged network traffic data sets. On the other hand, an unsupervised method uses network traffic analysis to find assaults. In this research, the K-Means clustering algorithm was developed as a semi-supervised approach for DDoS classification. The proposed algorithm is trained and tested with the CICIDS2017 dataset. After using the proposed hybrid feature selection methods and applying multiple training, testing, and carefully sorting DDoS traffic through a series of experiments, the optimum 2 centroids were found to be DDoS and normal. The generated centroids can be used to classify network traffic. So the proposed method succeeded to cluster the network traffic to safe and theat.

BART-based inference for Poisson processes

The effectiveness of Bayesian Additive Regression Trees (BART) has been demonstrated in a variety of contexts including non-parametric regression and classification. A BART scheme for estimating the intensity of inhomogeneous Poisson processes is introduced. Poisson intensity estimation is a vital task in various applications including medical imaging, astrophysics and network traffic analysis. The new approach enables full posterior inference of the intensity in a non-parametric regression setting. The performance of the novel scheme is demonstrated through simulation studies on synthetic and real datasets up to five dimensions, and the new scheme is compared with alternative approaches.

Fine-Grained High-Utility Dynamic Fingerprinting Extraction for Network Traffic Analysis

Previous network feature extraction methods used for network anomaly detection have some problems, such as being unable to extract features from the original network traffic, or that they can only extract coarse-grained features, as well as that they are highly dependent on manual analysis. To solve these problems, this paper proposes a fine-grained and highly practical dynamic application fingerprint extraction method. By putting forward a fine-grained high-utility dynamic fingerprinting (Huf) algorithm to build a Huf-Tree based on the N-gram (every substring of a larger string, of a fixed length n) model, combining it with the network traffic segment-IP address transition (IAT) method to achieve dynamic application fingerprint extraction, and through the utility of fingerprint, the calculation was performed to obtain a more valuable fingerprint, to achieve fine-grained and efficient flow characteristic extraction, and to solve the problem of this method being highly dependent on manual analysis. The experimental results show that the Huf algorithm can realize the dynamic application of fingerprint extraction and solve the existing problems.