Detecting Network Traffic Anomalies Research Articles

Performance Analysis of Random Forest Algorithm for Network Anomaly Detection using Feature Selection

As the volume and complexity of computer network traffic continue to increase, network administrators face a growing challenge in monitoring and discovering unusual activity. To keep the network safe and functioning, detecting anomalies is essential. Machine learning-based anomaly detection techniques have become increasingly popular in recent years. This is due to the fact that conventional anomaly detection methods make it difficult to detect unknown and complex attacks. This research aims to conduct a performance analysis of two feature selection methods using the random forest algorithm using the UNSW-NB15 dataset to determine which model is most effective in detecting network traffic anomalies. The models evaluated were random forest with the filter method and random forest with the wrapper method. A number of metrics used for model performance assessment are accuracy, F1-score, receiver operating characteristic curve, and precision-recall. Dataset collection, data pre-processing, feature selection, model construction, and evaluation are the main components of the research methodology. The research results show that the Random Forest approach with the Filter method has an accuracy of 0.8950, F1-score of 0.8333, ROC score of 0.8928, and a precision-recall value of 0.8347. Meanwhile, the approach using the Wrapper method obtained an accuracy of 0.9151, F1-score of 0.8510, ROC score of 0.9136, and a precision-recall value of 0.8637. This shows that the performance of Random Forest with the Wrapper method is superior in all assessment metrics. Random Forest with the Wrapper Method is the right choice of model for detecting network traffic anomalies because of its stable performance and ability to handle complex patterns

Real-Time Detection of Network Traffic Anomalies in Big Data Environments Using Deep Learning Models

Abstract: In light of the increasing sophistication of cyberattacks and the rapid growth in network traffic, it is essential to detect network traffic anomalies or intrusions as they occur. Manual inspection is inefficient due to the large volume, speed, and variety network traffic data. This paper suggests using deep learning techniques in order to build intelligent models which can detect network traffic anomalies automatically within big data environments. We present a framework for anomaly detection using long-short-term memory models (LSTM) and convolutional neural network (CNN). The models are based on data extracted from packet captures. The models are evaluated on benchmark intrusion datasets as well as a large scale real network traffic dataset. The results show that deep learning models are able to detect anomalies more effectively than traditional shallow learning methods. Models can handle high-volume streaming data with low latency and in real time. To improve detection efficiency, we also propose optimization methods such as model compression and transfer learning. This work shows the effectiveness of deep learning for real-time anomaly detection within big data environments

Detecting Brute Force Attacks on SSH and FTP Protocol Using Machine Learning: A Survey

The significance of detecting network traffic anomalies in cybersecurity cannot be overstated, especially given the increasing frequency and complexity of computer network attacks. As new Internet-related technologies emerge, so do more intricate attacks. One particularly daunting challenge is represented by dictionary-based brute-force attacks, which require effective real- time detection and mitigation methods. In this paper, we investigate Secure Shell or Secure Socket Shell, is a network protocol that gives users, particularly system administrators, a secure way to access a computer over an unsecured network(SSH)and File Transfer Protocol is a standard network protocol used for the transfer of files from one host to another over a TCP-based network, such as the Internet (FTP) brute-force attack detection by using Our research focuses on using the machine learning approach to detect SSH and FTP brute-force attacks. A reasonably thorough investigation of machine learners' efficacy in identifying brute force assaults on SSH and FTP is made possible by employing several classifiers. Brute-force assaults are a popular and risky method of obtaining usernames and passwords. Applying ethical hacking is an excellent technique to examine the effects of a brute-force assault. This article discusses many defense strategies and approaches to using brute-force assaults. The pros and cons of several defense strategies are enumerated, along with information on which kind of assault is easiest to identify. we made use of machine learning classifiers: Naive Bayes, Random Forest, Logistic Regression, we determined that the Random Forest algorithm achieved the highest level with an accuracy the contribution lies in demonstrating the feasibility of training and evaluating basic Random Forest models with two independent variables to classify CSE-CIC-IDS2018 dataset.

A Technique for Creating and Training an Artificial Neural Network to Detect Network Traffic Anomalies

The article presents a technique for creating and training an artificial neural network to recognize network traffic anomalies using relatively small samples of collected data to generate training data. Various data sources for machine learning and approaches to network traffic analysis are considered. There are data format and the method of generating them from the collected network traffic is described, as well as the steps of the methodology in detail. Using the technique, an artificial neural network was created and trained for the task of recognizing anomalies in the network traffic of the ICMP protocol. The results of testing and comparing various artificial neural network configurations and learning conditions for a given task are presented. The artificial neural network trained according to the method was tested on real network traffic. The presented technique can be applied without requiring changes to detect anomalies of various network protocols and network traffic using a suitable parameterizer and data markup.

Analysis of Machine Learning Application in Campus Network Traffic Anomaly Detection

Abstract In this paper, machine learning algorithms are first utilized to extract features of campus network traffic, and then the multi-attention mechanism is introduced to fuse the massive features extracted at different scales. Unsupervised learning is used to propose a method for detecting network traffic anomalies, and simulation experiments are conducted to verify the model’s performance. The results show that the detection rates of machine learning algorithms are all above 80%, the false alarm rate basically stays below 10%. The machine algorithms have higher accuracy than other algorithms in network data flow anomaly detection. This study has important reference value for campus network security research and verifies the important role of machine learning algorithms in detecting anomalies in campus network traffic.

A Network Traffic Abnormal Detection Method: Sketch-Based Profile Evolution

Network anomaly detection faces unique challenges from dynamic traffic, including large data volume, few attributes, and human factors that influence it, making it difficult to identify typical behavioral characteristics. To address this, we propose using Sketch-based Profile Evolution (SPE) to detect network traffic anomalies. Firstly, the Traffic Graph (TG) of the network terminal is generated using Sketch to identify abnormal data flow positions. Next, the Convolutional Neural Network and Long Short-Term Memory Network (CNN-LSTM) are used to develop traffic behavior profiles, which are then continuously updated using Evolution to detect behavior pattern changes in real-time data streams. SPE allows for direct processing of raw traffic datasets and continuous detection of constantly updated data streams. In experiments using real network traffic datasets, the SPE algorithm was found to be far more efficient and accurate than PCA and Basic Evolution for outlier detection. It is important to note that the value of φ can affect the results of anomaly detection.

BAE: Anomaly Detection Algorithm Based on Clustering and Autoencoder

In this paper, we propose an outlier-detection algorithm for detecting network traffic anomalies based on a clustering algorithm and an autoencoder model. The BIRCH clustering algorithm is employed as the pre-algorithm of the autoencoder to pre-classify datasets with complex data distribution characteristics, while the autoencoder model is used to detect outliers based on a threshold. The proposed BIRCH-Autoencoder (BAE) algorithm has been tested on four network security datasets, KDDCUP99, UNSW-NB15, CICIDS2017, and NSL-KDD, and compared with representative algorithms. The BAE algorithm achieved average F-scores of 96.160, 81.132, and 91.424 on the KDDCUP99, UNSW-NB15, and CICIDS2017 datasets, respectively. These experimental results demonstrate that the proposed approach can effectively and accurately detect anomalous data.

A Deep Learning Approach for Network Intrusion Detection Using a Small Features Vector

With the growth in network usage, there has been a corresponding growth in the nefarious exploitation of this technology. A wide array of techniques is now available that can be used to deal with cyberattacks, and one of them is network intrusion detection. Artificial Intelligence (AI) and Machine Learning (ML) techniques have extensively been employed to identify network anomalies. This paper provides an effective technique to evaluate the classification performance of a deep-learning-based Feedforward Neural Network (FFNN) classifier. A small feature vector is used to detect network traffic anomalies in the UNSW-NB15 and NSL-KDD datasets. The results show that a large feature set can have redundant and unuseful features, and it requires high computation power. The proposed technique exploits a small feature vector and achieves better classification accuracy.

A Network Traffic Anomaly Detection Method Based on Gaussian Mixture Model

How can we learn the normal behavior of some communication processes and predict whether a single communication is under attack, with massive network traffic data representing the time costs of each stage in a single communication process? This paper introduces a statistical method for detecting network traffic anomalies using the Gaussian mixture model. There are two aspects to our contributions. First, we show how to learn the normal behavior of a communication process under the assumption that its time costs are generated from the Gaussian mixture model. Secondly, we show that with the learned Gaussian mixture model, we can predict whether a data point is under attack by computing the likelihood that the data point is drawn from the learned Gaussian distribution. The experimental results show that our method reached high accuracy in some cases, while in some other cases that are more complicated, the data point may have more factors and cannot be represented simply by only one Gaussian mixture model.

Влияние ранжирования индикаторов атак на качество моделей машинного обучения в агентных системах непрерывной аутентификации

Security agents of authentication systems function in automatic mode and control the behavior of subjects, analyzing their dynamics using both traditional (statistical) methods and methods based on machine learning. The expansion of the cybersecurity fabric paradigm actualizes the improvement of adaptive explicable methods and machine learning models. Purpose: the purpose of the study was to assess the impact of ranking methods at compromise indicators, attacks indicators and other futures on the quality of detecting network traffic anomalies as part of the security fabric with continuous authentication. Probabilistic and explicable methods of binary classification were used, as well as nonlinear regressors based on decision trees. The results of the study showed that the methods of pre liminary ranking increase the F1-Score and functioning speed for supervised ML-models by an average of 7%. In unsupervised models, preliminary ranking does not significantly affect the training time, but increases the by 2-10%, which justifies their expediency in agent based systems of continuous authentication. Practical relevance: the models developed in the work substantiate the feasibility of mechanisms for preliminary ranking of compromise and attacks indicators, creating patterns prototypes of attack indicators in automatic mode. In general, uncontrolled models are not as accurate as controlled ones, which actualizes the improvement of either explicable uncontrolled approaches to detecting anomalies, or approaches based on methods with reinforcement.

Development of a model for detecting network traffic anomalies in distributed wireless ad hoc networks

Mobile ad hoc networks are one of the promising directions of the edge computing technology and they are used in various applications, in particular, in the development of intelligent transport systems. A feature of mobile ad hoc networks lies in the constantly changing dynamic network topology, as a result of which it is necessary to use reactive routing protocols when transmitting packets between nodes. Mobile ad hoc networks are vulnerable to cyber-attacks, so there is a need to develop measures to identify network threats and develop rules for responding to them based on machine learning models. The subject of this study is the development of a dynamic model for detecting network traffic anomalies in wireless distributed ad hoc networks. Within the framework of this study, methods and algorithms of data mining and machine learning were applied. The proposed approach to traffic monitoring in wireless distributed ad hoc networks consists in the implementation of two stages: initial traffic analysis to identify anomalous events and subsequent in-depth study of cybersecurity incidents to classify the type of attack. Within the framework of this approach, the corresponding models are constructed based on ensemble methods of machine learning. A comparative analysis and selection of the most efficient machine learning algorithms and their optimal hyperparameters has been carried out. In this paper, a formalization of the traffic anomaly detection model in distributed wireless ad hoc networks is carried out, the main quantitative metrics of network performance are identified, a generalized algorithm for detecting traffic anomalies in mobile ad hoc networks is presented, and an experimental study of the network segment simulation is carried out from the point of view of performance degradation during the implementation of various network attack scenarios. Network distributed denial of service attacks and cooperative blackhole attacks have the greatest negative impact on the performance of the mobile ad hoc network segment. In addition, the network simulation results were used to build a machine learning model to detect anomalies and classify types of attacks. The results of a comparative analysis of machine learning algorithms showed that the use of the LightGBM method is the most effective for detecting network traffic anomalies with an accuracy of 91 %, and for determining directly the type of attack being carried out with an accuracy of 90 %. The proposed approach for network anomalies detection through the use of trained traffic analysis models makes it possible to identify the considered types of attacks in due time. The future development direction of this research is the consideration of new scenarios for the emergence of network attacks and online additional training of the constructed identification models. The developed software tool for detecting network traffic anomalies in distributed mobile ad hoc networks can be used for any type of wireless ad hoc networks.

FamilyGuard: A Security Architecture for Anomaly Detection in Home Networks.

The residential environment is constantly evolving technologically. With this evolution, sensors have become intelligent interconnecting home appliances, personal computers, and mobile devices. Despite the benefits of this interaction, these devices are also prone to security threats and vulnerabilities. Ensuring the security of smart homes is challenging due to the heterogeneity of applications and protocols involved in this environment. This work proposes the FamilyGuard architecture to add a new layer of security and simplify management of the home environment by detecting network traffic anomalies. Experiments are carried out to validate the main components of the architecture. An anomaly detection module is also developed by using machine learning through one-class classifiers based on the network flow. The results show that the proposed solution can offer smart home users additional and personalized security features using low-cost devices.

Anomaly Detection Using Hybrid Neuro Genetic Model

With a steady increase in the population of Internet users, a plethora of network services have emerged on the global level. As an offshoot of this phenomenal rise in network services and their capabilities riding on the wave of internet, we are witnessing a massive risk of attacks on network security. Many security vulnerabilities are exposed and exploited by attackers, endangering the safety of massive amounts of data. To improve a network’s effectiveness, it’s critical to detect network traffic anomalies accurately and quickly. A new hybrid model that effectively detects anomalies in network services is proposed in this work. The genetic phase and NN phase represent the 2-phased approach making each one dependent on the other for weight assignment and prediction. The genetic phase generates optimal weights for classification of normal and anomaly patterns. The NN phase learns the input output relationship of network patterns employing GA in the training phase. Detection is accomplished using trained NN and it utilizes pre-processed KDD dataset containing normal and abnormal samples for training. The outcomes demonstrated that the suggested approach outperforms all other algorithms.

Adversarial Machine Learning: A Multilayer Review of the State-of-the-Art and Challenges for Wireless and Mobile Systems

Machine Learning (ML) models are susceptible to adversarial samples that appear as normal samples but have some imperceptible noise added to them with the intention of misleading a trained classifier and misclassifying the input. Adversarial Machine Learning (AML) was initially coined following upon researchers pointing out certain blind spots in image classifiers in computer vision field which were exploited by these adversarial samples to deceive the model. Although this has been investigated remarkably in computer vision, the impact of AML in wireless and mobile systems has recently attracted attention. Wireless and mobile networks have intensely benefited from the application of ML classifiers to detect network traffic anomalies and malware detection. However, ML detectors themselves can be exfiltrated/evaded by the samples carefully designed by attackers, raising security concerns for ML-based network applications. Thus, it is crucial to detect such samples to safeguard the network. This survey article presents a systematic mapping and a comprehensive literature review on AML to wireless and mobile systems from physical layer to network and application layers. The article reviews the state-of-the-art AML approaches in the generation and detection of adversarial samples. The samples can be generated by adversarial models such as Generative Adversarial Networks (GANS) and techniques such as Fast Gradient Sign Method (FGSM). The samples can be detected by adversarial models acting as classifiers or ML classifiers reinforced with knowledge on how to detect such samples. For each approach, a high-level overview is provided alongside its impact on solving the problems in wireless and mobile settings. Furthermore, this article provides detailed discussions to highlight the open issues and challenges faced by these approaches, as well as research opportunities which can be of interest to the researchers and developers in Artificial Intelligence (AI)-driven wireless and mobile networking.

Network Traffic Anomaly Detection Method Based on CAE and LSTM

This paper constructs a deep learning method for detecting network traffic anomalies to enhance the secure transmission of data in networks due to the complex, diverse and numerous types of anomalous traffic in current networks. The method combines multiple convolutional auto-encoders (Multi-CAE) with a long short-term memory network. The convolutional auto-encoders are obtained by combining stacked auto-encoders with convolutional layers, which can not only reduce feature loss but also effectively extract the spatial structure of samples. The use of Multi-CAE greatly improves the feature extraction capability, and combined with the long short-term memory network to extract temporal features, the effective features extracted in this paper are more comprehensive and less losses compared to the models used in other researches. A comparison of the loss values in the training of CAE (Convolutional Auto-Encoders) and SAE (Stacked Auto-Encoders) in the experiments shows that the loss values of CAE are about one-tenth lower than those of SAE, and the method consisting of Multi-CAE and LSTM for the USTC- TFC2016 dataset was trained with accuracy values up to 99.98%, and the precision, recall and f1-score parameters were also above 99%, outperforming other studies.

Apply machine learning techniques to detect malicious network traffic in cloud computing

Computer networks target several kinds of attacks every hour and day; they evolved to make significant risks. They pass new attacks and trends; these attacks target every open port available on the network. Several tools are designed for this purpose, such as mapping networks and vulnerabilities scanning. Recently, machine learning (ML) is a widespread technique offered to feed the Intrusion Detection System (IDS) to detect malicious network traffic. The core of ML models’ detection efficiency relies on the dataset’s quality to train the model. This research proposes a detection framework with an ML model for feeding IDS to detect network traffic anomalies. This detection model uses a dataset constructed from malicious and normal traffic. This research’s significant challenges are the extracted features used to train the ML model about various attacks to distinguish whether it is an anomaly or regular traffic. The dataset ISOT-CID network traffic part uses for the training ML model. We added some significant column features, and we approved that feature supports the ML model in the training phase. The ISOT-CID dataset traffic part contains two types of features, the first extracted from network traffic flow, and the others computed in specific interval time. We also presented a novel column feature added to the dataset and approved that it increases the detection quality. This feature is depending on the rambling packet payload length in the traffic flow. Our presented results and experiment produced by this research are significant and encourage other researchers and us to expand the work as future work.

Study of Machine Learning for Cloud Assisted IoT Security as a Service.

Machine learning (ML) has been emerging as a viable solution for intrusion detection systems (IDS) to secure IoT devices against different types of attacks. ML based IDS (ML-IDS) normally detect network traffic anomalies caused by known attacks as well as newly introduced attacks. Recent research focuses on the functionality metrics of ML techniques, depicting their prediction effectiveness, but overlooked their operational requirements. ML techniques are resource-demanding that require careful adaptation to fit the limited computing resources of a large sector of their operational platform, namely, embedded systems. In this paper, we propose cloud-based service architecture for managing ML models that best fit different IoT device operational configurations for security. An IoT device may benefit from such a service by offloading to the cloud heavy-weight activities such as feature selection, model building, training, and validation, thus reducing its IDS maintenance workload at the IoT device and get the security model back from the cloud as a service.

МЕТОД ВЫЯВЛЕНИЯ АНОМАЛИЙ СЕТЕВОГО ТРАФИКА, ОСНОВАННЫЙ НА ЕГО САМОПОДОБНОЙ СТРУКТУРЕ

The paper presents a method for detecting network traffic anomalies taking into account its self-similar structure. It is assumed that network traffic is a self-similar structure and is modeled by fractal Brownian motion. Existing methods of detecting network anomalies are studied. The result of scientific work is a new method for detecting network traffic anomalies. This method is based on a semi-controlled method of anomaly detection, which allows the process to be almost autonomous from human intervention. In addition, the method can be classified as a group of statistical methods, which makes it quite easy to implement. In contrast to the existing methods, this method uses samples of optimal volumes obtained in the minimum but sufficient time. This anomaly detection algorithm consists of two parts: calculation of samples (reference values) and comparison of the received traffic with the standard (analysis of network traffic anomalies). The calculation of standards is based on the calculation of the values of the self-similarity parameter (Hurst parameter) for some indicators from the package headers. The algorithm of anomaly search underlying the method can be used both to search for incoming anomalies (network attacks) and to search for anomalies in outgoing traffic (DLP-system).

A New Method of Detecting Network Traffic Anomalies

As Internet communications and applications become more and more complex, accurately describing network traffic information and rapidly monitoring network traffic anomalies have become increasingly challenging tasks. In this paper, we present a framework and method for monitoring network traffic through measuring the dynamic changes of host communities. An unweighted and undirected host interaction network (HIN) is established through extracting the social-behavioral characteristics of network traffic. Based on social-behavior similarity in HIN, host community is defined, and then five features are proposed to capture host community changes. Finally, this method is evaluated through two real-world network traffic, and the experimental results show that the method presented in this paper can effectively capture the dynamic changes of host communities to monitor network traffic.