Flow-based Intrusion Detection System Research Articles

High-Speed Network DDoS Attack Detection: A Survey.

Having a large number of device connections provides attackers with multiple ways to attack a network. This situation can lead to distributed denial-of-service (DDoS) attacks, which can cause fiscal harm and corrupt data. Thus, irregularity detection in traffic data is crucial in detecting malicious behavior in a network, which is essential for network security and the integrity of modern Cyber-Physical Systems (CPS). Nevertheless, studies have shown that current techniques are ineffective at detecting DDoS attacks on networks, especially in the case of high-speed networks (HSN), as detecting attacks on the latter is very complex due to their fast packet processing. This review aims to study and compare different approaches to detecting DDoS attacks, using machine learning (ML) techniques such as k-means, K-Nearest Neighbors (KNN), and Naive Bayes (NB) used in intrusion detection systems (IDSs) and flow-based IDSs, and expresses data paths for packet filtering for HSN performance. This review highlights the high-speed network accuracy evaluation factors, provides a detailed DDoS attack taxonomy, and classifies detection techniques. Moreover, the existing literature is inspected through a qualitative analysis, with respect to the factors extracted from the presented taxonomy of irregular traffic pattern detection. Different research directions are suggested to support researchers in identifying and designing the optimal solution by highlighting the issues and challenges of DDoS attacks on high-speed networks.

Investigating on the robustness of flow-based intrusion detection system against adversarial samples using Generative Adversarial Networks

Recently, Software Defined Networking (SDN) has emerged as the key technology in programming and orchestrating security policy in the security operations centers (SOCs) for heterogeneous networks. Typically, machine learning-based intrusion detection systems (ML-IDS) have been deployed and associated with SDN to leverage the features of a programmable network to defend against sophisticated cyberattacks in anomaly detection. Unfortunately, such ML-based IDSs are easily vulnerable to adversarial attacks due to the lack of diverse forms of malicious records in the training dataset. The missing data sample in the training phase can lead to a lower detection rate in real-world scenarios with adversarial settings. In this paper, we explore the ability of Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP), WGAN-GP with two timescale update rule (WGAN-GP TTUR), and AdvGAN in generating perturbed attack samples to bypass attack detectors. Then, this approach is used to continuously evaluate the robustness of ML-based IDSs and then upgrade them as a service in SDN. The experimental results on CICIDS2018 and InSDN datasets demonstrate that generated adversarial samples can be used to fool targeted IDS. Later, those created samples can supplement the original ones in retraining IDS to improve the resilience of the attack detector.

Flow-based intrusion detection system in Vehicular Ad hoc Network using context-aware feature extraction

Autonomous transportation systems have an immense impact on the way of our day-to-day commute or traveling. Connectedness is an inseparable part of these systems, and it is possible through Intelligent Transportation Systems applications. These applications use a network system known as Vehicular Ad hoc Network (VANET). However, the safety that is provided by VANETs can be easily compromised by malicious users. Hence there is a need for an Intrusion Detection System (IDS). We designed an IDS model that can collect network data cooperatively from vehicles and Roadside Units (RSUs). For training the core of our proposed IDS we generated synthetic network data with the help of the widely used Network Simulator 3 (ns-3) and Simulation of Urban Mobility (SUMO) simulation tools. We also generated separate test data that is not just a split part of the training data. This ensures the elimination of the misleading performance result of an overfitted model. We employed a multi-class IDS using Convolutional Neural Network (CNN) with a novel feature extraction method known as Context-Aware Feature Extraction-Based CNN (CAFECNN). The CAFECNN model takes advantage of the collected network flow data to detect both passive and active types of attacks. The results show that the proposed model is stronger in identification of hard-to-detect passive attacks compared to traditional machine learning methods. We also evaluated the model using real-time simulations and observed an immediate improvement in the performance of the network despite the intruders, especially in terms of packet delivery ratio and network throughput.

Flow-Based IDS Features Enrichment for ICMPv6-DDoS Attacks Detection

Internet Protocol version 6 (IPv6) and its core protocol, Internet Control Message Protocol version 6 (ICMPv6), need to be secured from attacks, such as Denial of Service (DoS) and Distributed DoS (DDoS), in order to be reliable for deployment. Several Intrusion Detection Systems (IDSs) have been built and proposed to detect ICMPv6-based DoS and DDoS attacks. However, these IDSs suffer from several drawbacks, such as the inability to detect novel attacks and a low detection accuracy due to their reliance on packet-based traffic representation. Furthermore, the existing IDSs that rely on flow-based traffic representation use simple heuristics features that do not contribute to detecting ICMPv6-based DoS and DDoS attacks. This paper proposes a flow-based IDS by enriching the existing features with a set of new features to improve the detection accuracy. The flow consists of packets with similar attributes (i.e., packets with the same source and destination IP address) and features that can differentiate between normal and malicious traffic behavior, such as the source IP address’s symmetry and the whole flow’s symmetry. The experimental results reveal that the enriched features significantly improved the IDS’s detection accuracy by 16.02% and that the false positive rate decreased by 19.17% compared with state-of-the-art IDSs.

Ensemble of Bio-inspired Algorithm with Statistical Measures for Feature Selection to Design a Flow-Based Intrusion Detection System

In today's high-speed network, the existing Intrusion Detection System (IDS) approaches experience more false alarm rates with low detection capability. Nowadays, IDS needs to analyze a considerable amount of data. The larger the amount of data results in the longer the time to analyze it, which delays attack detection. The IDS usability is defined as its capability to trigger an alarm early enough to minimize the damage that an ongoing attack can cause and provide a reduced range of warning (false alarm). These underline the necessity of feature selection in IDS to identify the informative features and overlook the irrelevant or redundant features that affect the IDS's detection rate and computational complexity. It implies that anticipating an ideal number of features from a flow-based intrusion dataset can improve IDS accuracy. Therefore, this paper proposes an ensemble of a bio-inspired algorithm (Krill Herd Algorithm) with statistical measures (Information Gain) to select optimal features for a flow-based IDS. This ensemble technique has shown improvement in the detection rate, decreases the false alarm rate, and reduces the computation time of the IDS.

Hybrid intrusion detection system based on Dempster-Shafer evidence theory

Cyber-attacks are becoming increasingly sophisticated, posing greater challenges in accurately detecting intrusions. Failure to prevent intrusions could degrade the credibility of security services. Intrusion Detection System (IDS) is one of the most effective paradigms to identify attack behaviors. This paper proposes a novel hybrid intrusion detection system called DST-IDS. The proposed method employs both packet-based and flow-based intrusion detection techniques and combines them with Dempster-Shafer Theory (DST). DST-IDS has an ensemble-like framework. It takes both traffic flows and their first N packets as inputs; flow-based IDS aims to predict traffic flows and packet-based IDS detects attacks in the corresponding packets; DST is then applied to fuse predictions of flow-based IDS and packet-based IDS to a final detection result. We also design a novel data collection/processing tool in DST-IDS to reduce the data volume required to perform intrusion detection and enable early detection. In addition, DST-IDS is designed to work with heterogeneous data distribution where the distribution of the training dataset can differ from the data distribution during implementation. This property drastically improves the practicality of DST-IDS. We run experiments on public datasets and real networks to evaluate the proposed method. The experimental results show that DST-IDS outperforms state-of-the-art benchmarks in terms of intrusion detection accuracy and detection speed. Particularly, DST-IDS provides real-time detection in real networks and handles well heterogeneous data distribution.

IDS for Industrial Applications: A Federated Learning Approach with Active Personalization.

Internet of Things (IoT) is a concept adopted in nearly every aspect of human life, leading to an explosive utilization of intelligent devices. Notably, such solutions are especially integrated in the industrial sector, to allow the remote monitoring and control of critical infrastructure. Such global integration of IoT solutions has led to an expanded attack surface against IoT-enabled infrastructures. Artificial intelligence and machine learning have demonstrated their ability to resolve issues that would have been impossible or difficult to address otherwise; thus, such solutions are closely associated with securing IoT. Classical collaborative and distributed machine learning approaches are known to compromise sensitive information. In our paper, we demonstrate the creation of a network flow-based Intrusion Detection System (IDS) aiming to protecting critical infrastructures, stemming from the pairing of two machine learning techniques, namely, federated learning and active learning. The former is utilized for privately training models in federation, while the latter is a semi-supervised approach applied for global model adaptation to each of the participant’s traffic. Experimental results indicate that global models perform significantly better for each participant, when locally personalized with just a few active learning queries. Specifically, we demonstrate how the accuracy increase can reach 7.07% in only 10 queries.

Flow‐based intrusion detection algorithm for supervisory control and data acquisition systems: A real‐time approach

Intrusion detection in supervisory control and data acquisition (SCADA) systems is integral because of the critical roles of these systems in industries. However, available approaches in the literature lack representative flow-based datasets and reliable real-time adaption and evaluation. A publicly available labelled dataset to support flow-based intrusion detection research specific to SCADA systems is presented. Cyberattacks were carried out against our SCADA system test bed to generate this flow-based dataset. Moreover, a flow-based intrusion detection system (IDS) is developed for SCADA systems using a deep learning algorithm. We used the dataset to develop this IDS model for real-time operations of SCADA systems to detect attacks momentarily after they happen. The results show empirical proof of the model’s adequacy when deployed online to detect cyberattacks in real time.

A flow-based intrusion detection framework for internet of things networks

The application of the Internet of Things concept in domains such as industrial control, building automation, human health, and environmental monitoring, introduces new privacy and security challenges. Consequently, traditional implementation of monitoring and security mechanisms cannot always be presently feasible and adequate due to the number of IoT devices, their heterogeneity and the typical limitations of their technical specifications. In this paper, we propose an IP flow-based Intrusion Detection System (IDS) framework to monitor and protect IoT networks from external and internal threats in real-time. The proposed framework collects IP flows from an IoT network and analyses them in order to monitor and detect attacks, intrusions, and other types of anomalies at different IoT architecture layers based on some flow features instead of using packet headers fields and their payload. The proposed framework was designed to consider both the IoT network architecture and other IoT contextual characteristics such as scalability, heterogeneity, interoperability, and the minimization of the use of IoT networks resources. The proposed IDS framework is network-based and relies on a hybrid architecture, as it involves both centralized analysis and distributed data collection components. In terms of detection method, the framework uses a specification-based approach drawn on normal traffic specifications. The experimental results show that this framework can achieve ≈ 100% success and 0% of false positives in detection of intrusions and anomalies. In terms of performance and scalability in the operation of the IDS components, we study and compare it with three different conventional IDS (Snort, Suricata, and Zeek) and the results demonstrate that the proposed solution can consume fewer computational resources (CPU, RAM, and persistent memory) when compared to those conventional IDS.

Abstracting Packet Header Information for Intrusion Detection in High-Speed Networks

Increase in network traffic coupled with increasing adoption of end-to-end encryption of network packets are two major factors threatening the potency, or even the relevance, of packet-based intrusion detection techniques. Also, end-to-end encryption makes it nearly impossible for network and host-based intrusion detection system to analyze traffic for potential threats and intrusion, hence, the need for an alternative approach. Flow-based intrusion detection system has been proposed as an alternative to a packet-based intrusion detection system as it relies on information embedded in packet header and various statistical analyses of network flow for detecting intrusion. This paper proposes packet header information abstraction model for intrusion detection on the UNSW-NB15 intrusion dataset. Four existing classification algorithms which include: Classification and Regression Tree (CART), Naïve Bayes (NB), K-Nearest Neighbour (KNN), and Support Vector Machine (SVM) are used to evaluate the degree of representativeness of the proposed model using accuracy, sensitivity and specificity evaluation metrics. An average accuracy of 97.95% was recorded across the four models with the minimum accuracy of 97.76 on SVM and best accuracy of 98.05% on CART while Sensitivity of 1.0 on both CART and NB shows that the model performs well in correctly identifying attacks in the network. The average specificity of 0.98 is also an indication of low false positive. Results obtained show that the proposed abstraction model achieves high accuracy, sensitivity and specificity. The model can be used as filter on a high-speed network whereby packets flagged as an attack can be subjected to further analysis.Keywords—Data Abstraction, Data Mining,Flow-based, Intrusion detection, Network Security

An intelligent flow-based and signature-based IDS for SDNs using ensemble feature selection and a multi-layer machine learning-based classifier

Software-defined networking is a new paradigm that overcomes problems associated with traditional network architecture by separating the control logic from data plane devices. It also enhances performance by providing a highly-programmable interface that adapts to dynamic changes in network policies. As software-defined networking controllers are prone to single-point failures, providing security is one of the biggest challenges in this framework. This paper intends to provide an intrusion detection mechanism in both the control plane and data plane to secure the controller and forwarding devices respectively. In the control plane, we imposed a flow-based intrusion detection system that inspects every new incoming flow towards the controller. In the data plane, we assigned a signature-based intrusion detection system to inspect traffic between Open Flow switches using port mirroring to analyse and detect malicious activity. Our flow-based system works with the help of trained, multi-layer machine learning-based classifier, while our signature-based system works with rule-based classifiers using the Snort intrusion detection system. The ensemble feature selection technique we adopted in the flow-based system helps to identify the prominent features and hasten the classification process. Our proposed work ensures a high level of security in the Software-defined networking environment by working simultaneously in both control plane and data plane.

Anomaly-Based Intrusion Detection From Network Flow Features Using Variational Autoencoder

The rapid increase in network traffic has recently led to the importance of flow-based intrusion detection systems processing a small amount of traffic data. Furthermore, anomaly-based methods, which can identify unknown attacks are also integrated into these systems. In this study, the focus is concentrated on the detection of anomalous network traffic (or intrusions) from flow-based data using unsupervised deep learning methods with semi-supervised learning approach. More specifically, Autoencoder and Variational Autoencoder methods were employed to identify unknown attacks using flow features. In the experiments carried out, the flow-based features extracted out of network traffic data, including typical and different types of attacks, were used. The Receiver Operating Characteristics (ROC) and the area under ROC curve, resulting from these methods were calculated and compared with One-Class Support Vector Machine. The ROC curves were examined in detail to analyze the performance of the methods in various threshold values. The experimental results show that Variational Autoencoder performs, for the most part, better than Autoencoder and One-Class Support Vector Machine.

Effects of Machine Learning Approach in Flow-Based Anomaly Detection on Software-Defined Networking

Recent advancements in software-defined networking (SDN) make it possible to overcome the management challenges of traditional networks by logically centralizing the control plane and decoupling it from the forwarding plane. Through a symmetric and centralized controller, SDN can prevent security breaches, but it can also bring in new threats and vulnerabilities. The central controller can be a single point of failure. Hence, flow-based anomaly detection system in OpenFlow Controller can secure SDN to a great extent. In this research, we investigated two different approaches of flow-based intrusion detection system in OpenFlow Controller. The first of which is based on machine-learning algorithm where NSL-KDD dataset with feature selection ensures the accuracy of 82% with random forest classifier using the gain ratio feature selection evaluator. In the later phase, the second approach is combined with a deep neural network (DNN)-based intrusion detection system based on gated recurrent unit-long short-term memory (GRU-LSTM) where we used a suitable ANOVA F-Test and recursive feature elimination selection method to boost classifier output and achieve an accuracy of 88%. Substantial experiments with comparative analysis clearly show that, deep learning would be a better choice for intrusion detection in OpenFlow Controller.

Flow-Based IDS for ICMPv6-Based DDoS Attacks Detection

The Internet Control Message Protocol version Six (ICMPv6) is categorized as the most important part of the Internet Protocol version Six (IPv6) due to its core functionalities. However, ICMPv6 protocol is vulnerable to different types of attacks such as Distributed Denial of Services (DDoS) attacks that are based on ICMPv6 messages. ICMPv6-based DDoS attacks are the most performed attacks against IPv6 networks and considered a grave problem of today Internet. Intrusion Detection Systems (IDSs) under different categories have been proposed to detect ICMPv6-based DDoS attacks. However, these IDSs are inefficient in detecting the attacks due to their limitations. The main limitation of the existing IDSs is the dependency on packet-based representation and features which are unsuitable for detecting DDoS attacks as experimentally proven. Therefore, this research proposes a new IDS, based on a flow-based representation of traffic, and a set of novel features for detecting the attacks. This is the first time a flow-based representation and features are proposed to detect ICMPv6-based DDoS attacks. Cross-validation and supplied set testing approaches have been applied to evaluate the proposed IDS using seven classifiers. The evaluation experiments were conducted based on real datasets and showed that the proposed flow-based IDS with the proposed novel features is efficient and reliable in detecting ICMPv6-based DDoS attacks with acceptable detection accuracies and false positive rates.

A two-stage flow-based intrusion detection model for next-generation networks.

The next-generation network provides state-of-the-art access-independent services over converged mobile and fixed networks. Security in the converged network environment is a major challenge. Traditional packet and protocol-based intrusion detection techniques cannot be used in next-generation networks due to slow throughput, low accuracy and their inability to inspect encrypted payload. An alternative solution for protection of next-generation networks is to use network flow records for detection of malicious activity in the network traffic. The network flow records are independent of access networks and user applications. In this paper, we propose a two-stage flow-based intrusion detection system for next-generation networks. The first stage uses an enhanced unsupervised one-class support vector machine which separates malicious flows from normal network traffic. The second stage uses a self-organizing map which automatically groups malicious flows into different alert clusters. We validated the proposed approach on two flow-based datasets and obtained promising results.

Flow Based Intrusion Detection System Using Multistage Neural Network

With the rapid expansion of computer networks during the past decade, security has become a crucial issue for computer systems. And to keep security at highest level, there is an increasing need for effective security monitors such as Network Intrusion Detection System to prevent such illicit. In the recent years many researchers focus their hard work on this field using different approaches to build dependable intrusion detection systems. One of these approaches is Flow-based intrusion detection systems that rely on aggregated network traffic flows. In this paper, Multistage Neural Network intrusion detection system based on aggregated flow data is proposed for detecting and classifying attacks in network traffic. The proposed system detects significant changes in the traffic that could be a possible attack in the first stage of neural network, while the second stage has the ability to recognize an attack, to differentiate one attack from another i.e. classifying attack, and the most important, to detect new attacks with high detection rate and low false negative. Two different neural network structures with the use of different training algorithms have been used in our proposed Intrusion Detection System. The experimental results show that the designed system is promising in terms of accuracy and low probability of false alarms, where the overall accuracy classification rate average is equal to 99.25%.

Cluster Ensemble with Link-Based Approach for Botnet Detection

Botnet detection is one of the most imminent tasks for cyber security. Among popular botnet countermeasures, an intrusion detection system is the prominent mechanism. In the past, packet-based intrusion detection systems were popular. However, flow-based intrusion detection systems have been preferred in recent years due to their ability to adapt to modern high-speed networks. A collection of flows from an enterprise network usually contains both botnet traffic and normal traffic. To classify this traffic, supervised machine learning algorithms, i.e., classifications, have been applied and achieved a high accuracy. In an effort to improve the ability of intrusion detection systems against botnets, some studies have suggested partitioning flows into clusters before applying the classifications and this step could significantly reduce the complexity of a flow set. However, the instability of individual clustering algorithms is still a constraint for botnet detection.To overcome this bottleneck, we propose a novel method that combines individual partitions to become a strong learner through the use of a link-based algorithm. Our experiments show that our cluster ensemble model outperforms existing botnet detection mechanisms with a high reliability. We also determine the balance between accuracy and computer resources for botnet detection, and thereby propose a range for the maximum duration time of flows in botnet research.

Applying One-Class Classification Techniques to IP Flow Records for Intrusion Detection

Flow-basedintrusiondetectionsystemsanalyzeIPflowrecordstodetectattacksagainst computer networks. IP flow records contain aggregated packet header information; therefore, the amount of data processed by the intrusion detection system is reduced. In addition, since no pay- load is analyzed, the end-to-end encryption does not affect the deployment of intermediate intru- sion detection system. In this paper, we evaluate one-class classification techniques for detection of malicious flows at an initial stage of a multi-stage flow-based intrusion detection system. The initial stage uses minimal flow attributes and only decide if the IP flow is normal or malicious. Since there is only one class of interest (malicious) at the initial stage, we use one-class classifi- cation for detection of malicious flows. In this paper, we review available one-class classification techniques and evaluate them on a flow-based dataset to determine their performance for detec- tion of malicious flows. Our results show that one-class classification techniques using boundary methods give best results in detection of malicious IP flows.

Towards Multi-Stage Intrusion Detection using IP Flow Records

Traditional network-based intrusion detection sys-tems using deep packet inspection are not feasible for modern high-speed networks due to slow processing and inability to read encrypted packet content. As an alternative to packet-based intrusion detection, researchers have focused on flow-based intrusion detection techniques. Flow-based intrusion detection systems analyze IP flow records for attack detection. IP flow records contain summarized traffic information. However, flow data is very large in high-speed networks and cannot be processed in real-time by the intrusion detection system. In this paper, an efficient multi-stage model for intrusion detection using IP flows records is proposed. The first stage in the model classifies the traffic as normal or malicious. The malicious flows are further analyzed by a second stage. The second stage associates an attack type with malicious IP flows. The proposed multi-stage model is efficient because the majority of IP flows are discarded in the first stage and only malicious flows are examined in detail. We also describe the implementation of our model using machine learning techniques.

Flow-based anomaly intrusion detection system using two neural network stages

Computer systems and networks suffer due to rapid increase of attacks, and in order to keep them safe from malicious activities or policy violations, there is need for effective security monitoring systems, such as Intrusion Detection Systems (IDS). Many researchers concentrate their efforts on this area using different approaches to build reliable intrusion detection systems. Flow-based intrusion detection systems are one of these approaches that rely on aggregated flow statistics of network traffic. Their main advantages are host independence and usability on high speed networks, since the metrics may be collected by network device hardware or standalone probes. In this paper, an intrusion detection system using two neural network stages based on flow-data is proposed for detecting and classifying attacks in network traffic. The first stage detects significant changes in the traffic that could be a potential attack, while the second stage defines if there is a known attack and in that case classifies the type of attack. The first stage is crucial for selecting time windows where attacks, known or unknown, are more probable. Two different neural network structures have been used, multilayer and radial basis function networks, with the objective to compare performance, memory consumption and the time required for network training. The experimental results demonstrate that the designed models are promising in terms of accuracy and computational time, with low probability of false alarms.