Machine Learning-based Intrusion Detection Systems Research Articles

Overview and Exploratory Analyses of CICIDS 2017 Intrusion Detection Dataset

Intrusion detection systems are used to detect attacks on a network. Machine learning (ML) approaches have been widely used to build such intrusion detection systems (IDSs) because they are more accurate when built from a very large and representative dataset. Recently, one of the benchmark datasets that are used to build ML-based intrusion detection models is the CICIDS2017 dataset. The data set is contained in eight groups and was collected from the Data Set & Repository of the Canadian Institute of Cyber Security. The data set is available in both PCAP and net flow formats. This study used the net flow records in the CIDIDS2017 dataset, as they were found to contain newer attacks, very large, and useful for traffic analysis. Exploratory data analysis (EDA) techniques were used to reveal various characteristics of the dataset. The general objective is to provide more insight into the nature, structure, and issues of the data set so as to identify the best ways to use it to achieve improved ML-based IDS models. Furthermore, some of the open problems that can arise from the use of the dataset in any machine learning-based intrusion detection systems are highlighted and possible solutions are briefly discussed. The EDA techniques used revealed important relationships between the input variables and the target class. The study concluded that the EDA can better influence the decision about future IDS research using the dataset.

RAIDS: Robust autoencoder-based intrusion detection system model against adversarial attacks

Machine learning-based intrusion detection systems (IDS) are essential security functions in conventional and software-defined networks alike. Their success and the security of the networks they protect depend on the accuracy of their classification results. Adversarial attacks against machine learning, which seriously threaten any IDS, are still not countered effectively. In this study, we first develop a method that employs generative adversarial networks to produce adversarial attack data. Then, we propose RAIDS, a robust IDS model, designed to be resilient against adversarial attacks. In RAIDS, an autoencoder's reconstruction error is used as a prediction value for a classifier. Also, to prevent the attacker from guessing about the feature set, multiple feature sets are created and used to train baseline machine learning classifiers. A LightGBM classifier is then trained with the results produced by two autoencoders and an ensemble of baseline machine learning classifiers. The results show that the proposed robust model can increase overall accuracy by at least 13.2% and F1-score by more than 110% against adversarial attacks without the need for adversarial training.

Performance Evaluation and Validation of Intelligent Security Mechanism in Software Defined Network

Network attacks are discovered using intrusion detection systems (IDS), one of the most crucial security solutions. Machine learning techniques-based intrusion detection approaches have been rapidly created as a result of the widespread use of standard machine learning algorithms in the security field. Unfortunately, as technology has advanced and there have been faults in the machine learning-based intrusion detection system, the system has consistently failed to fulfill the standards for cyber security. Generative adversarial networks (GANs) have drawn a lot of interest recently and have been utilized widely in anomaly detection due to their enormous capacity for learning difficult high-dimensional real time data distribution. Traditional machine learning algorithms for intrusion detection have a number of drawbacks that deep learning techniques can significantly mitigate. With the help of a real time dataset, this work suggests employing GANs and its variants to detect network intrusions in SDN. The feasibility and comparison results are also presented. For different kinds of datasets, the BiGAN outcomes outperform the GAN.

A Study on Heuristic Algorithms Combined With LR on a DNN-Based IDS Model to Detect IoT Attacks

Current security challenges are made more difficult by the complexity and difficulty of spotting cyberattacks due to the Internet of Things explosive growth in connected devices and apps. Therefore, various sophisticated attack detection techniques have been created to address these issues in recent years. Due to their effectiveness and scalability, machine learning-based intrusion detection systems (IDSs) have increased. However, several factors, such as the characteristics of the training dataset and the training model, affect how well these AI-based systems identify attacks. In particular, the heuristic algorithms (GA, PSO, CSO, FA) optimized by the logistic regression (LR) approach employ it to pick critical features of a dataset and deal with data imbalance problems. This study offers an intrusion detection system (IDS) based on a deep neural network and heuristic algorithms combined with LR to boost the accuracy of attack detections. Our proposed model has a high attack detection rate of up to 99% when testing on the IoT-23 dataset.

Detection of Cyber Attacks using Machine Learning ‎based Intrusion Detection System for IoT Based Smart ‎Cities

The world’s dynamics is evolving with artificial intelligence (AI) and the results are smart products. A smart city has smart city is collection of smart innovations powered with AI and internet of things (IoTs). Along with the ease and comfort that the concept of a smart city pointed at, many security concerns are being raised that hinders the path of its flourishment. An Intrusion Detection System (IDS) monitors the whole network traffic and alerts in case of any anomaly. A Machine Learning-based IDS intelligently senses the network threats, takes decisions about data packet legibility and alarm the user. Researchers have deployed various ML techniques to IDS to improve the detection accuracy. This work presents a comparative analysis of various ML algorithms trained over UNSW-NB15 dataset. ADA Boost, Linear Support Vector Machine (LSVM), Auto Encoder Classifier, ‎Quadratic Support Vector Machine (QSVM) and Multi-Layer Perceptron algorithms are being employed in the stimulation. ADA Boost showed an excellent accuracy of 98.3% in the results.

Analysis of Efficient Intrusion Detection System using Ensemble Learning

Abstract: Our increasingly connected world continues to face an ever-growing amount of network-based attacks. Intrusion detection systems (IDS) are essential security technology for detecting these attacks. Although numerous machine learningbased IDS have been proposed for detecting malicious network traffic, most have difficulty properly detecting and classifying the more uncommon attack types. The research in Cyber Security has raised the need to address the cybercrimes that have caused the requisition of intellectual properties such as the breakdown of computer systems and impairment of important data compromising the confidentiality authenticity and integrity of the user. Considering these scenarios, securing the computer systems and the user using an Intrusion Detection System (IDS) is essential. The performance of IDS was studied by developing an IDS dataset consisting of network traffic features to learn the attack patterns. Intrusion detection is a classification problem wherein various Ensemble Learning (ML) and Data Mining (DM) techniques are applied to classify the network data into normal and attack traffic. Moreover, the types of network attacks changed over the years, so updating the datasets used for evaluating IDS is necessary.

A Review on Challenges and Future Research Directions for Machine Learning-Based Intrusion Detection System

A Review on Challenges and Future Research Directions for Machine Learning-Based Intrusion Detection System

An explainable efficient flow-based Industrial IoT intrusion detection system

With the steady growth in the adoption of IoT in an industrial context, security challenges intensify. Security threats to IoT devices in a nuclear plant or an oil refinery are much higher risk compared to home appliances. In this paper, we present an explainable, efficient, and highly accurate machine-learning based intrusion detection system for industrial IoT, referred to as E2I3DS. This work successfully reduces the needed features within the WUSTL-IIOT-2021 dataset from 48 features to 11 only, while maintaining very high accuracy. Experiments utilizing the 11-feature dataset showed that proposed system has a higher accuracy of 99.97%, with a false positive rate of 0.01% and a false negative rate of 0.04%. The proposed system also proved to be very efficient with an excellent detection time of 0.1517 μs. In addition, the model was explained using Shapley additive explanations.

ULDC: Unsupervised Learning-Based Data Cleaning for Malicious Traffic With High Noise

AbstractSince the traffic of novel attacks exceeds current knowledge, realistic traffic labeling methods are prone to mislabeling, which has a significant impact on machine learning-based intrusion detection systems. Data cleaning typically relies on the ability of supervised deep neural networks to learn correct knowledge. Under high noise conditions, noisy labels can affect a supervised network and render it ineffective. To clean traffic datasets under high noise conditions, we propose an unsupervised learning-based data cleaning framework (called ULDC) that does not rely on labels and powerful supervised networks, hence reducing the impact of noisy labels. ULDC evaluates the confidence of observed labels through the distribution and similarity of samples in low dimensions. Moreover, ULDC maximizes the retention of hard samples through adaptive intra-class threshold evaluation, preserving more hard samples for training and improving generalization. In evaluations of ULDC on the CIRA-CIC-DoHBrw-2020 dataset, the percentage of data correction reached more than 75% under high noise, which is better than that of the state-of-the-art methods. ULDC is applicable to traffic data cleaning in both traditional networks and novel networks such as the Internet of Things and mobile networks, and it has been validated on datasets including CIC-IDS-2017 and IoT-23.

Intrusion detection model using optimized quantum neural network and elliptical curve cryptography for data security

Secure data transmission in wireless mesh networks is a necessary attribute for machine learning-based intrusion detection systems (IDS). Numerous attacks may have an adverse effect on the system's computing efficiency. In order to accurately detect an attack and enable data protection, Whale with Cuckoo search optimization (WCSO) based quantum neural network (QNN) and elliptical curve cryptography (ECC) is presented. Whale optimization algorithm (WOA) is used to choose the features in the network data that aid in precisely detecting intrusions. To identify attacks, the optimized quantum network which combines the WOA approach with the feed-forward and backpropagation algorithms, is used. Sensitive data retrieving requires an encryption procedure that is enabled by the ECC algorithm, which could safely save the data files in the server, in order to, secure the documentation with security measures. In the event that the data owner maintains sensitive data on a server, the document is encrypted using the encryption method. To determine the best key optimized ECC is used. The QNN with WOA-based IDS framework is a solid option for real-time intrusion detection analysis with high accuracy of 98.5%. Thus, the study has demonstrated that the suggested effort will also provide better secure data storage, resolving security concerns.

Intrusion Detection in Vehicle Controller Area Network (CAN) Bus Using Machine Learning: A Comparative Performance Study.

Electronic Control Units (ECUs) have been increasingly used in modern vehicles to control the operations of the vehicle, improve driving comfort, and safety. For the operation of the vehicle, these ECUs communicate using a Controller Area Network (CAN) protocol that has many security vulnerabilities. According to the report of Upstream 2022, more than 900 automotive cybersecurity incidents were reported in 2021 only. In addition to developing a more secure CAN protocol, intrusion detection can provide a path to mitigate cyberattacks on the vehicle. This paper proposes a machine learning-based intrusion detection system (IDS) using a Support Vector Machine (SVM), Decision Tree (DT), and K-Nearest Neighbor (KNN) and investigates the effectiveness of the IDS using multiple real-world datasets. The novelty of our developed IDS is that it has been trained and tested on multiple vehicular datasets (Kia Soul and a Chevrolet Spark) to detect and classify intrusion. Our IDS has achieved accuracy up to 99.9% with a high true positive and a low false negative rate. Finally, the comparison of our performance evaluation outcomes demonstrates that the proposed IDS outperforms the existing works in terms of its liability and efficiency to detect cyber-attacks with a minimal error rate.

Two-phase Defense Against Poisoning Attacks on Federated Learning-based Intrusion Detection

The Machine Learning-based Intrusion Detection System (ML-IDS) becomes more popular because it doesn't need to manually update the rules and can recognize variants better, However, due to the data privacy issue in ML-IDS, the Federated Learning-based IDS (FL-IDS) was proposed. In each round of federated learning, each participant first trains its local model and sends the model's weights to the global server, which then aggregates the received weights and distributes the aggregated global model to participants. An attacker will use poisoning attacks, including label-flipping attacks and backdoor attacks, to directly generate a malicious local model and indirectly pollute the global model. Currently, a few studies defend against poisoning attacks, but they only discuss label-flipping attacks in the image field. Therefore, we propose a two-phase defense mechanism, called Defending Poisoning Attacks in Federated Learning (DPA-FL), applied to intrusion detection. The first phase employs relative differences to quickly compare weights between participants because the local models of attackers and benign participants are quite different. The second phase tests the aggregated model with the dataset and tries to find the attackers when its accuracy is low. Experiment results show that DPA-FL can reach 96.5% accuracy in defending against poisoning attacks. Compared with other defense mechanisms, DPA-FL can improve F1-score by 20∼64% under backdoor attacks. Also, DPA-FL can exclude the attackers within twelve rounds when the attackers are few.

Investigating on the robustness of flow-based intrusion detection system against adversarial samples using Generative Adversarial Networks

Recently, Software Defined Networking (SDN) has emerged as the key technology in programming and orchestrating security policy in the security operations centers (SOCs) for heterogeneous networks. Typically, machine learning-based intrusion detection systems (ML-IDS) have been deployed and associated with SDN to leverage the features of a programmable network to defend against sophisticated cyberattacks in anomaly detection. Unfortunately, such ML-based IDSs are easily vulnerable to adversarial attacks due to the lack of diverse forms of malicious records in the training dataset. The missing data sample in the training phase can lead to a lower detection rate in real-world scenarios with adversarial settings. In this paper, we explore the ability of Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP), WGAN-GP with two timescale update rule (WGAN-GP TTUR), and AdvGAN in generating perturbed attack samples to bypass attack detectors. Then, this approach is used to continuously evaluate the robustness of ML-based IDSs and then upgrade them as a service in SDN. The experimental results on CICIDS2018 and InSDN datasets demonstrate that generated adversarial samples can be used to fool targeted IDS. Later, those created samples can supplement the original ones in retraining IDS to improve the resilience of the attack detector.

Hybrid Intrusion Detection System Based on Combination of Random Forest and Autoencoder

To cope with the rising threats posed by network attacks, machine learning-based intrusion detection systems (IDSs) have been intensively researched. However, there are several issues that need to be addressed. It is difficult to deal with unknown attacks that do not appear in the training set, and as a result, poor detection rates are produced for these unknown attacks. Furthermore, IDSs suffer from high false positive rate. As different models learn data characteristics from different perspectives, in this work we propose a hybrid IDS which leverages both random forest (RF) and autoencoder (AE). The hybrid model operates in two steps. In particular, in the first step, we utilize the probability output of the RF classifier to determine whether a sample belongs to attack. The unknown attacks can be identified with the assistance of the probability output. In the second step, an additional AE is coupled to reduce the false positive rate. To simulate an unknown attack in experiments, we explicitly remove some samples belonging to one attack class from the training set. Compared with various baselines, our suggested technique demonstrates a high detection rate. Furthermore, the additional AE detection module decreases the false positive rate.

Intrusion Detection System Based on One-Class Support Vector Machine and Gaussian Mixture Model

Intrusion detection systems (IDSs) play a significant role in the field of network security, dealing with the ever-increasing number of network threats. Machine learning-based IDSs have attracted a lot of interest owing to their powerful data-driven learning capabilities. However, it is challenging to train the supervised learning algorithms when there are no attack data at hand. Semi-supervised anomaly detection algorithms, which train the model with only normal data, are more suitable. In this study, we propose a novel semi-supervised anomaly detection-based IDS that leverages the capabilities of representation learning and two anomaly detectors. In detail, the autoencoder (AE) is applied to extract representative features of normal data in the first step, and then two semi-supervised detectors, the one-class support vector machine (OCSVM) and Gaussian mixture model (GMM), are trained on the derived features. The two detectors collaborate to detect anomalous samples. The OCSVM predicts the abnormal samples initially, and after that, the GMM is applied to recheck the misclassified samples further. The experiments demonstrate that the AE improves the detection rate, and two detectors are more promising than a single one.

Development of a Machine-Learning Intrusion Detection System and Testing of Its Performance Using a Generative Adversarial Network.

Intrusion detection and prevention are two of the most important issues to solve in network security infrastructure. Intrusion detection systems (IDSs) protect networks by using patterns to detect malicious traffic. As attackers have tried to dissimulate traffic in order to evade the rules applied, several machine learning-based IDSs have been developed. In this study, we focused on one such model involving several algorithms and used the NSL-KDD dataset as a benchmark to train and evaluate its performance. We demonstrate a way to create adversarial instances of network traffic that can be used to evade detection by a machine learning-based IDS. Moreover, this traffic can be used for training in order to improve performance in the case of new attacks. Thus, a generative adversarial network (GAN)-i.e., an architecture based on a deep-learning algorithm capable of creating generative models-was implemented. Furthermore, we tested the IDS performance using the generated adversarial traffic. The results showed that, even in the case of the GAN-generated traffic (which could successfully evade IDS detection), by using the adversarial traffic in the testing process, we could improve the machine learning-based IDS performance.

Ensemble adversarial training based defense against adversarial attacks for machine learning-based intrusion detection system

In this paper, a defence mechanism is proposed against adversarial attacks. The defence is based on an ensemble classifier that is adversarially trained. This is accomplished by generating adversarial attacks from four different attack methods, i.e., Jacobian-based saliency map attack (JSMA), projected gradient descent (PGD), momentum iterative method (MIM), and fast gradient signed method (FGSM). The adversarial examples are used to identify the robust machine-learning algorithms which eventually participate in the ensemble. The adversarial attacks are divided into seen and unseen attacks. To validate our work, the experiments are conducted using NSLKDD, UNSW-NB15 and CICIDS17 datasets. Grid search for the ensemble is used to optimise results. The parameter used for performance evaluations is accuracy, F1 score and AUC score. It is shown that an adversarially trained ensemble classifier produces better results.

An investigation of machine learning-based intrusion detection system in mobile ad hoc network

Building stable networks is one of the most demanding issues in the current era, as the world is increasingly reliant on computers and technology. The standard MANET protocols, software, and facilities presume a collaborative and networking atmosphere that does not consider protection. Intrusion detection systems (IDS) that track centralised network operations and detect malicious nodes are often used to supplement certain security because mitigation strategies are never sufficient. This study describes ML techniques for distributing valuable properties to IDS for green smart transportation on MANET. The performance of ML-IDSs and a review of their adequacy in MANETs help the users determine intrusion when learning about the MANET context. ML optimised KDD IDS. Ensemble learning in this IDS process gave anomaly scores to controlled packets. Our solution to actual MANET dataset shortages is this ML technique. ML techniques, simulation, and a functioning prototype had created a more resilient IDS for green smart transportation. ML-enhanced IDS detected and reduced MANET harmful activity. This research expanded IDS knowledge through ubiquitous learning.

Detection of DDoS Attacks Using SHAP-Based Feature Reduction

Machine learning techniques are widely used to protect cyberspace against malicious attacks. In this paper, we propose a machine learning-based intrusion detection system to alleviate Distributed Denial-of-Service (DDoS) attacks, which is one of the most prevalent attacks that disrupt the normal traffic of the targeted network. The model prediction is interpreted using the SHapley Additive exPlanations (SHAP) technique, which also provides the most essential features with the highest Shapley values. For the proposed model, the CICIDS2017 dataset from Kaggle is used for training the classification algorithms. The top features selected by the SHAP technique are used for training a Conditional Tabular Generative Adversarial Networks (CTGAN) for synthetic data generation. The CTGAN-generated data are then used to train prediction models such as Support Vector Classifier (SVC), Random Forest (RF), and Naïve Bayes (NB). The performance of the model is characterized using a confusion matrix. The experiment results prove that the attack detection rate is significantly improved after applying the SHAP feature selection technique.

Case Study on the Application of Deep Learning to Network Intruder Detection

Deep learning has seen considerable success in several application sectors. Unfortunately, little research has been done on its efficacy in the context of network intrusion detection. This article includes case studies that use deep learning to identify network anomalies both supervised and unsupervised. It has been demonstrated that deep neural networks (DNNs) outperform current machine learning-based intrusion detection systems in the presence of shifting IP addresses. We also demonstrate how auto encoders can support network anomaly detection.