Anomaly-based Network Intrusion Detection System Research Articles

Fed-ANIDS: Federated learning for anomaly-based network intrusion detection systems

Fed-ANIDS: Federated learning for anomaly-based network intrusion detection systems

Recursive Feature Elimination with Cross-Validation with Decision Tree: Feature Selection Method for Machine Learning-Based Intrusion Detection Systems

The frequency of cyber-attacks on the Internet of Things (IoT) networks has significantly increased in recent years. Anomaly-based network intrusion detection systems (NIDSs) offer an additional layer of network protection by detecting and reporting the infamous zero-day attacks. However, the efficiency of real-time detection systems relies on several factors, including the number of features utilized to make a prediction. Thus, minimizing them is crucial as it implies faster prediction and lower storage space. This paper utilizes recursive feature elimination with cross-validation using a decision tree model as an estimator (DT-RFECV) to select an optimal subset of 15 of UNSW-NB15’s 42 features and evaluates them using several ML classifiers, including tree-based ones, such as random forest. The proposed NIDS exhibits an accurate prediction model for network flow with a binary classification accuracy of 95.30% compared to 95.56% when using the entire feature set. The reported scores are comparable to those attained by the state-of-the-art systems despite decreasing the number of utilized features by about 65%.

Effective network intrusion detection using stacking-based ensemble approach

The increasing demand for communication between networked devices connected either through an intranet or the internet increases the need for a reliable and accurate network defense mechanism. Network intrusion detection systems (NIDSs), which are used to detect malicious or anomalous network traffic, are an integral part of network defense. This research aims to address some of the issues faced by anomaly-based network intrusion detection systems. In this research, we first identify some limitations of the legacy NIDS datasets, including a recent CICIDS2017 dataset, which lead us to develop our novel dataset, CIPMAIDS2023-1. Then, we propose a stacking-based ensemble approach that outperforms the overall state of the art for NIDS. Various attack scenarios were implemented along with benign user traffic on the network topology created using graphical network simulator-3 (GNS-3). Key flow features are extracted using cicflowmeter for each attack and are evaluated to analyze their behavior. Several different machine learning approaches are applied to the features extracted from the traffic data, and their performance is compared. The results show that the stacking-based ensemble approach is the most promising and achieves the highest weighted F1-score of 98.24%.

Anomaly-based Network Intrusion Detection System using Deep Intelligent Technique

Background and objectives: Computer systems and network infrastructures are still exposed to many security risks and cyber-attack vulnerabilities despite advancements of information security. Traditional signature-based intrusion detection systems and security solutions by matching rule-based mechanism and prior knowledge are insufficient of fully protecting computer networks against novel attacks. For this purpose, Anomaly-based Network Intrusion Detection System (A-NIDS) as cyber security tool is considered for identifying and detecting anomalous behavior in the flow-based network traffic alongside with firewalls and other security measures. The main objective of the research is to improve the detection rate and reduce false-positive rates of the classifier using anomaly-based technique. Methods: an intelligent technique using deep learning algorithm and mutual information feature selection (MIFS) method to select optimal features on the benchmark datasets. Proposed method accurately capable of classifying normal and anomalous states of the data packets in a comprehensive way by combination of Long-Short term memory (LSTM) algorithm and MIFS method. Results: The model achieved encouraging results in terms of accuracy 99.79%, 0.002 false-positive rate with minimum time compared to other models recorded only 81.75s on CSE-CIC-IDS2018 dataset. At the end of the study, comparative studies are conducted to verify the effectiveness of proposed method on three realistic and latest intrusion detection datasets, named CSE_CIC-IDS2018, CIC-IDS2017, and NF-CSE-CIC-IDS2018 dataset. Conclusions: Proposed model in a combination of LSTM NN and Feature selection method (MIFS) increased detection rate and reduced false-positive alarms, also the model able to detect low frequent attacks while other existing models are suffering from.

Addressing the class imbalance problem in network intrusion detection systems using data resampling and deep learning

Network intrusion detection systems (NIDS) are the most common tool used to detect malicious attacks on a network. They help prevent the ever-increasing different attacks and provide better security for the network. NIDS are classified into signature-based and anomaly-based detection. The most common type of NIDS is the anomaly-based NIDS which is based on machine learning models and is able to detect attacks with high accuracy. However, in recent years, NIDS has achieved even better results in detecting already known and novel attacks with the adoption of deep learning models. Benchmark datasets in intrusion detection try to simulate real-network traffic by including more normal traffic samples than the attack samples. This causes the training data to be imbalanced and causes difficulties in detecting certain types of attacks for the NIDS. In this paper, a data resampling technique is proposed based on Adaptive Synthetic (ADASYN) and Tomek Links algorithms in combination with different deep learning models to mitigate the class imbalance problem. The proposed model is evaluated on the benchmark NSL-KDD dataset using accuracy, precision, recall and F-score metrics. The experimental results show that in binary classification, the proposed method improves the performance of the NIDS and outperforms state-of-the-art models with an achieved accuracy of 99.8%. In multi-class classification, the results were also improved, outperforming state-of-the-art models with an achieved accuracy of 99.98%.

Detection of Network Attacks using Machine Learning and Deep Learning Models

Anomaly-based network intrusion detection systems are highly significant in detecting network attacks. Robust machine learning and deep learning models for identifying network intrusion and attack types are proposed in this paper. Proposed models have experimented with the UNSW-NB15 dataset of 49 features for nine different attack samples. Decision Tree classifier produced the best accuracy of 99.05% compared to ensemble models - Random forest(98.96%), Adaboost(97.87%), and XGBoost(98.08%). K-Nearest Neighbour classifier trained for various values of K and best performance obtained for K=7 with the accuracy of 95.58%. A Deep Learning model with two dense layers with ReLU activation and a third dense layer with a Sigmoid activation function is designed for binary classification and produced good accuracy of 98.44% with ADAM optimizer, 80:20 Train-Test Split Ratio. Network attack exploits are detected with an accuracy 95% by XGBoost, Fuzzers attack with 90% accuracy by Random Forest, Generic attacks with 99% accuracy by Random Forest, and Reconnaissance attacks with 79% by Decision Trees. All features are relevant and strong in network attack detection, which eliminates the requirement of feature selection.

D-Score: An expert-based method for assessing the detectability of IoT-related cyber-attacks

IoT devices are known to be vulnerable to various cyber-attacks, such as data exfiltration and the execution of flooding attacks as part of a DDoS attack. When it comes to detecting such attacks using network traffic analysis, it has been shown that some attack scenarios are not always equally easy to detect if they involve different IoT models. That is, when targeted at some IoT models, a given attack can be detected rather accurately, while when targeted at others the same attack may result in too many false alarms. In this research, we attempt to explain this variability of IoT attack detectability and devise a risk assessment method capable of addressing a key question: how easy is it for an anomaly-based network intrusion detection system to detect a given cyber-attack involving a specific IoT model? In the process of addressing this question we (a) investigate the predictability of IoT network traffic, (b) present a novel taxonomy for IoT attack detection which also encapsulates traffic predictability aspects, (c) propose an expert-based attack detectability estimation method which uses this taxonomy to derive a detectability score (termed ‘D-Score’) for a given combination of IoT model and attack scenario, and (d) empirically evaluate our method while comparing it with a data-driven method.

Toward A Holistic, Efficient, Stacking Ensemble Intrusion Detection System using a Real Cloud-based Dataset

Network intrusion detection is a key step in securing today’s constantly developing networks. Various experiments have been put forward to propose new methods for resisting harmful cyber behaviors. Though, as cyber-attacks turn out to be more complex, the present methodologies fail to adequately solve the problem. Thus, network intrusion detection is now a significant decision-making challenge that requires an effective and intelligent approach. Various machine learning algorithms such as decision trees, neural networks, K nearest neighbor, logistic regression, support vector machine, and Naive Bayes have been utilized to detect anomalies in network traffic. However, such algorithms require adequate datasets to train and evaluate anomaly-based network intrusion detection systems. This paper presents a testbed that could be a model for building real-world datasets, as well as a newly generated dataset, derived from real network traffic, for intrusion detection. To utilize this real dataset, the paper also presents an ensemble intrusion detection model using a meta-classification approach enabled by stacked generalization to address the issue of detection accuracy and false alarm rate in intrusion detection systems.

Towards Model Generalization for Intrusion Detection: Unsupervised Machine Learning Techniques

Through the ongoing digitization of the world, the number of connected devices is continuously growing without any foreseen decline in the near future. In particular, these devices increasingly include critical systems such as power grids and medical institutions, possibly causing tremendous consequences in the case of a successful cybersecurity attack. A network intrusion detection system (NIDS) is one of the main components to detect ongoing attacks by differentiating normal from malicious traffic. Anomaly-based NIDS, more specifically unsupervised methods previously proved promising for their ability to detect known as well as zero-day attacks without the need for a labeled dataset. Despite decades of development by researchers, anomaly-based NIDS are only rarely employed in real-world applications, most possibly due to the lack of generalization power of the proposed models. This article first evaluates four unsupervised machine learning methods on two recent datasets and then defines their generalization strength using a novel inter-dataset evaluation strategy estimating their adaptability. Results show that all models can present high classification scores on an individual dataset but fail to directly transfer those to a second unseen but related dataset. Specifically, the accuracy dropped on average 25.63% in an inter-dataset setting compared to the conventional evaluation approach. This generalization challenge can be observed and tackled in future research with the help of the proposed evaluation strategy in this paper.

Evaluating and Improving Adversarial Robustness of Machine Learning-Based Network Intrusion Detectors

Machine learning (ML), especially deep learning (DL) techniques have been increasingly used in anomaly-based network intrusion detection systems (NIDS). However, ML/DL has shown to be extremely vulnerable to adversarial attacks, especially in such security-sensitive systems. Many adversarial attacks have been proposed to evaluate the robustness of ML-based NIDSs. Unfortunately, existing attacks mostly focused on feature-space and/or white-box attacks, which make impractical assumptions in real-world scenarios, leaving the study on practical gray/black-box attacks largely unexplored. To bridge this gap, we conduct the first systematic study of the gray/black-box traffic-space adversarial attacks to evaluate the robustness of ML-based NIDSs. Our work outperforms previous ones in the following aspects: (i) practical-the proposed attack can automatically mutate original traffic with extremely limited knowledge and affordable overhead while preserving its functionality; (ii) generic-the proposed attack is effective for evaluating the robustness of various NIDSs using diverse ML/DL models and non-payload-based features; (iii) explainable-we propose an explanation method for the fragile robustness of ML-based NIDSs. Based on this, we also propose a defense scheme against adversarial attacks to improve system robustness. We extensively evaluate the robustness of various NIDSs using diverse feature sets and ML/DL models. Experimental results show our attack is effective (e.g., >97% evasion rate in half cases for Kitsune, a state-of-the-art NIDS) with affordable execution cost and the proposed defense method can effectively mitigate such attacks (evasion rate is reduced by >50% in most cases).

A Review on Machine Learning Approaches for Network Malicious Behavior Detection in Emerging Technologies.

Network anomaly detection systems (NADSs) play a significant role in every network defense system as they detect and prevent malicious activities. Therefore, this paper offers an exhaustive overview of different aspects of anomaly-based network intrusion detection systems (NIDSs). Additionally, contemporary malicious activities in network systems and the important properties of intrusion detection systems are discussed as well. The present survey explains important phases of NADSs, such as pre-processing, feature extraction and malicious behavior detection and recognition. In addition, with regard to the detection and recognition phase, recent machine learning approaches including supervised, unsupervised, new deep and ensemble learning techniques have been comprehensively discussed; moreover, some details about currently available benchmark datasets for training and evaluating machine learning techniques are provided by the researchers. In the end, potential challenges together with some future directions for machine learning-based NADSs are specified.

Training Guidance with KDD Cup 1999 and NSL-KDD Data Sets of ANIDINR: Anomaly-Based Network Intrusion Detection System

Training Guidance with KDD Cup 1999 and NSL-KDD Data Sets of ANIDINR: Anomaly-Based Network Intrusion Detection System

Training Guidance with KDD Cup 1999 and NSL-KDD Data Sets of ANIDINR: Anomaly-Based Network Intrusion Detection System

Abstract In today’s world, the protection of the computer networks remains one of the most crucial and difficult challenges in cyber security. In this work, a passive defence system ANIDINR is presented, aiming to monitor and protect computer networks. Our effort is focused on providing step-by-step guidance on methodologies selection and execution for the Machine and Deep Learning models’ training. Taking as an input two data sets, five MDL models are evaluated. Our goals are to minimise the percentage of Undetected Attack, the percentage of False Alarm Rate and the overall testing time. Based on this set-up, the proposed system is capable to predict in near-to-real time well-known and zero-day computer network attacks.

A COMBINATION OF TEMPORAL SEQUENCE LEARNING AND DATA DESCRIPTION FOR ANOMALYBASED NIDS

Through continuous observation and modeling of normal behavior in networks, Anomaly-based Network Intrusion Detection System (A-NIDS) offers a way to find possible threats via deviation from the normal model. The analysis of network traffic based on the time series model has the advantage of exploiting the relationship between packages within network traffic and observing trends of behaviors over a period of time. It will generate new sequences with good features that support anomaly detection in network traffic and provide the ability to detect new attacks. Besides, an anomaly detection technique, which focuses on the normal data and aims to build a description of it, will be an effective technique for anomaly detection in imbalanced data. In this paper, we propose a combination model of Long Short Term Memory (LSTM) architecture for processing time series and a data description Support Vector Data Description (SVDD) for anomaly detection in A-NIDS to obtain the advantages of them. This model helps parameters in LSTM and SVDD are jointly trained with the joint optimization method. Our experimental results with KDD99 dataset show that the proposed combined model obtains high performance in intrusion detection, especially DoS and Probe attacks with 98.0% and 99.8%, respectively.

Improving Intrusion Detection Model Prediction by Threshold Adaptation

Network traffic exhibits a high level of variability over short periods of time. This variability impacts negatively on the accuracy of anomaly-based network intrusion detection systems (IDS) that are built using predictive models in a batch learning setup. This work investigates how adapting the discriminating threshold of model predictions, specifically to the evaluated traffic, improves the detection rates of these intrusion detection models. Specifically, this research studied the adaptability features of three well known machine learning algorithms: C5.0, Random Forest and Support Vector Machine. Each algorithm’s ability to adapt their prediction thresholds was assessed and analysed under different scenarios that simulated real world settings using the prospective sampling approach. Multiple IDS datasets were used for the analysis, including a newly generated dataset (STA2018). This research demonstrated empirically the importance of threshold adaptation in improving the accuracy of detection models when training and evaluation traffic have different statistical properties. Tests were undertaken to analyse the effects of feature selection and data balancing on model accuracy when different significant features in traffic were used. The effects of threshold adaptation on improving accuracy were statistically analysed. Of the three compared algorithms, Random Forest was the most adaptable and had the highest detection rates.

A COMBINATION OF TEMPORAL SEQUENCE LEARNING AND DATA DESCRIPTION FOR ANOMALYBASED NIDS

Through continuous observation and modelling of normal behavior in networks, Anomaly-based Network Intrusion Detection System (A-NIDS) offers a way to find possible threats via deviation from the normal model. The analysis of network traffic based on time series model has the advantage of exploiting the relationship between packages within network traffic and observing trends of behaviors over a period of time. It will generate new sequences with good features that support anomaly detection in network traffic and provide the ability to detect new attacks. Besides, an anomaly detection technique, which focuses on the normal data and aims to build a description of it, will be an effective technique for anomaly detection in imbalanced data. In this paper, we propose a combination model of Long Short Term Memory (LSTM) architecture for processing time series and a data description Support Vector Data Description (SVDD) for anomaly detection in A-NIDS to obtain the advantages of them. This model helps parameters in LSTM and SVDD are jointly trained with joint optimization method. Our experimental results with KDD99 dataset show that the proposed combined model obtains high performance in intrusion detection, especially DoS and Probe attacks with 98.0% and 99.8%, respectively.

A Method for Guaranteeing Wireless Communication Based on a Combination of Deep and Shallow Learning

Wireless communication has changed and improved people's lives and society, especially with the arrival of the Internet of Things (IoT) era. Despite the maturity of wireless communication, the security issue of communication remains the most stubborn and troublesome problem due to the increasingly complex and large amounts of data. An intrusion detection system is the guarantee of secure communication. However, variable protocols and drastic growth in data volume make intrusion detection a difficult task. In this paper, we proposed a framework of anomaly-based network intrusion detection system to finish the detection job. First, UNSW-NB15 is selected as the research object. Based on this new dataset, we built a detection model combining a deep learning method and a shallow learning approach. The former one is a deep auto-encoder used for feature learning, which can discover important representations of data and accelerate detection. The latter one is a powerful support vector machine (SVM), where the artificial bee colony (ABC) algorithm is used to find optimal parameters for SVM with five-fold cross validation (5FCV). Various experiments are conducted and the simulation results prove that the proposed method performs quite better than some of state-of-the-art intrusion detection approaches, including the method based on the principal component analysis (PCA) and some other machine learning strategies.

A holistic review of Network Anomaly Detection Systems: A comprehensive survey

Network Anomaly Detection Systems (NADSs) are gaining a more important role in most network defense systems for detecting and preventing potential threats. The paper discusses various aspects of anomaly-based Network Intrusion Detection Systems (NIDSs). The paper explains cyber kill chain models and cyber-attacks that compromise network systems. Moreover, the paper describes various Decision Engine (DE) approaches, including new ensemble learning and deep learning approaches. The paper also provides more details about benchmark datasets for training and validating DE approaches. Most of NADSs’ applications, such as Data Centers, Internet of Things (IoT), as well as Fog and Cloud Computing, are also discussed. Finally, we present several experimental explanations which we follow by revealing various promising research directions.

TR-IDS: Anomaly-Based Intrusion Detection through Text-Convolutional Neural Network and Random Forest

As we head towards the IoT (Internet of Things) era, protecting network infrastructures and information security has become increasingly crucial. In recent years, Anomaly-Based Network Intrusion Detection Systems (ANIDSs) have gained extensive attention for their capability of detecting novel attacks. However, most ANIDSs focus on packet header information and omit the valuable information in payloads, despite the fact that payload-based attacks have become ubiquitous. In this paper, we propose a novel intrusion detection system named TR-IDS, which takes advantage of both statistical features and payload features. Word embedding and text-convolutional neural network (Text-CNN) are applied to extract effective information from payloads. After that, the sophisticated random forest algorithm is performed on the combination of statistical features and payload features. Extensive experimental evaluations demonstrate the effectiveness of the proposed methods.

An anomaly-based network intrusion detection system using ensemble clustering

An anomaly-based network intrusion detection system using ensemble clustering