UNSW-NB15 Dataset Research Articles

A Network Intrusion Detection Approach Using Extreme Gradient Boosting with Max-Depth Optimization and Feature Selection

Network intrusion detection system (NIDS) has become a vital tool to protect information anddetect attacks in computer networks. The performance of NIDSs can be evaluated by the numberof detected attacks and false alarm rates. Machine learning (ML) methods are commonly usedfor developing intrusion detection systems and combating the rapid evolution in the pattern ofattacks. Although there are several methods proposed in the state-of-the-art, the development ofthe most effective method is still of research interest and needs to be developed. In this paper,we develop an optimized approach using an extreme gradient boosting (XGB) classifier withcorrelation-based feature selection for accurate intrusion detection systems. We adopt the XGBclassifier in the proposed approach because it can bring down both variance and bias and hasseveral advantages such as parallelization, regularization, sparsity awareness hardware optimization,and tree pruning. The XGB uses the max-depth parameter as a specified criterion toprune the trees and improve the performance significantly. The proposed approach selects thebest value of the max-depth parameter through an exhaustive search optimization algorithm.We evaluate the approach on the UNSW-NB15 dataset that imitates the modern-day attacks ofnetwork traffic. The experimental results show the ability of the proposed approach to classifyingthe type of attacks and normal traffic with high accuracy results compared with the currentstate-of-the-art work on the same dataset with the same partitioning ratio of the test set.

BAE: Anomaly Detection Algorithm Based on Clustering and Autoencoder

In this paper, we propose an outlier-detection algorithm for detecting network traffic anomalies based on a clustering algorithm and an autoencoder model. The BIRCH clustering algorithm is employed as the pre-algorithm of the autoencoder to pre-classify datasets with complex data distribution characteristics, while the autoencoder model is used to detect outliers based on a threshold. The proposed BIRCH-Autoencoder (BAE) algorithm has been tested on four network security datasets, KDDCUP99, UNSW-NB15, CICIDS2017, and NSL-KDD, and compared with representative algorithms. The BAE algorithm achieved average F-scores of 96.160, 81.132, and 91.424 on the KDDCUP99, UNSW-NB15, and CICIDS2017 datasets, respectively. These experimental results demonstrate that the proposed approach can effectively and accurately detect anomalous data.

A Deep Learning Approach for Network Intrusion Detection Using a Small Features Vector

With the growth in network usage, there has been a corresponding growth in the nefarious exploitation of this technology. A wide array of techniques is now available that can be used to deal with cyberattacks, and one of them is network intrusion detection. Artificial Intelligence (AI) and Machine Learning (ML) techniques have extensively been employed to identify network anomalies. This paper provides an effective technique to evaluate the classification performance of a deep-learning-based Feedforward Neural Network (FFNN) classifier. A small feature vector is used to detect network traffic anomalies in the UNSW-NB15 and NSL-KDD datasets. The results show that a large feature set can have redundant and unuseful features, and it requires high computation power. The proposed technique exploits a small feature vector and achieves better classification accuracy.

Deep learning model for intrusion detection system utilizing convolution neural network

Abstract An integral part of any reliable network security infrastructure is the intrusion detection system (IDS). Early attack detection can stop adversaries from further intruding on a network. Machine learning (ML) and deep learning (DL) techniques to automate intrusion threat detection at a scale never previously envisioned have snowballed during the past 10 years. Researchers, software engineers, and network professionals have been encouraged to reconsider the use of ML techniques, notably in cybersecurity. This article proposes a system for detecting intrusion with two approaches, the first utilizing a proposed hybrid convolutional neural network (CNN) and Dense layers. The second utilizes naïve Bayes (NB) ML techniques and compares the two approaches to determine the best detection accuracy. The preprocessing of network data is necessary. The suggested technique is evaluated using the UNSW-NB15 Dataset to create a reliable classifier and an effective IDS. The experimental results for the proposed CNN-dense classifier outperformed the ML and DL models. CNN has a 99.8% accuracy rate compared to previous studies. At the same time, the Gaussian naïve Bayes, which is considered the best among the ML-utilized classifiers, yielded an 83% accuracy rate.

Leveraging Explainable Artificial Intelligence in Real-Time Cyberattack Identification: Intrusion Detection System Approach

Cyberattacks are part of the continuous race, where research in computer science both contributes to discovering new threats and vulnerabilities and also mitigates them. When new vulnerabilities are not reported but sold to attackers, they are called “zero-days,” and are particularly difficult to identify. Modern intrusion detection systems (IDS) that leverage artificial intelligence (AI) and machine learning (ML) are becoming essential in identifying these cyber threats. This study presents the design of an IDS using ML and Explainable AI (XAI) techniques for real-time classification of various detected cyberattacks. By utilizing frameworks such as Apache Kafka and Spark, along with libraries such as Scikit-learn and SHAP, the system identifies and classifies normal or anomalous network traffic in real-time. The XAI offers the IDS the option to explain the rationale behind each classification. The primary aim of this research is to develop a flexible and scalable IDS that can provide clear explanations for its decisions. The second aim is to compare and analyze different ML models to achieve the best results in terms of accuracy, f1, recall, and precision. Random Forest models proposed in this research article obtained the best results in figuring out the key features identified by the XAI model, which includes Ct_state_ttl, Sttl, Dmean, and Dbytes from the UNSW-NB15 dataset. Finally, this research work introduces different machine learning algorithms with superior performance metrics compared to other real-time classification methods.

RETRACTED ARTICLE: Fusion of transformer and ML-CNN-BiLSTM for network intrusion detection

Network intrusion detection system (NIDS) can effectively sense network attacks, which is of great significance for maintaining the security of cyberspace. To meet the requirements of efficient and accurate network status monitoring, a NIDS model using Transformer-based fusion deep learning architecture is proposed. Firstly, GAN-Cross is used to expand minority class sample data, thereby alleviating the issues of imbalanced minority class about the original dataset. Then, the Transformer module is used to adjust the ML-CNN-BiLSTM model to enhance the feature encoding ability of the intrusion model. Finally, the data enhancement model and feature enhancement model are integrated into the NIDS model, the detection model is optimized, the features of network state data are extracted at a deeper level, and the generalization ability of the detection model is enhanced. Some simulation experiments using UNSW-NB15 datasets show that the proposed fusion architecture can achieve efficient analysis of complex network traffic datasets, with an accuracy of 0.903, effectively improving the detection accuracy of NIDS and its ability to detect unknown attacks. The proposed model has good application value in ensuring the stable operation of network systems.

An intrusion detection system based on stacked ensemble learning for IoT network

As Internet of Things (IoT) technology advances rapidly, the need for security measures in this domain intensifies, mainly due to the inherent computational resource constraints and production cost limitations of IoT devices. These factors make them vulnerable to the botnet attacks. We propose an Intrusion Detection System (IDS) based on an effective stacked-based ensemble learning approach. Considering the computational resources constraints, we employ feature selection and hyperparameter optimization techniques to lessen the computational overhead and enhance detection performance. Our proposed IDS is tested on the Python platform and proven effective performance on NB-aIoT and UNSW-NB15 datasets. The experimental results show that our proposed IDS achieves a high average accuracy rate of 99.68% and has the potential to improve the security of IoT devices, ultimately benefiting users that rely on these devices.

Detection of Cyber Attacks using Machine Learning ‎based Intrusion Detection System for IoT Based Smart ‎Cities

The world’s dynamics is evolving with artificial intelligence (AI) and the results are smart products. A smart city has smart city is collection of smart innovations powered with AI and internet of things (IoTs). Along with the ease and comfort that the concept of a smart city pointed at, many security concerns are being raised that hinders the path of its flourishment. An Intrusion Detection System (IDS) monitors the whole network traffic and alerts in case of any anomaly. A Machine Learning-based IDS intelligently senses the network threats, takes decisions about data packet legibility and alarm the user. Researchers have deployed various ML techniques to IDS to improve the detection accuracy. This work presents a comparative analysis of various ML algorithms trained over UNSW-NB15 dataset. ADA Boost, Linear Support Vector Machine (LSVM), Auto Encoder Classifier, ‎Quadratic Support Vector Machine (QSVM) and Multi-Layer Perceptron algorithms are being employed in the stimulation. ADA Boost showed an excellent accuracy of 98.3% in the results.

Analysis of Intrusion Detection Systems in UNSW-NB15 and NSL-KDD Datasets with Machine Learning Algorithms

Recently, the need for Network-based systems and smart devices has been increasing rapidly. The use of smart devices in almost every field, the provision of services by private and public institutions over network servers, cloud technologies and database systems are almost completely remotely controlled. Due to these increasing requirements for network systems, malicious software and users, unfortunately, are increasing their interest in these areas. Some organizations are exposed to almost hundreds or even thousands of network attacks daily. Therefore, it is not enough to solve the attacks with a virus program or a firewall. Detection and correct analysis of network attacks is vital for the operation of the entire system. With deep learning and machine learning, attack detection and classification can be done successfully. In this study, a comprehensive attack detection process was performed on UNSW-NB15 and NSL-KDD datasets with existing machine learning algorithms. In the UNSW-NB115 dataset, 98.6% and 98.3% accuracy were obtained for two-class and multi-class, respectively, and 97.8% and 93.4% accuracy in the NSL-KDD dataset. The results prove that machine learning algorithms are lateral to the solution in intrusion detection systems.

An Intrusion Detection Method based on Fusion Neural Network

Aiming at the problems of class imbalance, insufficient feature learning, weak generalization ability, and representation capability in existing intrusion detection models, we propose a multi-scale feature fusion Intrusion Detection Model (MSFF). This model combines multi-scale one-dimensional convolution and bidirectional long short-term memory (LSTM) networks, and incorporates residual connections with identity mappings to address the problem of network degradation. The multi-scale convolution captures feature representations at different levels, thereby improving the expressive power of the model. The WGAN-GP algorithm is employed to augment the minority samples and balance the dataset. By performing convolution operations and extracting local window features and global features using bidirectional LSTM units, the model effectively captures temporal information and long-term dependencies. Experimental results demonstrate significant performance improvement compared to a single model. The MSFF model achieves an accuracy of 99.50% and 94.73% in binary classification experiments on the NSL-KDD and UNSW-NB15 datasets, respectively, and an accuracy of 99.50% and 83.78% in multi-class classification experiments.

A Big Data Analytical Framework for Intrusion Detection Based On Novel Elephant Herding Optimized Finite Dirichlet Mixture Models

For the purpose of identifying a wide variety of hostile activity in cyberspace, an Intrusion Detection System (IDS) is a crucial instrument. However, traditional IDSs have limitations in detecting zero-day attacks, which can lead to high false alarm rates. To address this issue, it is crucial to integrate the monitoring and analysis of network data with decision-making methods that can identify anomalous events accurately. By combining these approaches, organizations can develop more effective cybersecurity measures and better protect their networks from cyber threats. In this study, we proposed a novel called the Elephant Herding Optimized Finite Dirichlet Mixture Model (EHO-FDMM). This framework consists of three modules: capture and logging, pre-processing, and an innovative IDS method based on the EHO-FDMM. The NSL-KDD and UNSW-NB15 datasets are used to assess this framework's performance. The empirical findings show that selecting the optimum model that accurately fits the network data is aided by statistical analysis of the data. The EHO-FDMM-based intrusion detection method also offers a lower False Alarm Rate (FPR) and greater Detection Rate (DR) than the other three strong methods. The EHO-FDMM and exact interval of confidence bounds were used to create the suggested method's ability to detect even minute variations between legal and attack routes. These methods are based on correlations and proximity measurements, which are ineffective against contemporary assaults that imitate everyday actions.

Improved bacterial foraging optimization with deep learning based anomaly detection in smart cities

The Internet of Things (IoT) contains many smart devices that collect, store, communicate, and process data. IoT implementation has performed novel opportunities in industries, environments, businesses, and homes. Anomaly detection (AD) is helpful in IoT platforms that can recognize and prevent potential system failures, decrease downtime, enhance the quality of products and services, and improve overall operational efficacy. AD systems for IoT data contain statistical modelling, deep learning (DL), and machine learning (ML) approaches which detect patterns and anomalies in the data. This article introduces an Improved Bacterial Foraging Optimization with optimum deep learning for Anomaly Detection (IBFO-ODLAD) in the IoT network. The presented IBFO-ODLAD technique performs data normalization using Z-score normalization approach. For the feature selection process, the IBFO-ODLAD technique designs the IBFO algorithm to choose an optimal subset of features. In addition, the IBFO-ODLAD technique uses multiplicative long short term memory (MLSTM) model for intrusion detection and classification process. Furthermore, the Bayesian optimization algorithm (BOA) was executed for the optimum hyperparameter selection of the MLSTM model. The experimental outcome of the IBFO-ODLAD method was validated on the UNSW NB-15 dataset and UCI SECOM dataset. The experimental outcomes signified the improved performance of the IBFO-ODLAD algorithm with maximum accuracy of 98.89 % and 98.66 % validated on the UNSW NB-15 dataset and UCI SECOM dataset respectively.

An Efficient NIDPS with Improved Salp Swarm Feature Optimization Method

Network security problems arise these days due to many challenges in cyberspace. The malicious attacks on installed wide networks are rapidly spreading due to their vulnerability. Therefore, the user and system information are at high risk due to network attacks. To protect networks against these attacks, Network Intrusion Detection and Prevention Systems (NIDPS) are installed on them. These NIDPS can detect malicious attacks by monitoring abnormal behavior and patterns in network traffic. These systems were mainly developed using Artificial Intelligence (AI) algorithms. These intelligent NIDPS are also able to detect the attack type while detecting network attacks. Previous studies have proposed many NIDPS for network security. However, many challenges exist so far such as limited available data for training AI algorithms, class imbalance problems, and automated selection of the most important features. These problems need to be solved first, which will lead to the precise detection of network attacks. Therefore, the proposed framework used the highly imbalanced UNSW-NB15 dataset for binary and multiclass classification of network attacks. In this framework, firstly dataset normalization is applied using standard deviation and the mean of feature columns; secondly, an Improved Salp Swarm Algorithm (ISSA) is applied for automated feature selection separately on binary and multiclass subsets. Thirdly, after applying feature selection, the SMOTE–Tomek class balancing method is applied where at least four different ML classifiers are used for binary and multiclass classification. The achieved results outperformed as compared to previous studies and improved the overall performance of NIDPS.

Intrusion Detection using hybridized Meta-heuristic techniques with Weighted XGBoost Classifier

Due to the widespread global internet services, service providers and users face a primary problem defending their systems, specifically from a new category of attacks and breaches. Network Intrusion Detection system (NIDS) assesses the network packets and reports low-security violations to respective system administrators. In the case of large imbalance datasets with more non-relevant features, the accuracy in classifying ad predicting precise intrusions needs to be improved. Moreover, most state-of-art intrusion-detection models based on machine learning may face high false-positive rates, imbalanced data with low training performance, low accuracy in detection, and complexity in optimization of feature selection aiding classification for impersonation attacks. Hence to overcome those complications, the present study deliberates an efficient IDS Modified Wrapper-based Whale Sine–cosine algorithm (MWWSCA) with Weighted Extreme Gradient Boosting (XgBoost) Classifier. The proposed model hybridizes the modified wrapper Whale Optimization approach and Sine–Cosine algorithm feature selection method to pick out the most discriminative, associated, and approximate best optimal features, enhancing the quality of prediction, not to fall on towards optimal local solution. Moreover, it balances out the exploitation and exploration phase of the model. However, the algorithm’s performance may decline on classifying the multi-attack and binary attacks accurately and may be prone to an imbalance in classes; thus, a Weighted XGBoost classifier with regularization of the loss function is implemented in binary and multi-classification. It utilizes the best optimal features, assign high weights to weak minor class features, and handles class imbalance issues. Overfitting of the model is tackled through regularization in the loss function during the stage of feature classification. The experimental outcomes and comparative assessment, in multi-class and binary attack classification from UNSW-NB15 and CICIDS datasets, explicated outperforming results with high accuracy, Precision, Recall, and F1-Score metrics.

Abnormal Traffic Detection: Traffic Feature Extraction and DAE-GAN With Efficient Data Augmentation

Abnormal traffic detection is the core component of the network intrusion detection system. Although semisupervised methods can detect zero-day attack traffic, previous work suffers from high false alarms because the trained model is simply based on normal traffic. In this article, we propose an accurate abnormal traffic detection method using pseudoanomaly, consisting of an efficient feature extraction framework and a novel denoise autoencoder-generative adversarial network (DAE-GAN) model. The feature extraction framework adopts an innovative packet window scheme to extract spatial and temporal features from traffic flows. The DAE-GAN model has multiple DAEs to achieve efficient data augmentation and generate high-quality pseudoanomalies. The pseudoanomalies are obtained by adding noise on normal traffic and enhanced by adversarial learning in DAE-GAN. Our semisupervised detection method, exploiting both normal data and generated pseudoanomalies, achieves a precision of 98.6% on the NSL-KDD dataset and 98.5% on the UNSW-NB15 dataset. Compared with the state-of-the-art, the detection precision and recall under different user behaviors are significantly improved. The evaluation on four attack datasets shows that our method has a high flow-wise precision of over 99% and a high recall of 60.6%.

Intrusion Detection Method for Internet of Vehicles Based on Parallel Analysis of Spatio-Temporal Features.

The problems with network security that the Internet of Vehicles (IoV) faces are becoming more noticeable as it continues to evolve. Deep learning-based intrusion detection techniques can assist the IoV in preventing network threats. However, previous methods usually employ a single deep learning model to extract temporal or spatial features, or extract spatial features first and then temporal features in a serial manner. These methods usually have the problem of insufficient extraction of spatio-temporal features of the IoV, which affects the performance of intrusion detection and leads to a high false-positive rate. To solve the above problems, this paper proposes an intrusion detection method for IoV based on parallel analysis of spatio-temporal features (PA-STF). First, we built an optimal subset of features based on feature correlations of IoV traffic. Then, we used the temporal convolutional network (TCN) and long short-term memory (LSTM) to extract spatio-temporal features in the IoV traffic in a parallel manner. Finally, we fused the spatio-temporal features extracted in parallel based on the self-attention mechanism and used a multilayer perceptron to detect attacks in the Internet of Vehicles. The experimental results show that the PA-STF method reduces the false-positive rate by 1.95% and 1.57% on the NSL-KDD and UNSW-NB15 datasets, respectively, with the accuracy and F1 score also being superior.

Intrusion Detection System Using the G-ABC with Deep Neural Network in Cloud Environment

Cloud computing plays a pivotal role in sharing resources and information. It is challenging to secure cloud services from different intruders. Intrusion detection system (IDS) plays a vital role in detecting intruder attacks, and it is also used to monitor the traffic in the network. The paper is aimed to control the attacks using the machine learning (ML) technique integrated with the artificial bee colony (ABC) named Group-ABC (G-ABC). The IDS detector has been implemented and further simulation results have been determined using the G-ABC. The evaluation has been carried out using the measures such as precision, recall, accuracy, and F-measure. Different attacks such as user to root (U2R), probe, root to local (R2L), backdoors, worms, and denial-of-service (DoS) attacks have been detected. The simulation analysis is performed using two datasets, namely, the NSL-KDD dataset and UNSW-NB15 dataset, and comparative analysis is performed against the existing work to prove the effectiveness of the proposed IDS. The objective of the work is to determine the intruder attacker system using the deep learning technique.

An Effective Deep Learning Based Multi-Class Classification of DoS and DDoS Attack Detection

In the past few years, cybersecurity is becoming very important due to the rise in internet users. The internet attacks such as Denial of service (DoS) and Distributed Denial of Service (DDoS) attacks severely harm a website or server and make them unavailable to other users. Network Monitoring and control systems have found it challenging to identify the many classes of DoS and DDoS attacks since each operates uniquely. Hence a powerful technique is required for attack detection. Traditional machine learning techniques are inefficient in handling extensive network data and cannot extract high-level features for attack detection. Therefore, an effective deep learning-based intrusion detection system is developed in this paper for DoS and DDoS attack classification. This model includes various phases and starts with the Deep Convolutional Generative Adversarial Networks (DCGAN) based technique to address the class imbalance issue in the dataset. Then a deep learning algorithm based on ResNet-50 extracts the critical features for each class in the dataset. After that, an optimized AlexNet-based classifier is implemented for detecting the attacks separately, and the essential parameters of the classifier are optimized using the Atom search optimization algorithm. The proposed approach was evaluated on benchmark datasets, CCIDS2019 and UNSW-NB15, using key classification metrics and achieved 99.37% accuracy for the UNSW-NB15 dataset and 99.33% for the CICIDS2019 dataset. The investigational results demonstrate that the suggested approach performs superior to other competitive techniques in identifying DoS and DDoS attacks.

A Lightweight Intelligent Network Intrusion Detection System Using One-Class Autoencoder and Ensemble Learning for IoT.

Network intrusion detection technology is key to cybersecurity regarding the Internet of Things (IoT). The traditional intrusion detection system targeting Binary or Multi-Classification can detect known attacks, but it is difficult to resist unknown attacks (such as zero-day attacks). Unknown attacks require security experts to confirm and retrain the model, but new models do not keep up to date. This paper proposes a Lightweight Intelligent NIDS using a One-Class Bidirectional GRU Autoencoder and Ensemble Learning. It can not only accurately identify normal and abnormal data, but also identify unknown attacks as the type most similar to known attacks. First, a One-Class Classification model based on a Bidirectional GRU Autoencoder is introduced. This model is trained with normal data, and has high prediction accuracy in the case of abnormal data and unknown attack data. Second, a multi-classification recognition method based on ensemble learning is proposed. It uses Soft Voting to evaluate the results of various base classifiers, and identify unknown attacks (novelty data) as the type most similar to known attacks, so that exception classification becomes more accurate. Experiments are conducted on WSN-DS, UNSW-NB15, and KDD CUP99 datasets, and the recognition rates of the proposed models in the three datasets are raised to 97.91%, 98.92%, and 98.23% respectively. The results verify the feasibility, efficiency, and portability of the algorithm proposed in the paper.

Weibull Distributive Feature Scaling Multivariate Censored Extreme Learning Classification for Malicious IoT Network Traffic Detection

Malicious Network Traffic identification is the major problems in Internet of Things (IoT) infrastructure. From fast growth and popularization of IoT, threats and attacks in these networks are also increasing correspondingly. To detect Malicious Network Traffic on IoT surroundings, accurate scheme is significant to reduce protection risks on IoT. In this manuscript, Weibull distributive feature scaling multivariate censored extreme learning classification is proposed for Malicious IoT Network Traffic Detection (WDGMS- MCP-ELM-AD-IoT). Initially, the input datas are taken from UNSW-NB15 dataset. The datas are fed to pre-processing. During pre-processing, the redundancy eradication together with replacement of missing value is done for removing uncertainties based on local least squares. The pre-processed data is given to feature selection for choosing relevant features. The process of feature scaling is carried out by Weibull distributive generalized multidimensional scaling (WDGMS). Classification is done with the help of Multivariate Censored phi correlative Extreme learning machines classifier (MCP-ELM) depending on optimum features, which classifies normal or anomalous IoT Network Traffic. Among these, anomalous IoT Network Traffic involves fuzzers, analysis, backdoors, DoS, exploits, generic, reconnaissance, shell code, worm attacks. The proposed approach is activated in Python. The performance metrics, such as accuracy, F-Measure, precision specificity, sensitivity, and false positive rate is evaluated. The simulation of the proposed Weibull distributive feature scaling multivariate censored extreme learning classification for attack detection in IoT (WDGMS- MCP-ELM-AD-IoT) achieves better accuracy 39.45%, 34.58%, 28.29, better sensitivity 26.34%, 21.23%, 27.25%, better specificity 33.67%, 22.37%, 24.78% analyzed to the existing XGBoost-AD-IoT, DNN-AD-IoT and DRaNN-AD-IoT models respectively.