UNSW-NB15 Dataset Research Articles

Machine learning-based network intrusion detection for big and imbalanced data using oversampling, stacking feature embedding and feature extraction

Cybersecurity has emerged as a critical global concern. Intrusion Detection Systems (IDS) play a critical role in protecting interconnected networks by detecting malicious actors and activities. Machine Learning (ML)-based behavior analysis within the IDS has considerable potential for detecting dynamic cyber threats, identifying abnormalities, and identifying malicious conduct within the network. However, as the number of data grows, dimension reduction becomes an increasingly difficult task when training ML models. Addressing this, our paper introduces a novel ML-based network intrusion detection model that uses Random Oversampling (RO) to address data imbalance and Stacking Feature Embedding based on clustering results, as well as Principal Component Analysis (PCA) for dimension reduction and is specifically designed for large and imbalanced datasets. This model’s performance is carefully evaluated using three cutting-edge benchmark datasets: UNSW-NB15, CIC-IDS-2017, and CIC-IDS-2018. On the UNSW-NB15 dataset, our trials show that the RF and ET models achieve accuracy rates of 99.59% and 99.95%, respectively. Furthermore, using the CIC-IDS2017 dataset, DT, RF, and ET models reach 99.99% accuracy, while DT and RF models obtain 99.94% accuracy on CIC-IDS2018. These performance results continuously outperform the state-of-art, indicating significant progress in the field of network intrusion detection. This achievement demonstrates the efficacy of the suggested methodology, which can be used practically to accurately monitor and identify network traffic intrusions, thereby blocking possible threats.

Intrusion detection method based on improved social network search algorithm

The network security problem in today's world is becoming more and more prominent, and intrusion detection as a branch in the field of network security has been developed tremendously. At present, back propagation (BP) neural network is widely used in intrusion detection. However, its weights and thresholds are randomly initialized, so that fall into local optimal after training. To solve this problem, an intrusion detection model based on improved social network search (ISNS) algorithm to optimize BP neural network is proposed. The model first uses chaotic mapping and elite mechanism to improve the original Social Network Search (SNS) algorithm, and then uses the ISNS algorithm to filter the initial weights and thresholds of the BP neural network, and constructs the neural network with the optimal values to avoid falling into local optimum. The experimental results show that the proposed method solves the problem that the traditional BP neural network is easy to fall into local optimum. At the same time, the ISNS_BP model achieves better classification performance than other models, and the accuracy on the NSL-KDD and UNSW-NB15 datasets reaches 98.62% and 93.97%, respectively.

Bitterling fish optimization (BFO) algorithm

The bitterling fish is a prime example of intelligent behavior in nature for survival. The bitterling fish uses the oyster spawning strategy as their babysitter. The female bitterling fish looks for a male fish stronger than other fish to find the right pair. In order to solve optimization issues, the Bitterling Fish Optimization (BFO) algorithm is modeled in this manuscript based on the mating behavior of these fish. The bitterling fish optimization algorithm is more accurate than the gray wolf optimization algorithm, whale optimization algorithm, butterfly optimization algorithm, Harris Hawks optimization algorithm, and black widow optimization algorithm, according to experiments and implementations on various benchmark functions. Data mining and machine learning are two areas where meta-heuristic techniques are frequently used. In trials, the MLP artificial neural network and a binary version of the BFO algorithm are used to lower the detection error for intrusion traffic. The proposed method's accuracy, precision, and sensitivity index for detecting network intrusion are 99.14%, 98.87%, and 98.85%, respectively, according to experiments on the NSL KDD data set. Compared to machine learning approaches like NNIA, DT, RF, XGBoot, and CNN, the proposed method is more accurate at detecting intrusion. The BFO algorithm is used for feature selection in the UNSW-NB15 dataset, and the tests showed that the accuracy of the proposed method is 96.72% in this dataset. The proposed method of the BFO algorithm is also used to improve Kmeans clustering, and the tests performed on the dataset of covid 19, diabetes, and kidney disease show that the proposed method performs better than iECA*, ECA*, GENCLUST + + (G + +) methods. Deep has KNN, LVQ, SVM, ANN, and KNN.

Detection Model for Ambiguous Intrusion using SMOTE and LSTM for Network Security

In today's interconnected world, networks play a crucial role. Consequently, network security has become increasingly vital. To ensure network security, various methods are employed, including digital signatures, firewalls, and intrusion detection. Among these methods, intrusion detection systems have gained significant popularity due to their ability to identify new attacks. However, the accuracy of these systems still requires further improvement. One of the challenges is the potential bias introduced by using imbalance datasets that contains more information on normal activities than on attacks. To address it, SMOTE method was proposed and additionally, the study explores the use of Long Short-Term Memory (LSTM) for classification purposes. The experiments are conducted using two datasets: UNSW NB-15 and CICIDS 2017. The results obtained demonstrate that the proposed methods achieve an accuracy of 96% with the UNSW NB-15 dataset and 99% with the CICIDS 2017 dataset. These findings indicate an improvement of 3% and 1% respectively compared to existing literature.

Visual Intrusion Detection Based On CBAM-Capsule Networks

Abstract Intrusion detection has become a research focus in internet information security, with deep learning algorithms playing a crucial role in its development. Typically, intrusion detection data are transformed into a two-dimensional matrix by segmenting, stacking and padding them with zeros for input into deep learning models. However, this method consumes computational resources and fails to consider the correlation between features. In this paper, we transform the data into images through visualization operations and propose an information entropy weighted scheme to optimize the collision element problem during the transformation process. This method enhances the correlation between pixel frame features, leading to approximately 2% improvement in accuracy of the classification model when using the generated image samples for detection in experiments. To address the issues of insensitivity to target feature locations and incomplete feature extraction in traditional neural networks, this paper introduces a new network model called CBAM-CapsNet, which combines the advantages of the lightweight Convolutional Block Attention Module and capsule networks. Experimental results on the UNSW-NB15 and IDS-2017 datasets demonstrate that the proposed model achieves accuracies of 92.94% and 99.72%, respectively. The F1 scores obtained are 91.83% and 99.56%, indicating a high level of detection.

CNN-GRU-FF: a double-layer feature fusion-based network intrusion detection system using convolutional neural network and gated recurrent units

Identifying and preventing malicious network behavior is a challenge for establishing a secure network communication environment or system. Malicious activities in a network system can seriously threaten users’ privacy and potentially jeopardize the entire network infrastructure and functions. Furthermore, cyber-attacks have grown in complexity and number due to the ever-evolving digital landscape of computer and network devices in recent years. Analyzing network traffic using network intrusion detection systems (NIDSs) has become an integral security measure in modern networks to identify malicious and suspicious activities. However, most intrusion detection datasets contain imbalance classes, making it difficult for most existing classifiers to achieve good performance. In this paper, we propose a double-layer feature extraction and feature fusion technique (CNN-GRU-FF), which uses a modified focal loss function instead of the traditional cross-entropy to handle the class imbalance problem in the IDS datasets. We use the NSL-KDD and UNSW-NB15 datasets to evaluate the effectiveness of the proposed model. From the research findings, it is evident our CNN-GRU-FF method obtains a detection rate of 98.22% and 99.68% using the UNSW-NB15 and NSL-KDD datasets, respectively while maintaining low false alarm rates on both datasets. We compared the proposed model’s performance with seven baseline algorithms and other published methods in literature. It is evident from the performance results that our proposed method outperforms the state-of-the-art network intrusion detection methods.

Hybrid wrapper feature selection method based on genetic algorithm and extreme learning machine for intrusion detection

Intrusion detection systems play a critical role in the mitigation of cyber-attacks on the Internet of Things (IoT) environment. Due to the integration of many devices within the IoT environment, a huge amount of data is generated. The generated data sets in most cases consist of irrelevant and redundant features that affect the performance of the existing intrusion detection systems (IDS). The selection of optimal features plays a critical role in the enhancement of intrusion detection systems. This study proposes a sequential feature selection approach using an optimized extreme learning machine (ELM) with an SVM (support vector machine) classifier. The main challenge of ELM is the selection of the input parameters, which affect its performance. In this study, the genetic algorithm (GA) is used to optimize the weights of ELM to boost its performance. After the optimization, the algorithm is applied as an estimator in the sequential forward selection (wrapper technique) to select key features. The final obtained feature subset is applied for classification using SVM. The IoT_ToN network and UNSWNB15 datasets were used to test the model's performance. The performance of the model was compared with other existing state-of-the-art classifiers such as k-nearest neighbors, gradient boosting, random forest, and decision tree. The model had the best quality of the selected feature subset. The results indicate that the proposed model had a better intrusion detection performance with 99%, and 86% accuracy for IoT_ToN network dataset and UNSWNB15 datasets, respectively. The model can be used as a promising tool for enhancing the classification performance of IDS datasets.

Effective Multi-Stage Training Model for Edge Computing Devices in Intrusion Detection

Intrusion detection poses a significant challenge within expansive and persistently interconnected environments. As malicious code continues to advance and sophisticated attack methodologies proliferate, various advanced deep learning-based detection approaches have been proposed. Nevertheless, the complexity and accuracy of intrusion detection models still need further enhancement to render them more adaptable to diverse system categories, particularly within resource-constrained devices, such as those embedded in edge computing systems. This research introduces a three-stage training paradigm, augmented by an enhanced pruning methodology and model compression techniques. The objective is to elevate the system's effectiveness, concurrently maintaining a high level of accuracy for intrusion detection. Empirical assessments conducted on the UNSW-NB15 dataset evince that this solution notably reduces the model's dimensions, while upholding accuracy levels equivalent to similar proposals.

A robust intrusion detection system based on a shallow learning model and feature extraction techniques.

The escalating prevalence of cybersecurity risks calls for a focused strategy in order to attain efficient resolutions. This study introduces a detection model that employs a tailored methodology integrating feature selection using SHAP values, a shallow learning algorithm called PV-DM, and machine learning classifiers like XGBOOST. The efficacy of our suggested methodology is highlighted by employing the NSL-KDD and UNSW-NB15 datasets. Our approach in the NSL-KDD dataset exhibits exceptional performance, with an accuracy of 98.92%, precision of 98.92%, recall of 95.44%, and an F1-score of 96.77%. Notably, this performance is achieved by utilizing only four characteristics, indicating the efficiency of our approach. The proposed methodology achieves an accuracy of 82.86%, precision of 84.07%, recall of 77.70%, and an F1-score of 80.20% in the UNSW-NB15 dataset, using only six features. Our research findings provide substantial evidence of the enhanced performance of the proposed model compared to a traditional deep-learning model across all performance metrics.

Integrated threat intelligence platform for security operations in organizations

Organizations have to establish strong security operations to protect their digital assets since cyberattacks are becoming more prevalent and sophisticated. Integrating threat intelligence into security operations is a fundamental strategy for enhancing an organization’s security posture. However, the precision and dependability of the underlying machine learning classifiers employed for analysis determine how successful such platforms really are. In this paper, we leverage the UNSW-NB15 dataset to propose an integrated threat intelligence platform for security operations in organizations. In order to determine which machine learning classifier performs best, we run a variety of classifiers to the dataset, including Ensemble Learning, Stochastic Gradient Descent (SGD), Logistic Regression, and Ridge Classifier. Our findings demonstrate that the Ensemble Learning classifier beats the other classifiers, with accuracy, precision, recall, and F1 score of 97.02%, 98.34%, 99.02% and 98.17% respectively. This suggests that our proposed system is quite good at detecting potential threats and may offer insightful information for security operations in organizations.

A voting gray wolf optimizer-based ensemble learning models for intrusion detection in the Internet of Things

The Internet of Things (IoT) has garnered considerable attention from academic and industrial circles as a pivotal technology in recent years. The escalation of security risks is observed to be associated with the growing interest in IoT applications. Intrusion detection systems (IDS) have been devised as viable instruments for identifying and averting malicious actions in this context. Several techniques described in academic papers are thought to be very accurate, but they cannot be used in the real world because the datasets used to build and test the models do not accurately reflect and simulate the IoT network. Existing methods, on the other hand, deal with these issues, but they are not good enough for commercial use because of their lack of precision, low detection rate, receiver operating characteristic (ROC), and false acceptance rate (FAR). The effectiveness of these solutions is predominantly dependent on individual learners and is consequently influenced by the inherent limitations of each learning algorithm. This study introduces a new approach for detecting intrusion attacks in an IoT network, which involves the use of an ensemble learning technique based on gray wolf optimizer (GWO). The novelty of this study lies in the proposed voting gray wolf optimizer (GWO) ensemble model, which incorporates two crucial components: a traffic analyzer and a classification phase engine. The model employs a voting technique to combine the probability averages of the base learners. Secondly, the combination of feature selection and feature extraction techniques is to reduce dimensionality. Thirdly, the utilization of GWO is employed to optimize the parameters of ensemble models. Similarly, the approach employs the most authentic intrusion detection datasets that are accessible and amalgamates multiple learners to generate ensemble learners. The hybridization of information gain (IG) and principal component analysis (PCA) was employed to reduce dimensionality. The study utilized a novel GWO ensemble learning approach that incorporated a decision tree, random forest, K-nearest neighbor, and multilayer perceptron for classification. To evaluate the efficacy of the proposed model, two authentic datasets, namely, BoT-IoT and UNSW-NB15, were scrutinized. The GWO-optimized ensemble model demonstrates superior accuracy when compared to other machine learning-based and deep learning models. Specifically, the model achieves an accuracy rate of 99.98%, a DR of 99.97%, a precision rate of 99.94%, an ROC rate of 99.99%, and an FAR rate of 1.30 on the BoT-IoT dataset. According to the experimental results, the proposed ensemble model optimized by GWO achieved an accuracy of 100%, a DR of 99.9%, a precision of 99.59%, an ROC of 99.40%, and an FAR of 1.5 when tested on the UNSW-NB15 dataset.

A Network Intrusion Detection Method for Various Information Systems Based on Federated and Deep Learning

Under the premise of ensuring data privacy, traditional network intrusion detection (NID) methods cannot achieve high accuracy for different types of intrusions. A NID method combining transformer and federated learning (FedL) is proposed for this purpose. First, a multi-party collaborative learning framework was built based on FedL, which achieved data exchange and sharing. Then, by introducing the self-attention mechanism (AttM) to improve the traditional transformer, it could quickly converge. Finally, an NID model integrating transformer and FedL was constructed by combining DNN, GRU, and an encoder module composed of improved transformer, achieving accurate detection of network intrusion. The proposed NID method was compared with the other three methods. The results show that the proposed method has the highest NID accuracy and F1 score on the NSL-KDD and UNSW-NB15 dataset, with the highest accuracy reaching 99.65% and 89.25%, while the F1 score has the highest accuracy, reaching 99.45% and 88.13%, outperforming the other three comparative algorithms in terms of performance.

Intelligent Intrusion Detection System for Internet of Things using Machine Learning Techniques

The Internet of Things (IoT) is an interconnected framework of computer devices, software, and sensors to work collectively to automate and monitor different tasks and procedures without human intervention. IoTs provide benefits and ease in performing operations for various industries and home and health stakeholders. Still, it faces security threats and challenges that make the IoT framework unreliable regarding security and privacy. Intrusion detection systems have been designed and deployed to overcome these security challenges. Over the past few years, for making the IDS intelligent artificial intelligence has been integrated with them. Besides this, different feature selection techniques are also being used to make the datasets appropriate for Machine Learning (ML) classifiers. This research uses a filter-based feature selection technique to select the important features from two datasets, UNSW-NB15 and NSL-KDD. After that, datasets with selected features are given to the classifiers as input. This work also evaluates different ML classifiers such as Multiple Layers Perceptron (MLP), CatBoost, Light Gradient Boosting (LGBM), Extra-Tree Classifier (ETC), Random Forest (RF), and K-nearest neighbor (KNN). Different metrics, Accuracy, Precision, Recall, F1, Detection Rate, and FAR, have been used for performance analysis. Evaluation results proved the LGBM classifier effective among other classifiers with 98.70% accuracy on the UNSW-NB15 dataset and 99.8% on the NSL-KDD dataset after feature selection with mean absolute difference (MAD) feature selection technique. Results are also compared with existing models to provide the efficiency of the proposed model.

IoT Intrusion Detection System based on Machine Learning Algorithms usingthe UNSW-NB15 dataset

The evolution of communications systems with the advent of IoT is leading to an increase in attacks against them. This is due to the fact that the security of connected objects in the IoT is an emerging area which still requires preventive solutions against various attacks. At the network security level, Intrusion Detection Systems (IDS) are used to analyze network data and detect abnormal behavior in the network. In this work, we implemented different machine learning models to build an intrusion detection system based on the UNSW NB15 dataset. To do this, we did data cleaning and feature engineering on the data in the pre-processing phase. Then we used various models such as logistic regression, support vector machine(SVM)classifier, decision tree, random forest, eXtremeGradient Boosting(XGBoost)in order to predict attacks. Finally, an intrusion detection system is trained on various machine learning algorithms and we selected the most effective model. Experiments were carried out on the UNSW-NB15 dataset and subsequently we compared other machine learning algorithms, and this meansthat the random forest model on important parameters has a clear advantage in the detection of rare abnormal behaviors

Network anomaly detection and performance evaluation of Convolutional Neural Networks on UNSW-NB15 dataset

The present study uses the benchmark UNSW-NB15 datasets to detect network anomalies using the proposed deep learning-based Convolutional Neural Network (CNN) model. Several studies have already worked on network anomaly detection with some limits. However, earlier research has used old datasets and obtained limited accuracy. In addition, previous studies could not detect the latest malicious activities. Therefore, in the present study, firstly, we implemented the machine learning-based random forest method to diminish the dimensionality of the dataset and pick the most notable features. In this case, the random forest method has set only fifteen features from the forty-one and reduced the complexity of the data. Subsequently, the CNN model was trained and tested on the UNSW-NB15 dataset with 99.00% testing accuracy. The performance of the offered CNN model was assessed using precision, recall and the f1-values. The present study’s outcome can be used in real-time malicious activity detection.

An innovative network intrusion detection system (NIDS): Hierarchical deep learning model based on Unsw-Nb15 dataset

With the increasing prevalence of network intrusions, the development of effective network intrusion detection systems (NIDS) has become crucial. In this study, we propose a novel NIDS approach that combines the power of long short-term memory (LSTM) and attention mechanisms to analyze the spatial and temporal features of network traffic data. We utilize the benchmark UNSW-NB15 dataset, which exhibits a diverse distribution of patterns, including a significant disparity in the size of the training and testing sets. Unlike traditional machine learning techniques like support vector machines (SVM) and k-nearest neighbors (KNN) that often struggle with limited feature sets and lower accuracy, our proposed model overcomes these limitations. Notably, existing models applied to this dataset typically require manual feature selection and extraction, which can be time-consuming and less precise. In contrast, our model achieves superior results in binary classification by leveraging the advantages of LSTM and attention mechanisms. Through extensive experiments and evaluations with state-of-the-art ML/DL models, we demonstrate the effectiveness and superiority of our proposed approach. Our findings highlight the potential of combining LSTM and attention mechanisms for enhanced network intrusion detection.

A proposed method for detecting network intrusion using an ensemble learning (stacking -voting) approach with unbalanced data

The use of computer networks has become necessary in most human activities. However, these networks are exposed to potential threats affecting the confidentiality, integrity, and availability of data. Nowadays, the security of computer networks is based on tools and software such as antivirus software. Among the techniques used for machine protection, firewalls, data encryption, etc., were mentioned. These techniques constitute the first phase of computer network security. However, they remain limited and do not allow for full network protection. In this paper, a Network Intrusion Detection System (NIDS) was proposed for binary classification. This model was based on ensemble learning techniques, where the base models were carefully selected in a first layer. Several machine learning algorithms were individually studied to choose the best ones based on multiple metrics, including calculation speed. The SMOTE technique was used to balance the data, and cross-validation was employed to mitigate overfitting issues. Regarding the approaches used in this research, a stacking and voting model was employed, trained, and tested on a UNSW-NB15 dataset. The stacking classifier achieved a higher accuracy of 96 %, while the voting approach attained 95,6 %

Explosive Cyber Security Threats During COVID-19 Pandemic and a Novel Tree-Based Broad Learning System to Overcome

The rapid spread of the COVID-19 has not only affected personal health and economy, but also revolutionized people's lifestyles. As more people turn to work and socialize online, the development of unmanned technologies based on the Internet of Vehicles (IoV), such as unmanned delivery, unmanned vehicles, unmanned transportation, etc., will become an inevitable trend. However, all kinds of intelligent terminals for unmanned equipment require a large amount of data interaction with devices such as cloud servers, mobile terminals, and roadside terminals, which poses cyber security risks. Furthermore, the outbreak of COVID-19 has prompted people to put forward higher demands for the security of network communications. Unfortunately, the current intrusion detection methods based on machine learning still have weaknesses such as low accuracy and low efficiency when faced with unbalanced data distribution. To solve the above problems, we propose a novel Tree-based BLS (TBLS) intrusion detection method according to the idea of ensemble learning and decision tree (CART and J48). The performance of TBLS was tested on the NSL-KDD dataset and the UNSW-NB15 dataset respectively, which contain a variety of malicious traffic types for attacks on the IoV. The results show that our proposed method can achieve higher accuracy rate and lower false alarm rate, compared with the existing 16 solutions.

Advancing Network Intrusion Detection Systems with Machine Learning Techniques

This paper presents an approach to enhancing the efficiency and effectiveness of Network Intrusion Detection Systems (NIDS) by leveraging Machine Learning (ML) techniques, specifically Decision Trees (DT), Naïve Bayes (NB), and Support Vector Machine (SVM). The proposed methodology involves a comprehensive evaluation and comparison of these algorithms using the NSL-KDD and UNSW-NB15 datasets, employing standard evaluation metrics such as accuracy, precision, recall, and F1-score. The study identifies the most effective algorithm for practical NIDS deployment. By providing actionable insights and recommendations for implementing the most suitable ML algorithm, this research contributes significantly to the ongoing efforts in strengthening network security against evolving cyber threats.

BLSAE-SNIDS: A Bi-LSTM sparse autoencoder framework for satellite network intrusion detection

Due to disparities in tolerance, resource availability, and acquisition of labeled training data between satellite-terrestrial integrated networks (STINs) and terrestrial networks, the application of traditional terrestrial network intrusion detection techniques to satellite networks poses significant challenges. This paper presents a satellite network intrusion detection system named Bi-LSTM sparse selfencoder (BLSAE-SNIDS) to address this issue. Through the development of an innovative unsupervised training Bi-LSTM stacked self-encoder, BLSAE-SNIDS facilitates feature extraction from satellite network traffic, diminishes dimensionality, considerably reduces training and testing durations, and enhances the attack prediction accuracy of the classifier. To assess the efficacy of the proposed model, we conduct comprehensive experiments utilizing STIN and UNSW-NB15 datasets. The results obtained from the STIN dataset demonstrate that BLSAE-SNIDS achieves 99.99% accuracy with reduced computational and transmission overheads alongside enhanced flexibility. Furthermore, results from the UNSW-NB15 dataset exhibit BLSAE-SNIDS? proficiency in detecting various network intrusion attacks efficiently. These findings indicate that BLSAE-SNIDS suits general satellite security networks and offers a novel approach to designing security systems for polar satellite networks, thus exhibiting practical utility.