Network Intrusion Detection Dataset Research Articles

Implementation of Cyber Network’s Attacks Detection System with Deep Learning Designing Algorithms

The internet has become indispensable for modern communication, playing a vital role in the development of smart cities and communities. However, its effectiveness is contingent upon its security and resilience against interruptions. Intrusions, defined as unauthorized activities that compromise system integrity, pose a significant threat. These intrusions can be broadly categorized into host intrusions, which involve unauthorized access and manipulation of data within a system, and network intrusions, which target vulnerabilities within the network infrastructure. To mitigate these threats, system administrators rely on Network Intrusion Detection Systems (NIDS) to identify and respond to security breaches. However, designing an effective and adaptable NIDS capable of handling novel and evolving attack strategies presents a significant challenge. This paper proposes a deep learning-based approach for NIDS development, leveraging Self-Taught Learning (STL) and the NSL-KDD benchmark dataset for network intrusion detection. The proposed approach is evaluated using established metrics, including accuracy, F-measure, recall, and precision. Experimental results demonstrate the effectiveness of STL in the 5-class categorization, achieving an accuracy of 79.10% and an F-measure of 75.76%. This performance surpasses that of Softmax Regression (SMR), which attained 75.23% accuracy and a 72.14% F-measure. The paper concludes by comparing the proposed approach's performance with existing state-of-the-art methods.

Exploring Generative Adversarial Networks for Augmenting Network Intrusion Detection Tasks

The advent of generative networks and their adoption in numerous domains and communities have led to a wave of innovation and breakthroughs in artificial intelligence and machine learning. Generative Adversarial Networks (GANs) have expanded the scope of what is possible with machine learning, allowing for new applications in areas such as computer vision, natural language processing, and creative AI. GANs, in particular, have been used for a wide range of tasks, including image and video generation, data augmentation, style transfer, and anomaly detection. They have also been used for medical imaging and drug discovery, where they can generate synthetic data to augment small datasets, reduce the need for expensive experiments, and lower the number of real patients that must be included in medical trials. Given these developments, we propose using the power of generative adversarial networks to create and augment flow-based network traffic datasets. We evaluate a series of GAN architectures, including Wasserstein, conditional, energy-based, gradient penalty, and LSTM GANs. We evaluate their performance on a set of flow-based network traffic data collected from 16 subjects who used their computers for home, work, and study purposes. The performance of these GAN architectures is described according to metrics that involve networking principles, data distribution among a collection of flows, and temporal data distribution. Given the tendency of network intrusion detection datasets to have a very imbalanced data distribution, i.e., a large number of samples in the “normal traffic” category and a comparatively low number of samples assigned to the “intrusion” categories, we test our GANs by augmenting the intrusion data and checking whether this helps intrusion detection neural networks in their task. We publish the resulting UPBFlow dataset and code on GitHub 1 .

ALRN-RCS: Advanced Approach to Network Intrusion Detection Using Attention Long-Term Recurrent Networks and Chaotic Optimization

Detecting intrusions within a network is essential for protecting digital environments; it encompasses the observation and analysis of network traffic to recognize and counteract unauthorized or malicious activities. Conventional methods in network intrusion detection face several challenges such as polymorphic and evasive attacks, scalability issues, and anomaly-based complexity. To address these complexities, this paper proposed a novel method named Attention Long-term Recurrent Network-based Random Chaotic Sine (ALRN-RCS) algorithm for network intrusion detection. In this study, Long Short-Term Memory (LSTM) is utilized to capture complex patterns in network traffic and identify anomalous activities. Also, the attention-based Recurrent Neural Network (RNN) is employed to focus on relevant features within network traffic and enable precise intrusion detection. In this paper, we have incorporated the Chaotic Chimp Sine Cosine optimization algorithm, employing a random update strategy, to optimize hyperparameters and improve the overall efficacy of the proposed approach and the study conducted experiments on the datasets namely the UNSW NB-15, Network Intrusion Detection dataset, and the Segmented Image-based Network Intrusion Detection (SIDD) dataset. Diverse assessment criteria, including accuracy, F1-score, recall, AUC-ROC, and precision, are employed to assess the effectiveness of the ALRN-RCS method and to draw comparisons with established methodologies. The experimental results depict the effectiveness of the ALRN-RCS method for addressing the dynamic and sophisticated nature of modern cyber threats.

Anomaly detection in network traffic with ELSC learning algorithm

AbstractIn recent years, the internet has not only enhanced the quality of our lives but also made us susceptible to high‐frequency cyber‐attacks on communication networks. Detecting such attacks on network traffic is made possible by intrusion detection systems (IDS). IDSs can be broadly divided into two groups based on the type of detection they provide. According to the established rules, the first signature‐based IDS detects threats. Secondly, anomaly‐based IDS detects abnormal conditions in the network. Various machine and deep learning approaches have been used to detect anomalies in network traffic in the past. To improve the detection of anomalies in network traffic, researchers have compared several machine learning models, such as support vector machines (SVM), logistic regressions (LRs), K‐Nearest Neighbour (KNN), Nave Bayes (NBs), and boosting algorithms. The accuracy, precision, and recall of many studies have been satisfactory to an extent. Therefore, this paper proposes an ensemble learning‐based stacking classifier (ELSC) to achieve a better accuracy rate. In the proposed ELSC algorithm, KNN, NB, LR, and Decision Trees (DT) served as the base classifiers, while SVM served as the meta classifier. Based on a Network Intrusion detection dataset provided by Kaggle.com, ELSC is compared to base classifiers such as KNN, NB, LR, DT, SVM, and Linear Discriminate Analysis. As a result of the simulations, the proposed ELBS stacking classifier was found to outperform the other comparative models and converge with an accuracy of 99.4%.

Evaluating the Performance of Classification Algorithms on the UNSW-NB15 Dataset for Network Intrusion Detection

Network intrusion detection is a critical aspect of cybersecurity, aiming to distinguish between normal and malicious network activities. This study evaluates the performance of various machine learning algorithms on the UNSW-NB15 dataset for binary classification of network traffic into normal and attack categories. We employed several preprocessing steps, including handling missing values, encoding categorical features, and addressing class imbalance using a mix of Synthetic Minority Over-sampling Technique (SMOTE) and undersampling. The models evaluated include k-Nearest Neighbors (k-NN), Naive Bayes, Logistic Regression, Support Vector Machines (SVM), and Neural Networks. Our experimental results show that complex models like Neural Networks and SVMs significantly outperform simpler models. The Neural Network model achieved the highest accuracy of 92%, with a precision of 91%, recall of 93%, and an F1-score of 92%. SVM also performed robustly with an accuracy of 90%. Simpler models, while less effective, still achieved respectable performance, with Logistic Regression and k-NN reaching accuracies of 88% and 85%, respectively. The study highlights the importance of comprehensive preprocessing and the implementation of advanced machine learning techniques for effective network intrusion detection. The results suggest that while complex models offer superior detection capabilities, simpler models can still be valuable in resource-constrained environments. Future research should focus on applying these models to real-world data, exploring more advanced neural network architectures, and implementing cost-sensitive learning techniques to further enhance detection performance and efficiency.

A soft prototype-based autonomous fuzzy inference system for network intrusion detection

Nowadays, cyber-attacks have become a common and persistent issue affecting various human activities in modern societies. Due to the continuously evolving landscape of cyber-attacks and the growing concerns around “black box” models, there has been a strong demand for novel explainable and interpretable intrusion detection systems with online learning abilities. In this paper, a novel soft prototype-based autonomous fuzzy inference system (SPAFIS) is proposed for network intrusion detection. SPAFIS learns from network traffic data streams online on a chunk-by-chunk basis and autonomously identifies a set of meaningful, human-interpretable soft prototypes to build an IF-THEN fuzzy rule base for classification. Thanks to the utilization of soft prototypes, SPAFIS can precisely capture the underlying data structure and local patterns, and perform internal reasoning and decision-making in a human-interpretable manner based on the ensemble properties and mutual distances of data. To maintain a healthy and compact knowledge base, a pruning scheme is further introduced to SPAFIS, allowing itself to periodically examine the learned solution and remove redundant soft prototypes from its knowledge base. Numerical examples on public network intrusion detection datasets demonstrated the efficacy of the proposed SPAFIS in both offline and online application scenarios, outperforming the state-of-the-art alternatives.

Unsupervised intrusion detection system for in-vehicle communication networks

In-vehicle communication has been optimized day to day to keep updated of the technologies. Control area network (CAN) is used as a standard communication method because of its efficient and reliable connection. However, CAN is prone to several network level attacks because of its lack in security mechanisms. Various methods have been introduced to incorporate this in CAN. We proposed an unsupervised method of intrusion detection for in-vehicle communication networks by combining the optimal feature extracting ability of autoencoders and more precise clustering using fuzzy C-means (FCM). The proposed method is light weight and requires less computation time. We performed an extensive experiment and achieved an accuracy of 75.51 % with the ML350 in-vehicle intrusion dataset. By experimental result, the proposed method also works better for other intrusion detection problems like wireless intrusion detection datasets such as WNS-DS with accuracy of 84.05 % and network intrusion detection datasets such as KDDCup with accuracy 60.63 % , UNSW_NB15 with accuracy 73.62 % and Information Security Center of Excellence (ISCX) with accuracy 74.83 %. Overall, the proposed method outperforms the existing methods and avoids labeled datasets when training an in-vehicle intrusion detection model. The results of the experiment of our proposed method performed on various intrusion detection datasets indicate that the proposed approach is generalized and robust in detecting intrusions and can be effectively deployed in real time to monitor CAN traffic in vehicles and proactively alert during attacks.

PANACEA: a neural model ensemble for cyber-threat detection

Ensemble learning is a strategy commonly used to fuse different base models by creating a model ensemble that is expected more accurate on unseen data than the base models. This study describes a new cyber-threat detection method, called PANACEA, that uses ensemble learning coupled with adversarial training in deep learning, in order to gain accuracy with neural models trained in cybersecurity problems. The selection of the base models is one of the main challenges to handle, in order to train accurate ensembles. This study describes a model ensemble pruning approach based on eXplainable AI (XAI) to increase the ensemble diversity and gain accuracy in ensemble classification. We base on the idea that being able to identify base models that give relevance to different input feature sub-spaces may help in improving the accuracy of an ensemble trained to recognise different signatures of different cyber-attack patterns. To this purpose, we use a global XAI technique to measure the ensemble model diversity with respect to the effect of the input features on the accuracy of the base neural models combined in the ensemble. Experiments carried out on four benchmark cybersecurity datasets (three network intrusion detection datasets and one malware detection dataset) show the beneficial effects of the proposed combination of adversarial training, ensemble learning and XAI on the accuracy of multi-class classifications of cyber-data achieved by the neural model ensemble.

An optimized neural network for prediction of security threats on software testing

Software testing involves evaluating and confirming a software program or product to ensure it operates according to its intended functionality. Testing offers advantages like bug prevention, reduced development expenses, and improved performance. The problems are dialogue gap, ecological danger, creation of software quickly, cost of operation and upkeep, inadequate assessment, and incorrect testing estimates. The structure was initially educated using internet presentation data that included intrusion information. A novel Dove Swarm-based Deep Neural Method (DSbDNM) with the required traits and stages of processing has been developed. Moving forward, feature extraction and malicious behaviour forecast have both been completed. Also, the different types of assaults and negative behaviours were categorized. The developed prediction model is also examined by initiating and detecting an unidentified assault. Finally, the performance measures' accuracy, error rate, Precision, Recall and f-measure were computed. Moreover, the proposed system implementation is done in Python. Therefore, the proposed work performance can be enhanced and attain high accuracy in low computational time. For the DSbDNM dataset, the designed prototypical achieved 94.65 accuracy, 94.95 precision, 90.16 Recall and 92.02 F-measure for the NF-UQ-NIDS-v2 Dataset. Moreover, the Intrusion Detection Dataset attained an accuracy of 98, Precision of 98.8, Recall of 94.2, and F-score of 96 in the developed model. Subsequently, the Network Intrusion Detection Dataset attained an accuracy of 99, a precision of 99.2, a Recall of 95.8 and an F-measure of 97.1

Multi-Classification and Tree-Based Ensemble Network for the Intrusion Detection System in the Internet of Vehicles.

The Internet of Vehicles(IoV) employs vehicle-to-everything (V2X) technology to establish intricate interconnections among the Internet, the IoT network, and the Vehicle Networks (IVNs), forming a complex vehicle communication network. However, the vehicle communication network is very vulnerable to attacks. The implementation of an intrusion detection system (IDS) emerges as an essential requisite to ensure the security of in-vehicle/inter-vehicle communication in IoV. Within this context, the imbalanced nature of network traffic data and the diversity of network attacks stand as pivotal factors in IDS performance. On the one hand, network traffic data often heavily suffer from data imbalance, which impairs the detection performance. To address this issue, this paper employs a hybrid approach combining the Synthetic Minority Over-sampling Technique (SMOTE) and RandomUnderSampler to achieve a balanced class distribution. On the other hand, the diversity of network attacks constitutes another significant factor contributing to poor intrusion detection model performance. Most current machine learning-based IDSs mainly perform binary classification, while poorly dealing with multiclass classification. This paper proposes an adaptive tree-based ensemble network as the intrusion detection engine for the IDS in IoV. This engine employs a deep-layer structure, wherein diverse ML models are stacked as layers and are interconnected in a cascading manner, which enables accurate and efficient multiclass classification, facilitating the precise identification of diverse network attacks. Moreover, a machine learning-based approach is used for feature selection to reduce feature dimensionality, substantially alleviating the computational overhead. Finally, we evaluate the proposed IDS performance on various cyber-attacks from the in-vehicle and external networks in IoV by using the network intrusion detection dataset CICIDS2017 and the vehicle security dataset Car-Hacking. The experimental results demonstrate remarkable performance, with an F1-score of 0.965 on the CICIDS2017 dataset and an F1-score of 0.9999 on the Car-Hacking dataset. These scores demonstrate that our IDS can achieve efficient and precise multiclass classification. This research provides a valuable reference for ensuring the cybersecurity of IoV.

NIDS-FLGDP: Network Intrusion Detection Algorithm Based on Gaussian Differential Privacy Federated Learning

As a widely used network security defense technology, network intrusion detection has more deep learning methods used to improve the performance of intrusion detection. However, this method requires a large-scale network traffic data set for training, increasing privacy leakage risk. In this paper, a network intrusion detection algorithm based on Gaussian differential privacy federated learning (NIDS-FLGDP) is proposed. NIDS-FLGDP adopts the client–server architecture of federated learning, introduces the differential privacy of the Gaussian mechanism to ensure the security of the calculation process, uses the improved FedAvg algorithm to reduce communication overhead, and uses the improved 1D CNN to participate in collaborative training for the local model. Optimal parameters for Gaussian differential privacy and the optimal number of participating clients were determined from experiments. Model accuracy rates for binary classification and multi-classification training NIDS-FLGDP are 0.97, 0.975, 0.97 and 0.97, 0.985, 0.96, respectively, for KDD CUP99, NSL_KDD, and UNSW_NB15 network intrusion detection datasets. The results show that NIDS-FLGDP improves intrusion detection performance while protecting network traffic privacy compared with the previous methods. Its applicability and effectiveness have been fully verified, which provides a practical reference for the safe processing and analysis of a large number of diversified network traffic data in the future.

Machine Learning-Based Adaptive Synthetic Sampling Technique for Intrusion Detection

Traditional firewalls and data encryption techniques can no longer match the demands of current IoT network security due to the rising amount and variety of network threats. In order to manage IoT network risks, intrusion detection solutions have been advised. Even though machine learning (ML) helps the widely used intrusion detection techniques currently in use, these algorithms struggle with low detection rates and the requirement for extensive feature engineering. The deep learning model for IoT network intrusion detection is a method for traffic anomaly detection that is suggested by this study. To extract the sequence properties of data flow through a CNN, it combines an attention mechanism with a Long Short Term Memory (LSTM) network. This method uses adaptive synthetic sampling (ADASYN) to increase the size of minority-class samples. The proposed models demonstrated acceptable precision and recall for each class when used for binary-class classification, proving their stability and capacity to identify all classes correctly. The MLP classifier’s accuracy, precision, recall, and F1 value were 87%, 89%, 87%, and 89%, respectively, with an AUC score of 0.88. Overall, the proposed models performed well. The attack and all-class models exhibited good AUCs and macro metrics, the same as the proposed MLP classifier, which had an F1 score of 83% and an AUC score of 0.94. Additionally, it trained the MLP classifier and integrated the ADAM optimizer and category cross-entropy loss function for all-class classification. With an AUC value of 94%, it possessed 84% accuracy, 87% precision, 84% recall, and an 83% F1 score. A further indication of the hybrid model’s ability to combine the benefits of both models to improve overall performance was that it regularly outperformed the MLP model. This model’s accuracy and F1 score are better than those of earlier comparable algorithms, according to experimental results using the publicly accessible benchmark dataset for network intrusion detection (NSL–KDD).

Contrastive Learning Over Random Fourier Features for IoT Network Intrusion Detection

IoT networks connect dispersed and heterogeneous devices with high economic impact, making them an important cybersecurity target. The changing nature of cybersecurity attacks requires intrusion detectors with the ability to detect new attacks. Our proposed model is a network intrusion detector created specifically for the needs of IoT networks with the aim of detecting previously unseen attacks. It is based on a shallow neural network architecture following a novel contrastive learning scheme. In this scheme, both the network features and the labels are projected into a common representation (embedding) space where a similarity score is defined. The labels act as a prototype for each type of traffic in embedding space, and classification is based on the proximity of samples to these class prototypes. The dimensionality and structure of the embedding space are critical. In this work, we explore the advantages of having an embedding space with expanded dimensionality using a kernel approximation technique (Random Fourier Features) that is integrated and learned within the neural network. To avoid overfitting, we investigate the importance of various regularization techniques (L2 and contractive). The resulting model is tested against three network intrusion detection datasets to assess its ability to detect known and unknown attacks (zero-shot learning). The experimental results show a higher ability of the proposed model to detect unknown attacks than similar models and alternative machine learning models, in the literature.

Convolutional neural network-based high-precision and speed detection system on CIDDS-001

The growing interconnection of complex infrastructures gives very advanced communication functionalities, which gives a massive increase in connected devices and an associated flow volume. Cloud computing is constantly threatened by sophisticated attacks, which poses challenges for a security system. The Cloud will obsolete existing detection procedures against cyber-attacks where they would not be adapted accordingly. Intrusion detection is a classification problem wherein various machine learning and data mining techniques are applied to classify the network data into normal and attack traffic. Therefore, the proposal of new rapid and effective detection approaches is an absolute necessity. In this work, a proposed framework is a network anomaly detection system based on Deep Learning. The analysis carried out was based on the hyper-parameters of the layers of our model. This proposed model is a combination of two techniques; namely, a reduction of dimensions based on the approach of the main components and the second is based on a dense supervised neural network based on convolution neural network (CNN) for a multi-classification of normal and intrusive events from a recent data-set Coburg Network Intrusion Detection data-set (CIDDS-001). The experiments carried out show that the very precise choice of hyper-parameters gives better results. By running the proposed CNN model, it is capable of detecting attacks with an accuracy of 99.13% and an execution time of 12 s.

Analysis of Neural Network Intrusion Detection Methods and Datasets for their Training

Approaches based on neural network classifiers to the detection of computer attacks are considered. The problems of training such classifiers are discussed. Data sets on computer attacks for wired and wireless systems are considered. The results of evaluating such sets by the degree of imbalance are given. The problems of learning on unbalanced data sets and approaches to balancing the training set in the case of rare attacks, including those using generative adversarial networks, are described.

Exploring Unsupervised Learning with Clustering and Deep Autoencoder to Detect DDoS Attack

<p>With the proliferation of services available on the Internet, network attacks have become one of the seri-ous issues. The distributed denial of service (DDoS) attack is such a devastating attack, which poses an enormous threat to network communication and applications and easily disrupts services. To defense against DDoS attacks effectively, this paper proposes a novel DDoS attack detection method that trains detection models in an unsupervised learning manner using preprocessed and unlabeled normal network traffic data, which can not only avoid the impact of unbalanced training data on the detection model per-formance but also detect unknown attacks. Specifically, the proposed method firstly uses Balanced Itera-tive Reducing and Clustering Using Hierarchies algorithm (BIRCH) to pre-cluster the normal network traf-fic data, and then explores autoencoder (AE) to build the detection model in an unsupervised manner based on the cluster subsets. In order to verify the performance of our method, we perform experiments on benchmark network intrusion detection datasets KDDCUP99 and UNSWNB15. The results show that, compared with the state-of-the-art DDoS detection models that used supervised learning and unsuper-vised learning, our proposed method achieves better performance in terms of detection accuracy rate and false positive rate (FPR).</p> <p> </p>

A Novel Deep Supervised Learning-Based Approach for Intrusion Detection in IoT Systems.

The Internet of Things (IoT) has become one of the most important concepts in various aspects of our modern life in recent years. However, the most critical challenge for the world-wide use of the IoT is to address its security issues. One of the most important tasks to address the security challenges in the IoT is to detect intrusion in the network. Although the machine/deep learning-based solutions have been repeatedly used to detect network intrusion through recent years, there is still considerable potential to improve the accuracy and performance of the classifier (intrusion detector). In this paper, we develop a novel training algorithm to better tune the parameters of the used deep architecture. To specifically do so, we first introduce a novel neighborhood search-based particle swarm optimization (NSBPSO) algorithm to improve the exploitation/exploration of the PSO algorithm. Next, we use the advantage of NSBPSO to optimally train the deep architecture as our network intrusion detector in order to obtain better accuracy and performance. For evaluating the performance of the proposed classifier, we use two network intrusion detection datasets named UNSW-NB15 and Bot-IoT to rate the accuracy and performance of the proposed classifier.

METHODS OF USING THE NEURAL NETWORK TO DETECT NEW TYPES OF NETWORK ATTACKS

The report discusses methods for tuning the hyperparameters of an artificial neural network in a system for detecting and classifying network intrusions. Assuming that the surface of the multidimensional space of hyperparameters is convex, an algorithm is proposed that selects the optimal set of hyperparameters in the search space according to the criterion of maximum accuracy of network intrusion classification. As a result of experiments using three different network intrusion detection data sets – KDDCup 99, NSL-KDD and UNSW-NB15 - the optimal hyperparameters of the MLP neural network were found. It is shown that the proposed method for automatic tuning of neural network hyperparameters makes it possible to achieve high intrusion detection results even on the simplest neural network under the condition of low computational costs. These results are not inferior to the results of modern models, where the hyperparameters were manually selected by the researchers.

Anomaly Detection in Multi-Host Environment Based on Federated Hypersphere Classifier

Detecting anomalous inputs is essential in many mission-critical systems in various domains, particularly cybersecurity. In particular, deep neural network-based anomaly detection methods have been successful for anomaly detection tasks with the recent advancements in deep learning technology. Nevertheless, the existing methods have considered somewhat idealized problems where it is enough to learn a single detector based on a single dataset. In this paper, we consider a more practical problem where multiple hosts in an organization collect their input data, while data sharing among the hosts is prohibitive due to security reasons, and only a few of them have experienced abnormal inputs. Furthermore, the data distribution of the hosts can be skewed; for example, a particular type of input can be observed by a limited subset of hosts. We propose the federated hypersphere classifier (FHC), which is a new anomaly detection method based on an improved hypersphere classifier suited for running in the federated learning framework to perform anomaly detection in such an environment. Our experiments with image and network intrusion detection datasets show that our method outperforms the state-of-the-art anomaly detection methods trained in a host-wise fashion by learning a consensus model as if we have accessed the input data from all hosts but without communicating such data.

Novel hybrid firefly algorithm: an application to enhance XGBoost tuning for intrusion detection classification

The research proposed in this article presents a novel improved version of the widely adopted firefly algorithm and its application for tuning and optimising XGBoost classifier hyper-parameters for network intrusion detection. One of the greatest issues in the domain of network intrusion detection systems are relatively high false positives and false negatives rates. In the proposed study, by using XGBoost classifier optimised with improved firefly algorithm, this challenge is addressed. Based on the established practice from the modern literature, the proposed improved firefly algorithm was first validated on 28 well-known CEC2013 benchmark instances a comparative analysis with the original firefly algorithm and other state-of-the-art metaheuristics was conducted. Afterwards, the devised method was adopted and tested for XGBoost hyper-parameters optimisation and the tuned classifier was tested on the widely used benchmarking NSL-KDD dataset and more recent USNW-NB15 dataset for network intrusion detection. Obtained experimental results prove that the proposed metaheuristics has significant potential in tackling machine learning hyper-parameters optimisation challenge and that it can be used for improving classification accuracy and average precision of network intrusion detection systems.