KDD Dataset Research Articles

Improved minority attack detection in Intrusion Detection System using efficient feature selection algorithms

AbstractMachine Learning and Data Mining algorithms are used extensively to enhance the performance of Intrusion Detection Systems. The number of training instances and the dimensionality of data are crucial factors affecting the performance of the model built during the training of any supervised learning algorithms. A sufficient proportion of instances having relevant features from all classes of attacks and normal traffic are considered most desirable while building the classification model that classifies the network traffic into attack and normal. This paper proposes a methodology to improve the accuracy of the model by giving importance to the relevant features that can contribute to model building. The feature selection using correlation‐based and information gain‐based techniques during training and testing contributes much to the detection of stealthier attacks and minority attacks. Then the features of the less detected attacks are identified as the second phase of the filter that is used to improve the performance. The relevant features of stealthy attacks are identified based on the correlation of corresponding features of the attack and normal data as the attacks are made stealthy mostly by making it resemble the normal traffic. Finally, the attacks that are rarely found in the training data are oversampled to improve their detection. CICIDS 2017 data set is employed as it comprises stealthier attacks generated using modern tools. NSL KDD data set is also used for evaluation to compare the proposed work with existing literature as it is used in most of the available literature. The results show superior performance with an accuracy of 99.8%, false positive rate of 0.2%, and a detection rate and 99.8%.

Multi Level Deep Learning Model for Network Anomaly Detection

The increasing use of internet-based solutions and services in private and corporate sectors has resulted in a significant increase in personal internet involvement. This shift, concurrently exposed a heightened susceptibility to potential vulnerabilities, given the ability of malicious actors to exploit external networks, network services, or corporate infrastructures utilized for personal purposes. In recent times, there has been considerable interest in harnessing deep learning methodologies for enhancing cybersecurity, owing to their utilization of sophisticated learning algorithms for addressing pertinent online security challenges. Machine Learning (ML) and Deep Learning (DL) paradigms have been extensively applied across diverse dimensions of cybersecurity, encompassing tasks such as vulnerability assessment, malware classification, spam detection, and spoofing identification.
 In this paper, for hierarchical intrusion detection is proposed a novel multi-stage approach, The proposed system comprises two distinct classification modalities: multi-class classification and binary classification, contingent upon the nature of the attack within the dataset. the KDD99 dataset was leveraged to assess the classification performance of the proposed model. Both classification approaches involve main preprocessing steps, such as feature selection, feature normalization, building a Convolutional Neural Network (CNN) classifier apply on KDD99 dataset, deploying the CNN classifier for anomaly detection.

Algorithms for Cybersecurity in CAVs Based On Deep Learning and Their Applications

This paper is concerned with the study of some novel techniques that using artificial intelligence to protect networks of CAVs from cyberattacks, where we use some machine learning algorithms to detect attacks and compare the machine learning algorithms used for this in terms of accuracy and required operating time. Also, WEKA tool will be used for the desired comparison, as the experiments are carried out on a new dataset, which is a dataset abbreviated from the KDD99 dataset.

A MACHINE LEARNING BASED CLASSIFICATION AND PREDICTION TECHNIQUE FOR DDOS ATTACKS

Distributed network attacks are referred to, usually, as Distributed Denial of Service (DDoS) attacks. These attacks take advantage of specific limitations that apply to any arrangement asset, such as the framework of the authorized organization's site. In the existing research study, the author worked on an old KDD dataset. It is necessary to work with the latest dataset to identify the current state of DDoS attacks. This paper, used a machine learning approach for DDoS attack types classification and prediction. For this purpose, used Random Forest and XGBoost classication algorithms. To access the research proposed a complete framework for DDoS attacks prediction. For the proposed work, the UNWS-np-15 dataset was extracted from the GitHub repository and Python was used as a simulator. After applying the machine learning models, we generated a confusion matrix for identification of the model performance. In the first classification, the results showed that both Precision (PR) and Recall (RE) are 89% for the Random Forest algorithm. The average Accuracy (AC) of our proposed model is 89% which is superb and enough good. In the second classification, the results showed that both Precision (PR) and Recall (RE) are approximately 90% for the XGBoost algorithm. The average Accuracy (AC) of our suggested model is 90%. By comparing our work to the existing research works, the accuracy of the defect determination was significantly improved which is approximately 85% and 79%, respectively.

Detecting Network Intrusion Anomalies with RNN (LSTM)- Based Deep Learning Models

As more and more network devices and services are used, there is a growingneed for security measures as hackers attempt to take down or steal data from target systems. One of the key components of network perimetersecurity isthe intrusion detectionsystem (IDS), which looks through operating system logs or network traffic packets to identify threats. Although previous research has shown the effectiveness of several machine learning algorithms, relatively few of these studies have made use of the time-series information included in network traffic data. Neural network-based methods have notincorporated category data either. In this paper, we offer models for network intrusion detection based on categorical information using the embedding technique and sequential information using the long short-term memory (LSTM) network. Using the extensive networktraffic dataset KDD CUP 99, we have tested the models. The findings of the trial confirm that the suggested strategy improves performance, with a 99.72% binary classification accuracy. Keywords: Machine Learning, Deep Learning ,Network Intrusion Detection, Long Short-Term Memory ,Feature Selection

Novel Network Intrusion Detection Based on Feature Filtering Using FLAME and New Cuckoo Selection in a Genetic Algorithm

Recently, networks have faced a significant challenge in terms of security due to constant unauthorized access by hackers, resulting in the compromise of network user data. To enhance network security, there are various approaches that can be employed, including the utilization of firewalls, encryption, and antivirus software. Among these methods, one type of system that can be implemented is an intrusion detection system (IDS), which actively monitors the network to identify any intrusions. In order to effectively detect any unauthorized or malicious activities, sophisticated techniques such as genetic algorithms, cuckoo searches, and FLAME are employed. This research proposes a novel IDS that aims to improve the detection of intrusions. The proposed IDS initially conducts feature filtration using fuzzy clustering through the local approximation of the membership algorithm (FLAME), which effectively reduces the number of features that need to be analyzed and processed. Subsequently, the system detects attacks by implementing an extended classifier system (XCS) that incorporates a genetic algorithm (GA) to enhance the accuracy of intrusion detection. By incorporating the cuckoo search and selection within GA, while considering different crossover and mutation probabilities, instead of solely relying on traditional GA, it is anticipated that intrusion detection accuracy will be improved. To evaluate the performance of the proposed IDS, it was tested on the KDD99 dataset and compared with several other IDSs that were also tested on the same dataset. The experimental results clearly indicate that the proposed IDS significantly enhances the detection rate (DR) and accuracy while also significantly reducing the false alarm rate (FAR) and these impressive results were 100%, 99.99% and 0.05% respectively. The performance comparisons reveal that, overall, the proposed IDS outperforms several existing IDSs in terms of intrusion detection performance.

Optimized machine learning enabled intrusion detection 2 system for internet of medical things

This research paper proposes an intrusion detection system that combines particle swarm optimization and AdaBoost algorithms to classify and detect malware-related records in innovative health app platforms. The research employs the NSL KDD dataset, consisting of 125,973 instances and 41 features, with data split into training (20 %) and testing (80 %) sets. A feature selection process using Particle Swarm Optimization identifies 12 relevant features for intrusion detection. The intrusion detection system effectively categorizes various attack types, including Denial of Service (DoS), User-to-root (U2R), Root-to-local (R2L), and Probe attacks. Regarding classifier performance, AdaBoost exhibits the highest recall value (0.966667), highlighting its strong intrusion detection capabilities. The experimental results demonstrate that the PSO-AdaBoost approach achieves superior accuracy, precision, and recall in intrusion detection. By integrating machine learning-based intrusion detection systems into smart health apps, healthcare providers can enhance patient care, reduce costs, and ensure the confidentiality of sensitive medical information. This research highlights the potential of ML-IDSs in strengthening the security infrastructure surrounding medical IoT devices and improving patient outcomes in the interconnected world of the Internet of Medical Things.

Dynamic Firewall Rule Building Engine for Hybrid Cloud

Growth in the cloud computing resource management aspects have also increases the risks associated with the services hosted on the cloud.The foundational challenges are firstly to accommodate the ease of access without violating the security aspects and the time constraints to provide service responses on time, to provide the security to the active services. A good number of researches are observed to achieve the best firewall security to the hosted services. Nonetheless, the existing methods are often criticised for higher complexity and the non-performing aspects for the newer attack types. Henceforth, this work aims to solve the existing research bottlenecks. The proposed work firstly aims to solve the higher complexity of the deployed strategy by reducing the attribute sets with a sense of accuracy, time complexity and information loss. Further, this work proposes a dynamic firewall rule engine design strategy for detection of the attacks using the thresholding method. The proposed algorithms are testing on the benchmarked KDD dataset and as outcome a nearly 99.7% accuracy is observed and the time complexity is reduced to nearly 40%. Hence, this proposed work demonstrates a state-of-the-art technology for firewall design for hybrid cloud and shall be considered as a new benchmark in this domain of the research.

Comparative Analysis of Data Security and Cloud Storage Models Using NSL KDD Dataset

Cloud computing is becoming increasingly important in many enterprises, and researchers are focusing on safeguarding cloud computing. Due to the extensive variety of service options it offers, A significant amount of interest from the scientific community has been focused on cloud computing. The two biggest problems with cloud computing are security and privacy. The key challenge is maintaining privacy, which expands rapidly with the number of users. A perfect security system must efficiently ensure each security aspect. This study provides a literature review illustrating the security in the cloud with respect to privacy, integrity, confidentiality and availability, and it also provides a comparison table illustrating the differences between various security and storage models with respect to the approaches and components of the models offered. This study also compares Naïve Bayes and SVM on the accuracy, recall and precision metrics using the NSL KDD dataset.

Digital Investigation of Network Traffic Using Machine Learning

In this study, an intelligent system that can gather and process network packets is built. Machine learning techniques are used to create a traffic classifier that divides packets into hazardous and non-malicious categories. The system utilizing resources was previously classified using a number of conventional techniques; however, this strategy adds machine learning., a study area that is currently active and has so far yielded promising results. The major aims of this paper are to monitor traffic, analyze incursions, and control them. The flow of data collection is used to develop a traffic classification system based on features of observed internet packets. This classification will aid IT managers in recognizing the vague assault that is becoming more common in the IT industry The suggested methods described in this research help gather network data and detect which threat was launched in a specific network to distinguish between malicious and benign packets. This paper’s major goal is to create a proactive system for detecting network attacks using classifiers based on machine learning that can recognize new packets and distinguish between hostile and benign network packets using rules from the KDD dataset. The algorithm is trained to employ the characteristics of the NSL-KDD dataset.

Anomaly Detection Model of Network Dataflow Based on an Improved Grey Wolf Algorithm and CNN

With the popularization of the network and the expansion of its application scope, the problem of abnormal network traffic caused by network attacks, malicious software, traffic peaks, or network device failures is becoming increasingly prominent. This problem not only leads to a decline in network performance and service quality but also may pose a serious threat to network security. This paper proposes a hybrid data processing model based on deep learning for network anomaly detection to improve anomaly detection performance. First, the Grey Wolf optimization algorithm is improved to select high-quality data features, which are then converted to RGB images and input into an anomaly detection model. An anomaly detection model of network dataflow based on a convolutional neural network is designed to recognize network anomalies, including DoS (Denial of Service), R2L (Remote to Local), U2R (User to Root), and Probe (Probing). To verify the effectiveness of the improved Grey Wolf algorithm and the anomaly detection model, we conducted experiments on the KDD99 and UNSW-NB15 datasets. The proposed method achieves an average detection rate of 0.986, which is much higher than all the counterparts. Experimental results show that the accuracy and the detection rates of our method were improved, while the false alarm rate has been reduced, proving the effectiveness of our approach in network anomaly classification tasks.

A Hypertuned Lightweight and Scalable LSTM Model for Hybrid Network Intrusion Detection

Given the increasing frequency of network attacks, there is an urgent need for more effective network security measures. While traditional approaches such as firewalls and data encryption have been implemented, there is still room for improvement in their effectiveness. To effectively address this concern, it is essential to integrate Artificial Intelligence (AI)-based solutions into historical methods. However, AI-driven approaches often encounter challenges, including lower detection rates and the complexity of feature engineering requirements. Finding solutions to overcome these hurdles is critical for enhancing the effectiveness of intrusion detection systems. This research paper introduces a deep learning-based approach for network intrusion detection to overcome these challenges. The proposed approach utilizes various classification algorithms, including the AutoEncoder (AE), Long-short-term-memory (LSTM), Multi-Layer Perceptron (MLP), Linear Support Vector Machine (L-SVM), Quantum Support Vector Machine (Q-SVM), Linear Discriminant Analysis (LDA), and Quadratic Discriminant Analysis (QDA). To validate the effectiveness of the proposed approach, three datasets, namely IOT23, CICIDS2017, and NSL KDD, are used for experimentation. The results demonstrate impressive accuracy, particularly with the LSTM algorithm, achieving a 97.7% accuracy rate on the NSL KDD dataset, 99% accuracy rate on the CICIDS2017 dataset, and 98.7% accuracy on the IOT23 dataset. These findings highlight the potential of deep learning algorithms in enhancing network intrusion detection. By providing network administrators with robust security measures for accurate and timely intrusion detection, the proposed approach contributes to network safety and helps mitigate the impact of network attacks.

An effective technique for detecting minority attacks in NIDS using deep learning and sampling approach

Anomaly-based intrusion detection system have been consistently used in business organizations and military to detect a breach in network by identifying any activity that deviates from the baseline pattern. In this paper, we propose an effective intrusion detection technique to identify and predict the minority attacks with three layers. Here, the first layer utilizes a Weighted Deep Neural Network (WDNN) for identifying the suspicious traffic samples in network and it is passed to the second layer. Layer 2 classifies the traffic samples as normal or majority and minority attacks using Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM). Any traffic sample classified as minority attack is sent to Layer 3 that utilizes XGBoost algorithm. Layer 3 classifies the samples into their respective minority attack classes. To boost the detection rate of minority attacks, system employs a One-Sided Selection under-sampling algorithm to remove noisy samples from the majority attack classes. An Adaptive Synthetic (ADASYN) oversampling algorithm generates synthetic samples of minority attack classes. To evaluate the system, the datasets namely NSL KDD, CICIDS-2017 and CIDDS 001 dataset are used. The system attained an overall accuracy of 97.94% on NSL KDD dataset, 98.3% on CICIDS-2017 dataset and 97.9% on CIDDS 001 dataset.

Anomaly Detection in 6G Networks Using Machine Learning Methods

While the cloudification of networks with a micro-services-oriented design is a well-known feature of 5G, the 6G era of networks is closely related to intelligent network orchestration and management. Consequently, artificial intelligence (AI), machine learning (ML), and deep learning (DL) have a big part to play in the 6G paradigm that is being imagined. Future end-to-end automation of networks requires proactive threat detection, the use of clever mitigation strategies, and confirmation that 6G networks will be self-sustaining. To strengthen and consolidate the role of AI in safeguarding 6G networks, this article explores how AI may be employed in 6G security. In order to achieve this, a novel anomaly detection system for 6G networks (AD6GNs) based on ensemble learning (EL) for communication networks was redeveloped in this study. The first stage in the EL-ADCN process is pre-processing. The second stage is the feature selection approach. It applies the reimplemented hybrid approach using a comparison of the ensemble learning and feature selection random forest algorithms (CFS-RF). NB2015, CIC_IDS2017, NSL KDD, and CICDDOS2019 are the three datasets, each given a reduced dimensionality, and the top subset characteristic for each is determined separately. Hybrid EL techniques are used in the third step to find intrusions. The average voting methodology is employed as an aggregation method, and two classifiers—support vector machines (SVM) and random forests (RF)—are modified to be used as EL algorithms for bagging and adaboosting, respectively. Testing the concept of the last step involves employing classification forms that are binary and multi-class. The best experimental results were obtained by applying 30, 35, 40, and 40 features of the reimplemented system to the three datasets: NSL_KDD, UNSW_NB2015, CIC_IDS2017, and CICDDOS2019. For the NSL_KDD dataset, the accuracy was 99.5% with a false alarm rate of 0.0038; the accuracy was 99.9% for the UNSW_NB2015 dataset with a false alarm rate of 0.0076; and the accuracy was 99.8% for the CIC_IDS2017 dataset with a false alarm rate of 0.0009. However, the accuracy was 99.95426% for the CICDDOS2019 dataset, with a false alarm rate of 0.00113.

Research on Adaptive 1DCNN Network Intrusion Detection Technology Based on BSGM Mixed Sampling.

The development of internet technology has brought us benefits, but at the same time, there has been a surge in network attack incidents, posing a serious threat to network security. In the real world, the amount of attack data is much smaller than normal data, leading to a severe class imbalance problem that affects the performance of classifiers. Additionally, when using CNN for detection and classification, manual adjustment of parameters is required, making it difficult to obtain the optimal number of convolutional kernels. Therefore, we propose a hybrid sampling technique called Borderline-SMOTE and Gaussian Mixture Model (GMM), referred to as BSGM, which combines the two approaches. We utilize the Quantum Particle Swarm Optimization (QPSO) algorithm to automatically determine the optimal number of convolutional kernels for each one-dimensional convolutional layer, thereby enhancing the detection rate of minority classes. In our experiments, we conducted binary and multi-class experiments using the KDD99 dataset. We compared our proposed BSGM-QPSO-1DCNN method with ROS-CNN, SMOTE-CNN, RUS-SMOTE-CNN, RUS-SMOTE-RF, and RUS-SMOTE-MLP as benchmark models for intrusion detection. The experimental results show the following: (i) BSGM-QPSO-1DCNN achieves high accuracy rates of 99.93% and 99.94% in binary and multi-class experiments, respectively; (ii) the precision rates for the minority classes R2L and U2R are improved by 68% and 66%, respectively. Our research demonstrates that BSGM-QPSO-1DCNN is an efficient solution for addressing the imbalanced data issue in this field, and it outperforms the five intrusion detection methods used in this study.

A Comparative Study for SDN Security Based on Machine Learning

In the past decade, traditional networks have been utilized to transfer data between more than one node. The primary problem related to formal networks is their stable essence, which makes them incapable of meeting the requirements of nodes recently inserted into the network. Thus, formal networks are substituted by a Software Defined Network (SDN). The latter can be utilized to construct a structure for intensive data applications like big data. In this paper, a comparative investigation of Deep Neural Network (DNN) and Machine Learning (ML) techniques that uses various feature selection techniques is undertaken. The ML techniques employed in this approach are decision tree (DT), Naïve Bayes (NB), Support Vector Machine (SVM). The proposed approach is tested experimentally and evaluated using an available NSL–KDD dataset. This dataset includes 41 features and 148,517 samples. To evaluate the techniques, several estimation measurements are calculated. The results prove that DT is the most accurate and effective approach. Furthermore, the evaluation measurements indicate the efficacy of the presented approach compared to earlier studies.

Secure Cloud SDN Educational Management with Internet + Learning Management System

The education management model refers to the system and processes that colleges and universities use to manage and oversee their academic programs and operations. However, with the advent of digital technologies, there has been a growing trend towards the Internet+ college education management model, which integrates digital technologies into all aspects of college education management. This model includes the use of online learning platforms and tools, such as learning management systems (LMS), to deliver courses and manage student progress. It also includes the use of digital technologies for administrative tasks such as admissions, enrolment, and financial aid. However, the educational management model is subjected to the challenge of security for educational data management. Hence, this paper constructed a secure framework model of the Ethereum SDN Cloud Architecture (ESDNarc). The ESDNarc model uses the Software-defined Network (SDN) for the decentralized management of the network, secure transactions, and improved efficiency. The ESDNarch model incorporates the SDN with the cryptography scheme the secure the data. The constructed model uses the double-hashing Elliptical Curve Cryptography (DHECC) for the data stored in the Ethereum blockchain. The performance of the constructed model is evaluated with the KDD data set. Simulation analysis stated that ESDNarch significantly increases the data security in the cloud model for the attacks in the network.

Towards unsupervised keyphrase extraction via an autoregressive approach

Keyphrase extraction is a technique used to capture the core information of documents and is an upstream task for advanced information retrieval systems, particularly in the academic realm. Current unsupervised methods are primarily built on a score-and-rank framework with a consistent inability to acquire mutual information between extracted keyphrases, especially with graph-based models. Utilizing the autoregressive structure that is typically used in sequence-to-sequence text generation models, we propose a plug-and-play optimizer named C-Decay that can be integrated into any graph-based unsupervised keyphrase extraction model for a stable performance boost, and that mitigates the bias of certain semantically or lexically dominant tokens by optimizing the origin score distribution output by graph-based models directly. The architecture of C-Decay includes the keyphrase pool, the gain vector and the decay factor, where the keyphrase pool is designed to realize an autoregressive structure and the gain vector and the decay factor are the optimization operator. Herein, we examine three graph-based models integrated with C-Decay, and the experiment is conducted on four datasets KDD, Semeval, Nguyen, and Krapivin. Moreover, we prove that C-Decay can improve accuracy and F-Measure by an average of approximately 50% and 20%, respectively.

A big data analytics for DDOS attack detection using optimized ensemble framework in Internet of Things

Internet of Things (IoT) devices significantly threaten tech-dependant businesses and communities. With the rapid integration of IoT devices into local networks, a system to identify and prevent malicious activity on these devices is essential. These devices' inherent mobility, nature, and intelligence highlight the importance of establishing secure boundaries to safeguard against potential threats. Along with expanding the Internet's ability to handle traffic, the number of IoT devices linked to the Internet has also grown. Due to this modification, conventional methodologies and outdated data processing methods are no longer adequate for attack detection. Due to the growth in network traffic, it is more challenging to identify IoT attacks and malicious data in their early phases. This paper proposes a paradigm for identifying malicious network traffic. The framework uses two well-known classification-based techniques for detecting malicious network traffic, including a Support Vector Machine (SVM) and deep neural network model, i.e., a Convolutional Neural Network (CNN) embedded with a Gated Recurrent Unit (GRU), which is tuned using a Slime Mould Algorithm (SMA) for improved accuracy of 98.45% and 94.84%. The proposed technique is used to analyse the KDD dataset, and the results are assessed in terms of specificity, prediction time, training, and accuracy. The experimental findings suggest that our proposed model can detect DDoS attacks in less time than current solutions.

Network Anomaly Detection System using Deep Learning with Feature Selection Through PSO

The more computer systems that communicate and cooperate, the more crucial it is to make our lives simpler. At the same time, it highlights faults that people are unable to correct. Due to faults, cyber-security procedures are required to communicate data. Secure communication requires both the installation of security measures and the development of security measures to address changing security concerns. In this study, it is suggested that network intrusion detection systems be able to adapt and be resilient. This could be done by using deep learning architectures. Deep learning is used in this article to find and group network attacks. There are some tools that can help intrusion detection systems that are more flexible learn to recognise new or zero-day network behaviour features, which can help them get rid of bad guys and make it less likely that they'll get into your network. The model's efficacy was tested using the KDD dataset, which combines real-world network traffic with fake attack operations.