KDD Datasets Research Articles

An effective networks intrusion detection approach based on hybrid Harris Hawks and multi-layer perceptron

This paper proposes an Intrusion Detection System (IDS) employing the Harris Hawks Optimization algorithm (HHO) to optimize Multilayer Perceptron learning by optimizing bias and weight parameters. HHO-MLP aims to select optimal parameters in its learning process to minimize intrusion detection errors in networks. HHO-MLP has been implemented using EvoloPy NN framework, an open-source Python tool specialized for training MLPs using evolutionary algorithms. For purposes of comparing the HHO model against other evolutionary methodologies currently available, specificity and sensitivity measures, accuracy measures, and mse and rmse measures have been calculated using KDD datasets. Experiments have demonstrated the HHO MLP method is effective at identifying malicious patterns. HHO-MLP has been tested against evolutionary algorithms like Butterfly Optimization Algorithm (BOA), Grasshopper Optimization Algorithms (GOA), and Black Widow Optimizations (BOW), with validation by Random Forest (RF), XGBoost. HHO-MLP showed superior performance by attaining top scores with accuracy rate of 93.17%, sensitivity level of 89.25%, and specificity percentage of 95.41%.

Mitigation of Attacks via Improved Network Security in IoT Network using Machine Learning

In this paper, we develop a support vector machine (SVM) based attack mitigation technique from the IoT network. The SVM aims to classify the features related to the attacks based on pre-processed and feature extracted information. The simulation is conducted in terms of accuracy, precision, recall and f-measure over KDD datasets. The results show that the proposed SVM classifier obtains high grade of classification accuracy in both training and testing datasets.

Detection and Investigation of DDoS Attacks in Network Traffic using Machine Learning Algorithms

The Internet of Things (IoT) represents the start of a new age in information technology (IoT). Objects (things) such as smart TVs, telephones, and smartwatches may now connect to the Internet. New services and software improve many consumers' lives. Online lessons based on COVID-9 are also included in child education devices. Multiple device integration is becoming more widespread as the Internet of Things (IoT) grows in popularity. While IoT devices offer tremendous advantages, they may also create network disruptions. This article summarises current DDoS intrusion detection research utilizing machine learning methods. This study examines the detection performance of DDoS attacks utilizing WEKA tools using the most recent NSL KDD datasets. Logistic Regression (LR), Naive Bayes (NB), SVM, K-NN, Decision Tree (DT), and Random Forest (RF) are examples of Machine Learning algorithms. Using K-Nearest Neighbors in the presented assessment (K-NN), accuracy was attained. Finally, future research questions are addressed.

Denial of Service Intrusion Detection System (IDS) Based on Naïve Bayes Classifier using NSL KDD and KDD Cup 99 Datasets

Intrusion Detection Systems (IDS) become necessary to protect data from intruders and reduce the damage of the information system and networks especially in cloud environment which is next generation Internet based computing system that supplies customizable services to the end user to work or access to the various cloud applications. This paper concentrates the views to be noted that; the attacks in cloud environment have high rates of Denial of service (DoS) attacks compared with the usual network environment. This paper will introduce Naïve Bayes (NB) Classifier supported by discrete the continuous feature and feature selection methods to classify network events as an attack (DoS, Probe, R2L and U2R) or normal. The influence of use all features and use set of features by applying two methods of feature selection methods has been studied in this paper. The performance of the proposed system was evaluated by using KDD 99 CUP and NSL KDD Datasets, and from experimental works the results are; proposal improves the performance of NIDS in term of accuracy and detecting DOS attack, where it detected 94%, 97% and 98% of DoS attacks for three experimental test datasets in KDD Cup 99 dataset when used twelve features selected by gain ratio, while in NSL KDD Dataset the accuracy of detecting DoS is 86%, 87% and 88% for three experimental test datasets when select only ten features by applied gain ratio.

Similarity Based Feature Transformation for Network Anomaly Detection

The fundamental objective behind any network intrusion detection system is to automate the detection process whenever intrusions occur in the network. The problem of the network anomaly detection is to determine, if the network incoming traffic is legitimate (or) anomalous. Automated detection systems designed to identify incoming anomalous traffic patterns usually apply widely used machine learning techniques. However, irrespective of any system model which is developed to identify anomalous traffic, all these models requires comparing anomalous and normal traffic patterns. Such comparisons implicitly depend on the ability of the underlying machine learning model to gauge the similarity between a known legitimate observation and the target. The efficiency of any network anomalous detection system depends on the use of distance (or) similarity measures and how they are actually applied. A novel distance function which can be applied to determine the similarity between two conditional feature pattern vectors is an important contribution of present research. Feature dimensionality is another important issue for any machine learning algorithm. In the present work, feature reduction is achieved using the proposed feature transformation technique. However, our approach for feature transformation uses the proposed gaussian distance function to achieve dimensionality reduction to represent the original input dataset in the new transformation space. We have also proposed new computation expressions for determining equivalent deviation and threshold in gaussian space. Experiments are performed on KDD and NSL-KDD datasets by considering widely applied classifier algorithms in various state-of-art research contributions. For performance validation of machine learning models, k-fold cross validation is applied by setting k to 10 through considering evaluation parameters such as accuracy, precision and recall. Experiment results have proved that our approach for anomaly detection that applies the proposed feature transformation technique proved comparatively better to detection methods CANN, GARUDA, and UTTAMA addressed in the recent research literature.

GOAMLP: Network Intrusion Detection With Multilayer Perceptron and Grasshopper Optimization Algorithm

In this paper, an intrusion detection system is introduced that uses data mining and machine learning concepts to detect network intrusion patterns. In the proposed method, an artificial neural network (ANN) is used as a learning technique in intrusion detection. The metaheuristic algorithm with the swarm-based approach is used to reduce intrusion detection errors. In the proposed method, the Grasshopper Optimization Algorithm (GOA) is used for better and more accurate learning of ANNs to reduce intrusion detection error rate. The role of the GOAMLP algorithm is to minimize the intrusion detection error in the neural network by selecting useful parameters such as weight and bias. Our implementation in MATLAB software and using the KDD and UNSW datasets show that the proposed method detects abnormal, malicious traffic and attacks with high accuracy. The GOAMLP method outperforms and is more accurate than the existing state-of-the-art techniques such as RF, XGBoost, and embedded learning of ANN with BOA, HHO, and BWO algorithms in network intrusion detection.

An improved ensemble approach for effective intrusion detection

Nowadays, one critical challenge of cybersecurity administrators is the protection of online resources from network intrusions. Despite several academic and industry research initiatives, full protection of online resources from these network intrusions is not feasible. Therefore, several techniques have been developed that use network audit data for accurate detection of network intrusions effectively and efficiently and are used in network intrusion detection systems (NIDSs). But, most of NIDSs reported low detection accuracy with high false alarm rate and provide a single solution that lacks in classification trade-offs. In this paper, the authors present a hybrid approach of multi-objective genetic algorithm and neural networks for creating a set of ensemble solutions for detecting network intrusions effectively. The proposed approach works in two phases that initially creates a set of non-dominating solutions or Pareto optimal solutions of base techniques and then creates ensemble solutions. In the outcome of individual solutions or models in the ensemble are aggregated using most popular method of majority voting. The proposed hybrid approach is evaluated using benchmark datasets of NSL_KDD and ISCX-2012 datasets for intrusion detection. The evaluation results using benchmark datasets demonstrate that the proposed hybrid approach enables detecting network intrusions effectively as compared to conventional ensemble approaches, namely bagging and boosting. The resultant ensemble solutions are non-dominating and provide classification trade-offs for cybersecurity administrators. The results also show that the proposed hybrid approach detects both minority and majority intrusion types accurately. The proposed hybrid approach demonstrated a detection accuracy of 97% and 88% with FPR of 2.4% and 2% for ISCX-2012 and NSL_KDD datasets, respectively.

A SURVEY ON DIFFERENT MACHINE LEARNING ALGORITHMS AND WEAK CLASSIFIERS BASED ON KDD AND NSL-KDD DATASETS

Network intrusion detection often finds a difficulty in creating classifiers that could handle unequal distributed attack categories. Generally, attacks such as Remote to Local (R2L) and User to Root (U2R) attacks are very rare attacks and even in KDD dataset, these attacks are only 2% of overall datasets. So, these result in model not able to efficiently learn the characteristics of rare categories and this will result in poor detection rates of rare attack categories like R2L and U2R attacks. We even compared the accuracy of KDD and NSL-KDD datasets using different classifiers in WEKA.

A Survey on Different Machine Learning Algorithms and Weak Classifiers Based on KDD and NSL-KDD Datasets

A Survey on Different Machine Learning Algorithms and Weak Classifiers Based on KDD and NSL-KDD Datasets

GARUDA: Gaussian dissimilarity measure for feature representation and anomaly detection in Internet of things

The objective of any anomaly detection system is to efficiently detect several types of malicious traffic patterns that cannot be detected by conventional firewall systems. Designing an efficient intrusion detection system has three primary challenges that include addressing high dimensionality problem, choice of learning algorithm, and distance or similarity measure used to find the similarity value between any two traffic patterns or input observations. Feature representation and dimensionality reduction have been studied and addressed widely in the literature and have also been applied for the design of intrusion detection systems (IDS). The choice of classifiers is also studied and applied widely in the design of IDS. However, at the heart of IDS lies the choice of distance measure that is required for an IDS to judge an incoming observation as normal or abnormal. This challenge has been understudied and relatively less addressed in the research literature both from academia and from industry. This research aims at introducing a novel distance measure that can be used to perform feature clustering and feature representation for efficient intrusion detection. Recent studies such as CANN proposed feature reduction techniques for improving detection and accuracy rates of IDS that used Euclidean distance. However, accuracies of attack classes such as U2R and R2L are not significantly promising. Our approach GARUDA is based on clustering feature patterns incrementally and then representing features in different transformation space through using a novel fuzzy Gaussian dissimilarity measure. Experiments are conducted on both KDD and NSL-KDD datasets. The accuracy and detection rates of proposed approach are compared for classifiers such as kNN, J48, naive Bayes, along with CANN and CLAPP approaches. Experiment results proved that proposed approach resulted in the improved accuracy and detection rates for U2R and R2L attack classes when compared to other approaches.

Network Intrusion Detection System (NIDS) in Cloud Environment based on Hidden Naïve Bayes Multiclass Classifier

Cloud Environment is next generation internet based computing system that supplies customiza-ble services to the end user to work or access to the various cloud applications. In order to provide security and decrease the damage of information system, network and computer system it is im-portant to provide intrusion detection system (IDS. Now Cloud environment are under threads from network intrusions, as one of most prevalent and offensive means Denial of Service (DoS) attacks that cause dangerous impact on cloud computing systems. This paper propose Hidden naïve Bayes (HNB) Classifier to handle DoS attacks which is a data mining (DM) model used to relaxes the conditional independence assumption of Naïve Bayes classifier (NB), proposed sys-tem used HNB Classifier supported with discretization and feature selection where select the best feature enhance the performance of the system and reduce consuming time. To evaluate the per-formance of proposal system, KDD 99 CUP and NSL KDD Datasets has been used. The experi-mental results show that the HNB classifier improves the performance of NIDS in terms of accu-racy and detecting DoS attacks, where the accuracy of detect DoS is 100% in three test KDD cup 99 dataset by used only 12 feature that selected by use gain ratio while in NSL KDD Dataset the accuracy of detect DoS attack is 90 % in three Experimental NSL KDD dataset by select 10 fea-ture only.

Intrusion detection systems using a hybrid SVD-based feature extraction method

Intrusion detection systems (IDSs) are able to diagnose network anomalies with the help of machine learning techniques. This paper presents a novel singular value decomposition (SVD)-based method that creates a new feature, which is applied to an IDS. The main goal is to build an effective model on datasets, which have the least possible number of features. Using the least possible number of features is inevitable in case of improving the efficiency and de-escalating the effect of curse of dimensionality in datasets with large number of features. The proposed method combines the SVD method with four classification algorithms; decision tree, Naive Bayes, neural networks and SVM, to obtain a high accuracy in anomaly detection. This method is applied on the KDD CUP 99 and NSL_KDD datasets. Results of simulations indicate that the proposed method provides a considerable improvement in accuracy, compared with ordinary feature selection methods.

Combining analytic kernel models for energy-efficient data modeling and classification

Energy-efficient computing has now become a key challenge not only for data-center operations, but also for many other energy-driven systems, with the focus on reducing of all energy-related costs, and operational expenses, as well as its corresponding and environmental impacts. However, current intelligent data models are typically performance driven. For instance, most data-driven machine-learning approaches are often known to require high computational cost in order to find the global optima. Designing more accurate intelligent data models to satisfy the market needs will hence lead to a higher likelihood of energy waste due to the increased computational cost. This paper thus introduces an energy-efficient framework for large-scale data modeling and classification/prediction. It can achieve a predictive accuracy comparable to or better than the state-of-the-art machine-learning models, while at the same time, maintaining a low computational cost when dealing with large-scale data. The effectiveness of the proposed approaches has been demonstrated by our experiments with two large-scale KDD data sets: Mtv-1 and Mtv-2.

Practical Comparison Between Genetic Algorithm and Clonal Selection Theory on KDD Dataset

This paper compares between two models: Common Genetic algorithm and the new Clonal selection theory in the field of Intrusion Detection. Genetic algorithms (GA) which is a model of genetic evolution, while Clonal selection theory (CST) is from models of the natural immune system NIS, the two models are from two different fields of Artificial Intelligence AI but they have portion of shared operations and objectives. The comparison to be done by applying the two models on some records of Knowledge Discovery and Data mining tools which is known by the name KDD data sets (its records the data of the interring packets to the computer system from the internet), to produce population ( in case of GA) or antibodies (in case of CST) can recognize these abnormal records.

On the use of ROC analysis for the optimization of abstaining classifiers

Classifiers that refrain from classification in certain cases can significantly reduce the misclassification cost. However, the parameters for such abstaining classifiers are often set in a rather ad-hoc manner. We propose a method to optimally build a specific type of abstaining binary classifiers using ROC analysis. These classifiers are built based on optimization criteria in the following three models: cost-based, bounded-abstention and bounded-improvement. We show that selecting the optimal classifier in the first model is similar to known iso-performance lines and uses only the slopes of ROC curves, whereas selecting the optimal classifier in the remaining two models is not straightforward. We investigate the properties of the convex-down ROCCH (ROC Convex Hull) and present a simple and efficient algorithm for finding the optimal classifier in these models, namely, the bounded-abstention and bounded-improvement models. We demonstrate the application of these models to effectively reduce misclassification cost in real-life classification systems. The method has been validated with an ROC building algorithm and cross-validation on 15 UCI KDD datasets.