KDD Cup Dataset Research Articles

A Hybrid Modified Deep Learning Architecture for Intrusion Detection System with Optimal Feature Selection

With the exponentially evolving trends in technology, IoT networks are vulnerable to serious security issues, allowing intruders to break into networks without authorization and manipulate the data. Their actions can be recognized and avoided by using a system that can detect intrusions. This paper presents a hybrid intelligent system and inverted hour-glass-based layered network classifier for feature selection and classification processes, respectively. To accomplish this task, three different datasets have been utilized in the proposed model for identifying old and new attacks. Moreover, a hybrid optimization feature selection technique has been implemented for selecting only those features that can enhance the accuracy of the detection rate. Finally, the classification is performed by using the inverted hour-glass-based layered network model in which data are up-sampled with the increase in the number of layers for effective training. Data up-sampling is performed when small subset of datapoints are observed for any class, which in turn helps in improving the accuracy of the proposed model. The proposed model demonstrated an accuracy of 99.967%, 99.567%, and 99.726% for NSL-KDD, KDD-CUP99, and UNSW NB15 datasets, respectively, which is significantly better than the traditional CNID model. These results demonstrate that our model can detect different attacks with high accuracy and is expected to show good results for new datasets as well. Additionally, to reduce the computational cost of the proposed model, we have implemented it on CPU-based core i3 processors, which are much cheaper than GPU processors.

ASTREAM: Data-Stream-Driven Scalable Anomaly Detection With Accuracy Guarantee in IIoT Environment

Intrusion detection exerts a crucial influence on securing the IIoT driven by anomaly detection approaches. Dissimilar with the static data, the intrusion detection data is in the form of a dynamic data stream possessing the properties of infiniteness, correlations, and data distribution change.However, these properties cause some issues for current anomaly detection approaches.Firstly, it is impractical to save the whole dataset due to the infiniteness.Secondly, the correlations are hardly considered.Thirdly, the data distribution change cant be appropriately handled due to a lack of model update and change detection strategy. Thus, we propose ASTREAM(anomaly detection in data streams), a novel anomaly detection approach that merges sliding window, model update, and change detection strategies into LSHiForest to achieve accurate and efficient anomaly detection with better scalability. ASTREAM has the following characteristics:(a)the sliding window can be utilized to handle the infiniteness of data streams;(b)the introduced PCA can consider the correlations between different attributes;(c)the change detection and model update can detect data distribution change in time and train the new model. Comprehensive experiments are implemented on the KDDCUP99 dataset to validate ASTREAM performance.Experiment results reveal that ASTREAM outperforms baselines in aspects of accuracy and efficiency and has better scalability.

DDoS Risk Assesment using Machine Learning Techniques

The result of exploiting holes in safeguarding Internet of Things (IoT) devices, the amount of cyber- attacks and data breaches has skyrocketed across various corporations, companies, and sectors. Because its capacity to extract and learn deep characteristics of known assaults and detect novel attacks without the need for manual feature engineering, machine learning is used in cyber-attack detection. Despite the use of improved Machine Learning (ML) techniques for intrusion detection, the assault remains a huge danger to the Internet. The primary target of this research is to identify and detect assaults on the network. The expansion of social networks is now increasing on a daily basis. However, detecting the assaults is a difficult task. By examining the information in the KDDCUP Dataset, this project will dynamically detect the attack. The method of feature scaling is employed to standardize a range of independent variables or data constituents. The feature reduction PCA technique is used to locate the directions of highest variance in high-dimensional data and project it onto a new subspace with the same or less dimensions than the original one. Finally, the ML classification technique is used to categorise the data. Based on the assaults and the typical event, the final report is created.. Key Words: DDOS, Machine learning, Cyber Attacks,PCA Algorithm.

BAE: Anomaly Detection Algorithm Based on Clustering and Autoencoder

In this paper, we propose an outlier-detection algorithm for detecting network traffic anomalies based on a clustering algorithm and an autoencoder model. The BIRCH clustering algorithm is employed as the pre-algorithm of the autoencoder to pre-classify datasets with complex data distribution characteristics, while the autoencoder model is used to detect outliers based on a threshold. The proposed BIRCH-Autoencoder (BAE) algorithm has been tested on four network security datasets, KDDCUP99, UNSW-NB15, CICIDS2017, and NSL-KDD, and compared with representative algorithms. The BAE algorithm achieved average F-scores of 96.160, 81.132, and 91.424 on the KDDCUP99, UNSW-NB15, and CICIDS2017 datasets, respectively. These experimental results demonstrate that the proposed approach can effectively and accurately detect anomalous data.

A Lightweight Intelligent Network Intrusion Detection System Using One-Class Autoencoder and Ensemble Learning for IoT.

Network intrusion detection technology is key to cybersecurity regarding the Internet of Things (IoT). The traditional intrusion detection system targeting Binary or Multi-Classification can detect known attacks, but it is difficult to resist unknown attacks (such as zero-day attacks). Unknown attacks require security experts to confirm and retrain the model, but new models do not keep up to date. This paper proposes a Lightweight Intelligent NIDS using a One-Class Bidirectional GRU Autoencoder and Ensemble Learning. It can not only accurately identify normal and abnormal data, but also identify unknown attacks as the type most similar to known attacks. First, a One-Class Classification model based on a Bidirectional GRU Autoencoder is introduced. This model is trained with normal data, and has high prediction accuracy in the case of abnormal data and unknown attack data. Second, a multi-classification recognition method based on ensemble learning is proposed. It uses Soft Voting to evaluate the results of various base classifiers, and identify unknown attacks (novelty data) as the type most similar to known attacks, so that exception classification becomes more accurate. Experiments are conducted on WSN-DS, UNSW-NB15, and KDD CUP99 datasets, and the recognition rates of the proposed models in the three datasets are raised to 97.91%, 98.92%, and 98.23% respectively. The results verify the feasibility, efficiency, and portability of the algorithm proposed in the paper.

Three-Way Selection Random Forest Optimization Model for Anomaly Traffic Detection

With the application and development of Internet technology, network traffic is growing rapidly, and the situation of network security is becoming more and more serious. As an important way to protect network security, abnormal traffic detection has been paid more and more attention. In this paper, the uncertainty of the samples in the abnormal traffic detection dataset is studied. Combining the three-way decision idea with the random forest algorithm, a three-way selection random forest optimization model for abnormal traffic detection is proposed. Firstly, the three-way decision idea is integrated into the random selection process of feature attributes, and the attribute importance based on decision boundary entropy is calculated. The feature attributes are divided into the normal domain, abnormal domain, and uncertain domain, and the three-way attribute random selection rules are designed to randomly select the feature attributes that conform to the rules from different domains. Secondly, the classifier evaluation function is constructed by combining pure accuracy and diversity, and the anomaly traffic detection base classifier with a high evaluation value is selected for integration to eliminate the unstable factors caused by randomness in the process of base classifier generation. Thirdly, the optimal node weight combination of the base classifier is obtained by iterative calculation of the gray wolf optimization algorithm to further improve the prediction effect and robustness of the model. Finally, the model is applied to the abnormal traffic detection dataset. The experimental results show that the prediction accuracy of the three-way selection random forest optimization model on CIC-IDS2017, KDDCUP99, and NSLKDD datasets is 96.1%, 95.2%, and 95.3%, respectively, which has a better detection effect than other machine learning algorithms.

Detection of Distributed Denial of Service (DDoS) attacks using convolutional neural networks

The rapid evolution of the Internet has brought tremendous benefits to the world at large. To effectively leverage on the importance of the internet, there is the need for a secured and reliable network. But currently, there are lots of network attacks against network infrastructures. One of such attack is the Distributed Denial of Service (DDoS) attacks which is an attempt by hackers to deny authorized users access internet service availability using many attack machines. In this paper, a Convolutional Neural Network based detection model is proposed to proffer solution to the challenges of DDoS attacks. The dataset for the modelling was sourced from the KDD Cup-99 Dataset. The evaluation of the experiment conducted was based on three standard metrics of accuracy, sensitivity and specificity. The experimental results showed that the developed model had an accuracy of 99.72%, specificity of 99.69% and sensitivity of 99.71%. Furthermore, the performance of the model was compared with other existing traditional learning models, the results indicated that the model presented in this work performed significantly better.

Divergence Based Feature Selection for Pattern Recognizing of the Performance of Intrusion Detection in Mobile Communications Merged with the Computer Communication Networks

The Feature Selection concept is the procedure where in the data is simplified removing the irrelevant features. Divergence method is another strategy of method where in the relation among the attributes and class is measured to understand their contribution to performance. Nowadays the mobile network is integrated with the other networks like the computer networks transmitting all kinds of data leading to attacks in networks known as intrusion in computer networks equally applicable to mobile communications. So in this paper the intrusion detection method involving the other other mode of communication is considered in mobile interaction network. The proposed algorithm performs feature selection using the divergence evaluation method to reduce the feature set. A 10% KDDCUP99 data set was used for the evaluation of the proposed algorithm, and performance metrics were evaluated using the C4.5 classifier. The metrics TPR, FPR and consistency were compared with the mutual information based DMIFSA, RMIFSA, MMIFSA methods and the proposed method is implemented on Python 3.8 that proved to achieve the high accuracy of 99.94% as compared to other methods and also reduce the redundant features. The consistency in Accuracy is maintained almost from 4 features to 10 features in proposed method as compared to other methods that indicates the stability of the system is achieved.

Intrusion Detection System Based on Machine Learning Algorithms:( SVM and Genetic Algorithm)

The widespread utilization of the internet and computer systems has resulted in notable security concerns, characterized by a surge in intrusions and vulnerabilities. Malicious users manipulate internal systems, resulting in the exploitation of software flaws and default setups. With the integration of the internet into society, there is an emergence of new risks such as viruses and worms, which highlights the importance of implementing robust security measures. Intrusion detection systems (IDS) are security technologies utilized to monitor and analyze network traffic or system activity with the purpose of identifying hostile behavior. This article presents a proposed method for detecting intrusion in network traffic using a hybrid approach, which combines a genetic algorithm and an SVM algorithm. The model underwent training and testing on the KDDCup99 dataset, with a reduction in features from 42 to 29 using the hybrid approach. The results demonstrated that throughout the system testing, it exhibited a remarkable accuracy of 0.999. Additionally, it achieved a true positive value of 0.9987 and a false negative rate of 0.012.

Analysis of Network Traffic Measurements using Machine Learning

In recent times, an exponential increase in internet traffic has been observed as a result of advancing development of the Internet of Things, mobile networks with sensors, and communication functions within various devices. Further, the COVID-19 pandemic has inevitably led to an explosion of social network traffic. Within this context, considerable attention has been drawn to research on network traffic analysis based on machine learning.In this paper, we design and develop a new machine learning framework for network traffic analysis whereby normal and abnormal traffic is distinguished from one another. To achieve this, we combine together well-known machine learning algorithms and network traffic analysis techniques. Using one of the most widely used datasets KDD CUP'99 in the Weka and Apache Spark environments, we compare and investigate results obtained from time series type analysis of various aspects including malicious codes, feature extraction, data formalization, network traffic measurement tool implementation. Experimental analysis showed that while both the logistic regression and the support vector machine algorithm were excellent for performance evaluation, among these, the logistic regression algorithm performs better. The quantitative analysis results of our proposed machine learning framework show that this approach is reliable and practical. In addition, we determined that the framework developed in the Apache Spark environment exhibits a much faster processing speed in the Spark environment than in Weka as there are more datasets used to create and classify machine learning models

Detection of Real-Time Malicious Intrusions and Attacks in IoT Empowered Cybersecurity Infrastructures

Computer viruses, malicious, and other hostile attacks can affect a computer network. Intrusion detection is a key component of network security as an active defence technology. Traditional intrusion detection systems struggle with issues like poor accuracy, ineffective detection, a high percentage of false positives, and an inability to handle new types of intrusions. To address these issues, we propose a deep learning-based novel method to detect cybersecurity vulnerabilities and breaches in cyber-physical systems. The proposed framework contrasts the unsupervised and deep learning-based discriminative approaches. This paper presents a generative adversarial network to detect cyber threats in IoT-driven IICs networks. The results demonstrate a performance increase of approximately 95% to 97% in terms of accuracy, reliability, and efficiency in detecting all types of attacks with a dropout value of 0.2 and an epoch value of 25. The output of well-known state-of-the-art DL classifiers achieved the highest true rate (TNR) and highest detection rate (HDR) when detecting the following attacks: (BruteForceXXS, BruteForceWEB, DoS_Hulk_Attack, and DOS_LOIC_HTTP_Attack) on the NSL-KDD, KDDCup99, and UNSW-NB15 datasets. It also maintained the confidentiality and integrity of users’ and systems’ sensitive information during the training and testing phases.

Classification Model for IDS Using Auto Cryptographic Denoising Technique

Intrusion detection systems (IDS) are one of the most promising ways for securing data and networks; In recent decades, IDS has used a variety of categorization algorithms. These classifiers, on the other hand, do not work effectively unless they are combined with additional algorithms that can alter the classifier’s parameters or select the optimal sub-set of features for the problem. Optimizers are used in tandem with classifiers to increase the stability and with efficiency of the classifiers in detecting invasion. These algorithms, on the other hand, have a number of limitations, particularly when used to detect new types of threats. In this paper, the NSL KDD dataset and KDD Cup 99 is used to find the performance of the proposed classifier model and compared; These two IDS dataset is preprocessed, then Auto Cryptographic Denoising (ACD) adopted to remove noise in the feature of the IDS dataset; the classifier algorithms, K-Means and Neural network classifies the dataset with adam optimizer. IDS classifier is evaluated by measuring performance measures like f-measure, recall, precision, detection rate and accuracy. The neural network obtained the highest classifying accuracy as 91.12% with drop-out function that shows the efficiency of the classifier model with drop-out function for KDD Cup99 dataset. Explaining their power and limitations in the proposed methodology that could be used in future works in the IDS area.

Inverse halftoning based on sparse representation with boosted dictionary

Under the imbalanced dataset, the performance of the base-classifier, the computing method of weight of base-classifier and the selection method of the base-classifier have a great impact on the performance of the ensemble classifier. In order to solve above problem to improve the generalization performance of ensemble classifier, a selective ensemble learning algorithm based on under-sampling for imbalanced dataset is proposed. First, the proposed algorithm calculates the number K of under-sampling samples according to the relationship between class sample density. Then, we use the improved K-means clustering algorithm to under-sample the majority class samples and obtain K cluster centers. Then, all cluster centers (or the sample of the nearest cluster center) are regarded as new majority samples to construct a new balanced training subset combine with the minority class?s samples. Repeat those processes to generate multiple training subsets and get multiple base-classifiers. However, with the increasing of iterations, the number of base-classifiers increase, and the similarity among the base-classifiers will also increase. Therefore, it is necessary to select some base-classifier with good classification performance and large difference for ensemble. In the stage of selecting base-classifiers, according to the difference and performance of base-classifiers, we use the idea of maximum correlation and minimum redundancy to select base-classifiers. In the ensemble stage, G-mean or F-mean is selected to evaluate the classification performance of base-classifier for imbalanced dataset. That is to say, it is selected to compute the weight of each base-classifier. And then the weighted voting method is used for ensemble. Finally, the simulation results on the artificial dataset, UCI dataset and KDDCUP dataset show that the algorithm has good generalization performance on imbalanced dataset, especially on the dataset with high imbalance degree.

Fuzzy based intrusion detection system in MANET

The rapid development and popularization of the Mobile Adhoc Network (MANET) have brought many security issues in network. Intrusion detection system, an effective security technology which can efficiently detect malicious data in complex network environments and ensure computer network security. Because of the complexity of MANET, traditional Intrusion Detection system IDSs are ineffective in this new context, several methods including Support Vector Machine (SVM) have been used to detect intrusion. Most existing technologies strive for low execution times and energy efficiency while achieving accurate detection rates. To overcome these disadvantages, a novel Principal Component Analysis based Fuzzy Extreme learning machine (PCA-FELM) has been proposed in this paper. Initially, the features are extracted by using Principal Component Analysis and then the extracted features are classified by using Fuzzy Extreme Learning Machine. The proposed PCA-FELM is implemented using MAT LAB simulator. The proposed PCA-FELM is compared with existing methods such as DBN-IDS, GOA-SVM and SDAE-ELM and the proposed method achieves higher accuracy of 99.08% than other existing methods. Experiments on the Knowledge Discovery and Data Mining Tools Competition, KDD Cup99 dataset show that the proposed PCA-FELM model have superior performance than other existing techniques.

Intrusion detection system in distributed cloud computing: Hybrid clustering and classification methods

Cloud Computing is popular nowadays due to its storage and data access services. Security and privacy are prime concerns when network threats increase. Cloud computing offers organizations and enterprises a scalable, flexible, and cost-effective infrastructure to store data on the Web. An anomaly-based IDS implementation protects the integrity of the data in a database by identifying and quarantining records when something appears to have changed unexpectedly. Machine learning based clustering and classification methods are used for anomaly based IDS attack classification and scalability in advanced networking environments. Machine learning is a fast, efficient, and adaptable approach to develop intrusion detection models that can deal with emerging threats, i.e., known and unknown attacks (including zero-day attacks). This paper proposes an efficient Hybrid clustering and classification models for implementing an anomaly-based IDS for malicious attack type classifications such as normal (no intrusion), DoS, Probe, U2R, and R2L using threshold-based functions, and the results are tested with two different threshold values (e), 0.01 & 0.5. The experiments have been performed on two tested datasets, namely, NSL-KDD and KDDcup99. Detection rate, False alarm ratio, and accuracy have been used to study the performance of the proposed methodology. After applying the proposed approach, the K-means with random forest has been shown at two different threshold values to have a better classification accuracy, detection rate, and false alarm rate of 99.85%, 99.78% and 0.09% on the NSL-KDD dataset and 98.27%, 98.12% and 2.08% respectively on the KDDcup99 dataset.

Deep Learning-Based Intrusion Detection Methods in Cyber-Physical Systems: Challenges and Future Trends

A cyber-physical system (CPS) integrates various interconnected physical processes, computing resources, and networking units, as well as monitors the process and applications of the computing systems. Interconnection of the physical and cyber world initiates threatening security challenges, especially with the increasing complexity of communication networks. Despite efforts to combat these challenges, it is difficult to detect and analyze cyber-physical attacks in a complex CPS. Machine learning-based models have been adopted by researchers to analyze cyber-physical security systems. This paper discusses the security threats, vulnerabilities, challenges, and attacks of CPS. Initially, the CPS architecture is presented as a layered approach including the physical layer, network layer, and application layer in terms of functionality. Then, different cyber-physical attacks regarding each layer are elaborated, in addition to challenges and key issues associated with each layer. Afterward, deep learning models are analyzed for malicious URLs and intrusion detection in cyber-physical systems. A multilayer perceptron architecture is utilized for experiments using the malicious URL detection dataset and KDD Cup99 dataset, and its performance is compared with existing works. Lastly, we provide a roadmap of future research directions for cyber-physical security to investigate attacks concerning their source, complexity, and impact.

Cyber Intrusion Detection Based on a Mutative Scale Chaotic Bat Algorithm with Backpropagation Neural Network

In the current complex cyber environment, malicious attacks take various forms and constantly change. To overcome the problems of low detection efficiency and high false alarm rate in traditional intrusion detection techniques, a new method of network intrusion detection combining the variable scale chaos bat (MSCB) algorithm and BP neural network (BPNN) called MSCB-BPNN is proposed in this paper. In the proposed approach, a new MSCB algorithm is proposed to improve the BPNN by optimizing its thresholds and weights so as to prevent it from falling into local optimum. To verify the practical classification performance of the proposed method for intrusion detection, some experiments are carried out. In these experiments, the proposed approach is compared with BPNN, SA-BPNN, and GA-BPNN on the benchmark intrusion detection datasets KDD cup 99 and UNSW-NB15. The experimental results show that the accuracy of the proposed method can reach 99.4% and 99.8% accuracy on the KDD cup 99 training and test sets, respectively, while 89.0% and 93.9% accuracy on the UNSW-NB15 training and test sets, respectively, which are higher than BPNN, SA-BPNN, and GA-BPNN. Furthermore, the precision and recall ratio of MSCB-BPNN are also superior to other approaches. Thus, the proposed model has significant advantages over BPNN, SA-BPNN, and GA-BPNN methods.

A Hybrid Machine Learning Based Feature Selection Technique for Attack Detection in NIDS

Network security has arisen as a critical investigation area in the last period as the internet and communiqué technologies have grown in popularity. The network is protected by firewalls, antivirus software, and intrusion detection systems (IDS) and its related assets in a cyberspace against unauthorised access. These include a network-based intrusion detection system (NIDS) that constantly screens network traffic for hostile and suspicious behaviour in order to ensure the confidentiality, integrity, and availability of the data being transmitted over the Internet. However, researchers have made huge efforts to improve detection accuracy while decreasing False Alarm Rates (FAR) and to detect novel incursions, but they still confront difficulties. There has been an increase in the use of IDS systems based on machine learning and deep learning (ML and DL) in recent years. The study’s main goal is to detect attacks using a hybrid ML technique, and essential traits are identified via the Crow Search Algorithm (CSA). In order to improve classification accuracy, the decision tree is mapped to the Extreme Learning Machine (ELM). The decision tree selects features that are good at classifying. Additional weights for the selected features are calculated to improve their categorization accuracy. On the KDDCup dataset, the experiment is carried out in terms of a variety of parameter metrics.

Predicting Students Drop-out in MOOC from Learning Behavior Using Machine Learning

In this paper, we investigate how to predict student drop-out using machine-learning approaches. We discuss three methods: logistic regression, random forest and gradient boosting (GB). We used the KDD cup 2015 dataset. Since our dataset is unbalanced, a sampling technique called synthetic minority over-sampling technique (SMOTE) was used. Six models are then obtained, for each model we used a grid search cross-validation to find which are the parameters that optimize the mapping function according to the accuracy and the area under the receiver operating characteristic (AUC ROC) score. In order to validate our scores, we used the 10-fold cross-validation technique. Our findings show that ensemble machine-learning methods produce better results than basic machine-learning methods. The obtained results have proved that the extracted features and the used techniques are able to predict the drop-out with high efficiency. These relevant models were used with autoregressive moving average (ARIMA) model for time series data to detect the student at high risk of drop-out in earlier weeks of the course. The obtained AUC ROC for KDD Cup dataset with GB reaches 0.89 which is 0.1 close to the KDD cup winner.

Design of Intrusion Detection System based on Cyborg intelligence for security of Cloud Network Traffic of Smart Cities

The Internet of things (IoT) is an important technology that is highly beneficial in establishing smart items, connections and cities. However, there are worries regarding security and privacy vulnerabilities in IoT in which some emerge from numerous sources, including cyberattacks, unsecured networks, data, connections or communication. This paper provides an ensemble intrusion strategy based on Cyborg Intelligence (machine learning and biological intelligence) framework to boost security of IoT enabled networks utilized for network traffic of smart cities. To do this, multiple algorithms such Random Forest, Bayesian network (BN), C5.0, CART and Artificial Neural Network were investigated to determine their usefulness in identifying threats and attacks-botnets in IoT networks based on cyborg intelligence using the KDDcup99 dataset. The results reveal that the AdaBoost ensemble learning based on Cyborg Intelligence Intrusion Detection framework facilitates dissimilar network characteristics with the capacity to swiftly identify different botnet assaults efficiently. The suggested framework has obtained good accuracy, detection rate and a decreased false positive rate in comparison to other standard methodologies. The conclusion of this study would be a valuable complement to the efforts toward protecting IoT-powered networks and the accomplishment of safer smart cities.