KDD Cup Dataset Research Articles

Multi Stage Filter Using Enhanced Adaboost for Network Intrusion Detection

Based on the analysis and distribution of network attacks in KDDCup99 dataset and real time traffic, this paper proposes a design of multi stage filter which is an efficient and effective approach in dealing with various categories of attacks in networks. The first stage of the filter is designed using Enhanced Adaboost with Decision tree algorithm to detect the frequent attacks occurs in the network and the second stage of the filter is designed using enhanced Adaboost with Naive Byes algorithm to detect the moderate attacks occurs in the network. The final stage of the filter is used to detect the infrequent attack which is designed using the enhanced Adaboost algorithm with Naive Bayes as a base learner. Performance of this design is tested with the KDDCup99 dataset and is shown to have high detection rate with low false alarm rates.

The sum is greater than the parts

Many competing models have been proposed in the past decade for predicting student knowledge within educational software. Recent research attempted to combine these models in an effort to improve performance but have yielded inconsistent results. While work in the 2010 KDD Cup data set showed the benefits of ensemble methods, work in the Genetics Tutor failed to show similar benefits. We hypothesize that the key factor has been data set size. We explore the potential for improving student performance prediction with ensemble methods in a data set drawn from a different tutoring system, the ASSISTments Platform, which contains 15 times the number of responses of the Genetics Tutor data set. We evaluated the predictive performance of eight student models and eight methods of ensembling predictions. Within this data set, ensemble approaches were more effective than any single method with the best ensemble approach producing predictions of student performance 10% better than the best individual student knowledge model.

An Approach of Automatic Data Mining Algorithm for Intrusion Detection and Prevention System

The Network security system plays a major role in the communication world, here every one's need security and assurance for their communication. Normally the hacking, Intrusion software's are using threats, malicious to enter in to the network and they are keep recording or monitoring the communication for this action the hackers may use following techniques Rexd, Rsh, Defult account/Null password, Generic Recon Mechanisms, NIS (Formerly yp), and Rpc Portmapper. The normal firewalls can address only insider threats. The main focus of our research is applied on two different platforms, the first one is processing on TCP-Dump portion using KDDCUP99 data set and mining, the second one is finding of best algorithm for data mining Intrusion on UNIX. From the available kddcup'99 dataset two subsets are taken with the record size of 997 and 11438. For UNIX User data all instances are taken. The classification rate and the false negative rate are used as the performance criteria with 3 fold cross validation. It is found that PART, SMO, Hyper pipes, Filtered Classifier, Random forest, Naive Bayes Updateable, KStar are giving high classification rate with low false positive rate. For UNIX User data ZeroR is giving high performance. Real time data is also applied to finalize the best algorithm under each category of classifier. Later online implementation has to be done.

Knowledge Reduction Based on Divide and Conquer Method in Rough Set Theory

The divide and conquer method is a typical granular computing method using multiple levels of abstraction and granulations. So far, although some achievements based on divided and conquer method in the rough set theory have been acquired, the systematic methods for knowledge reduction based on divide and conquer method are still absent. In this paper, the knowledge reduction approaches based on divide and conquer method, under equivalence relation and under tolerance relation, are presented, respectively. After that, a systematic approach, named as the abstract process for knowledge reduction based on divide and conquer method in rough set theory, is proposed. Based on the presented approach, two algorithms for knowledge reduction, including an algorithm for attribute reduction and an algorithm for attribute value reduction, are presented. Some experimental evaluations are done to test the methods on uci data sets and KDDCUP99 data sets. The experimental results illustrate that the proposed approaches are efficient to process large data sets with good recognition rate, compared with KNN, SVM, C4.5, Naive Bayes, and CART.

Intrusion Detection System using Bayesian Network and Hidden Markov Model

Across the globe, billions of dollars are spending every year to provide security to the network systems to prevent the intrusions. Some consider the disruption of the vital systems as a serious threat which disables the work of hospitals, banks, military and various internet services across the world. To avert this impending threat, there are many possible solutions: one of these solutions is intrusion detection systems (IDS). The paper proposes to discuss the IDS model in its elaboration using Bayesian Network and the Hidden Markov Model (HMM) approach with KDDCUP dataset. The IDS framework has been designed with various levels of processing such as model learning with training data and constructing the Bayesian Network and this structure has been used as HMM state transition diagram. The preprocessed KDDCUP dataset has been used to train and test the model. The IDS model has been trained and tested for normal and attack type connection records separately. The results evince that the performance of the model is of high order for classification of normal and intrusions attacks.

Applying Variable Coe_cient functions to Self-Organizing Feature Maps for Network Intrusion Detection on the 1999 KDD Cup Dataset

Self-Organizing Feature Maps (SOFM's) can be a valuable element in a network intrusion detection system. When classification is performed on a segment of network tra_c, the usual method for class determination is selecting the class which has the smallest measurement of the Euclidean distance from the multi-dimensional network tra_c sample to the class’ multi-dimensional prototype. This minimum distance is calculated with equivalent weights for each dimension of data in the network tra_c sample. In this paper we explore the possibility of applying di_erent randomly generated weightings to each dimension of data in the network tra_c sample to increase positive classifications of the network sample data provided by the 1999 KDD Cup Dataset. We show that there is improvement, and recommend that further studies be done in choosing the right evolutionary functions to help modify the hotspots and achieve better results.

Outlier Identification using Symmetric Neighborhoods

In recent research, outlier mining has been widely used in areas such as telecommunication, health care, finance and network intrusion detection. By applying outlier mining in network anomaly detection, rarely occurring attacks can be identified. In this paper we propose an outlier mining method based on symmetric neighborhood relationships. We evaluate the method with UCI ML Repository datasets, benchmark dataset KDD Cup 1999 and real time intrusion datasets. The experimental results are compared with existing approaches and performance is excellent.

English

Intrusions are serious questions in network systems. Numerous intrusion detection techniques are present to tackle these problems but the dilemma is performance. To raise performance, it is momentous to raise the detection rates and decrease false alarm rates. The contemporary methods use Principal Component Analysis (PCA) to project features space to principal feature space and choose features corresponding to the highest eigenvalues, but the features corresponding to the highest eigenvalues may not have the best possible sensitivity for the classifier due to ignoring several sensitive features. Therefore, we applied a Genetic Algorithm (GA) to search the principal feature space for a subset of features with optimal sensitivity. So, in this research, a method for optimal features subset selection is proposed to overcome performance issues using PCA, GA and Multilayer Perceptron (MLP). The KDD-cup dataset is used. This method is capable to minimize amount of features and maximize the detection rates.   Key words: KDD-cup, PCA, MLP, GA, detection rate, and false alarm.

Optimized intrusion detection mechanism using soft computing techniques

Intrusion detection is an important technique in computer and network security. A variety of intrusion detection approaches be present to resolve this severe issue but the main problem is performance. It is important to increase the detection rates and reduce false alarm rates in the area of intrusion detection. Therefore, in this research, an optimized intrusion detection mechanism using soft computing techniques is proposed to overcome performance issues. The KDD-cup dataset is used that is a benchmark for evaluating the security detection mechanisms. The Principal Component Analysis (PCA) is applied to transform the input samples into a new feature space. The selecting of an appropriate number of principal components is a critical problem. So, Genetic Algorithm (GA) is used in the optimum selection of principal components instead of using traditional method. The Support Vector Machine (SVM) is used for classification purpose. The performance of this approach is addresses. Further, a comparative analysis is made with existing approaches. Consequently, this method provides optimal intrusion detection mechanism which is capable to minimize amount of features and maximize the detection rates.

The Study of Invasion Examination Algorithm Based on Improvement Fuzzy C Means

This paper put forward intrusion detection algorithm based on improved fuzzy C means (FCM) algorithm and execute the anomaly detection on KDDCUP data set, build intrusion detection system based improved algorithm and analyze the feasibility of the system. Through the fuzzy C means value's improvement algorithm, solve the fuzzy C means value algorithm problem that the algorithm sensitive to selection of the initial values and easily to fall in the local best solution. Thereby under the condition guarantee integrality and consistency of data attribute values, get rid of blindness of selecting initial value and reduce clustering time and algorithm complexity, enhance speed of the algorithm.

Distributed denial of service attack detection using an ensemble of neural classifier

The vulnerabilities in the Communication (TCP/IP) protocol stack and the availability of more sophisticated attack tools breed in more and more network hackers to attack the network intentionally or unintentionally, leading to Distributed Denial of Service (DDoS) attack. The DDoS attacks could be detected using the existing machine learning techniques such as neural classifiers. These classifiers lack generalization capabilities which result in less performance leading to high false positives. This paper evaluates the performance of a comprehensive set of machine learning algorithms for selecting the base classifier using the publicly available KDD Cup dataset. Based on the outcome of the experiments, Resilient Back Propagation (RBP) was chosen as base classifier for our research. The improvement in performance of the RBP classifier is the focus of this paper. Our proposed classification algorithm, RBPBoost, is achieved by combining ensemble of classifier outputs and Neyman Pearson cost minimization strategy, for final classification decision. Publicly available datasets such as KDD Cup, DARPA 1999, DARPA 2000, and CONFICKER were used for the simulation experiments. RBPBoost was trained and tested with DARPA, CONFICKER, and our own lab datasets. Detection accuracy and Cost per sample were the two metrics evaluated to analyze the performance of the RBPBoost classification algorithm. From the simulation results, it is evident that RBPBoost algorithm achieves high detection accuracy (99.4%) with fewer false alarms and outperforms the existing ensemble algorithms. RBPBoost algorithm outperforms the existing algorithms with maximum gain of 6.6% and minimum gain of 0.8%.

A granular agent evolutionary algorithm for classification

By inspiration of the granular evolutionary algorithm, a Granular Agent Evolutionary Classification (GAEC) algorithm for the classification task in data mining is proposed. The method uses the granular agent to denote the set of some examples that have similar attributions and the knowledge base guides the evolution of granular agent. Also some granular evolutionary operators are designed for classification problem. Assimilation operator, exchange operator, and differentiation operator reflect the competitive, cooperative and self-learning ability of agent, respectively. Finally, some classification rules are extracted from granular agents by some strategy to forecast the sort of new data. Empirical study contains UCI data sets, KDDCUP99 data sets and remote image recognition. The test results show that the algorithm has a good classification prediction, and only need a small price for the training time. In most UCI data sets, the performance of GAEC is better than G-NET, OCEC and C4.5, which have good performance. At the same time, some Gaussian White Noise attributes are added to these UCI data sets and the results show GACE has some anti-noise abilities. To test the scalability of GAEC, two functions along two dimensions, the number of training examples and the number of attributes are used. Also, GAEC are applied to some real world fields, intrusion detection system and remote sensing image recognition. The experiments for KDDCUP99 verify GAEC has capability to deal with massive data in real world and good predicting capability for unknown type data. At last, the accuracy rate of GAEC is also good for the remote sensing image recognition.

Immune Multiagent System for Network Intrusion Detection using Non-linear Classification Algorithm

growth of intelligent intrusion and diverse attack techniques in network systems stimulate computer scientists and mathematical researchers to challenge the dangers of intelligent attacks. In this work, we integrate artificial immune algorithm with non-linear classification of pattern recognition and machine learning methods to solve the problem of intrusion detection in network systems. A new non classification algorithm was developed based on the danger theory model of human immune system (HIS).The abstract model of system algorithm is inspired from HIS cell mechanism mainly, the Dendritic cell behavior and T-cell mechanisms. Classification techniques using k-nearest neighbor (k-NN) or Gaussian Mixture (GMM) almost have the common sense that they believe the neighboring data. The algorithm tested use KDD Cup dataset and the result shows a significant improvement in detection accuracy and reducing the false alerts.

Survey on Frequent Itemset Mining Algorithms

Many researchers invented ideas to generate the frequent itemsets. The time required for generating frequent itemsets plays an important role. Some algorithms are designed, considering only the time factor. Our study includes depth analysis of algorithms and discusses some problems of generating frequent itemsets from the algorithm. We have explored the unifying feature among the internal working of various mining algorithms. Some implementations were done with KDD cup Dataset to explore the relative merits of each algorithm. The work yields a detailed analysis of the algorithms to elucidate the performance with standard dataset like Adult, Mushroom etc. The comparative study of algorithms includes aspects like different support values, size of transactions and different datasets.

Research and Application of an improved Support Vector Clustering Algorithm on Anomaly Detection

Anomaly detection approaches build models of normal data and detect deviations from the normal model in observed data. Anomaly detection applied to intrusion detection and computer security has been an active area of research. The major benefit of anomaly detection algorithms is their ability to potentially detect unforeseen attacks. In this paper, a novel weighted support vector clustering algorithm for anomaly detection is proposed. The weight to each input point is defined according to the position of samples in sphere space. The results of experiment demonstrate that the algorithm has excellent capability and applying it in intrusion detection system can be an effective way via using the data sets of KDD cup 99.

Improved fuzzy C-means clustering algorithm and its application to intrusion detection

Concerning the limits of the Fuzzy C-Means(FCM) clustering algorithm,a new kind of improved algorithm with two stages was put forward.The objective function in FCM clustering algorithm was improved by taking into account the modulus of dot density function and the weight of eigenvector;furthermore,the iterative reasoning formula and the description of algorithm were presented.This algorithm solved the problems of the sample's unequal distribution and the eigenvector of the sample contributing unevenly to the classes,which improved the clustering precision effectively.Experiments on data sets KDD CUP 99 show that the algorithm has good reliability and feasibility.

Design of multiple-level hybrid classifier for intrusion detection system using Bayesian clustering and decision trees

With increasing connectivity between computers, the need to keep networks secure progressively becomes more vital. Intrusion detection systems (IDS) have become an essential component of computer security to supplement existing defenses. This paper proposes a multiple-level hybrid classifier, a novel intrusion detection system, which combines the supervised tree classifiers and unsupervised Bayesian clustering to detect intrusions. Performance of this new approach is measured using the KDDCUP99 dataset and is shown to have high detection and low false alarm rates.

Quantum Particle swarm optimization based network Intrusion feature selection and Detection

Considering the relevance among features, which filter-based feature selection method fails to deal with, a kind of hybrid quantum particle swarm optimization and support vector machines based network intrusion feature selection wrapper algorithm is put forward. The subset of features is represented using quantum superposition characteristic and probability representation, among which superposition characteristic can make a single particle represent several states, thus potentially increases population diversity. Every particle in the quantum particle swarm stands for a selected subset of features. A probabilistic mutation is adopted to avoid local optimal and a taboo search table is used to enlarge particle swarm's search space and avoid repeated computation. The fitness of particle is defined as the correct classification percentage by SVM using a training set whose patterns are represented using only the selected subset of features. The results of experiments demonstrate that the proposed method can be an effective and efficient way for feature selection and detection via using the data sets of KDD cup 99.

A framework for an adaptive anomaly detection system with fuzzy data mining

In this paper, we present an adaptive anomaly detection framework that is applicable to network-based intrusion detection. Our framework employs fuzzy cluster algorithm to detect anomalies in an online, adaptive fashion without a priori knowledge of the underlying data. We evaluate our method by performing experiments over network records from the KDD CUP99 data set.

Intrusion detection using a fuzzy genetics-based learning algorithm

Fuzzy systems have demonstrated their ability to solve different kinds of problems in various applications domains. Currently, there is an increasing interest to augment fuzzy systems with learning and adaptation capabilities. Two of the most successful approaches to hybridize fuzzy systems with learning and adaptation methods have been made in the realm of soft computing. Neural fuzzy systems and genetic fuzzy systems hybridize the approximate reasoning method of fuzzy systems with the learning capabilities of neural networks and evolutionary algorithms. The objective of this paper is to describe a fuzzy genetics-based learning algorithm and discuss its usage to detect intrusion in a computer network. Experiments were performed with DARPA data sets [KDD-cup data set. http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html], which have information on computer networks, during normal behaviour and intrusive behaviour. This paper presents some results and reports the performance of generated fuzzy rules in detecting intrusion in a computer network.