The data preprocessing in improving the classification quality of network intrusion detection systems

Abstract

Stream-based intrusion detection is a growing problem in computer network security environments. Many previous researches have applied machine learning as a method to detect attacks in network intrusion detection systems. However, these methods still have limitations of low accuracy and high false alarm rate. To improve the quality of classification, this paper proposes two solutions in the data preprocessing stage, that is, the solution of feature selection and resampling of the training dataset before they are used for training the classifiers. This is based on the fact that there is a lot of class imbalanced data in the training dataset used for network intrusion detection systems, as well as that there are many features in the dataset that are irrelevant to the classification goal, this reduces the quality of classification and increases the computation time. The data after preprocessing by the proposed algorithms is used to train the classifiers using different machine learning algorithms including: Decision Trees, Naive Bayes, Logistic Regression, Support Vector Machines, k Nearest Neighbor and Artificial Neural Network. The training and testing results on the UNSW-NB15 dataset show that: as with the Reconnaissance attack type, the proposed feature selection solution for F-Measure achieves 96.31%, an increase of 19.64%; the proposed oversampling solution for F-Measure achieves 6.99%, an increase of 3.17% and the proposed undersampling solution for F-Measure achieves 94.65%, an increase of 11.42%.