Extreme minority class detection in imbalanced data for network intrusion

Abstract

As the amount of traffic on the Internet increases, so does the number of new and sophisticated network attacks. Intrusion detection systems are the most important tools for accurate detection of potential threats. Due to the dynamic nature of network attacks, deep learning neural networks play a significant role in intrusion detection, as they have proven to be effective in processing large amounts of data. However, deep learning networks often have difficulties to effectively detect attack classes that are in minority, when trained with imbalanced cybersecurity data. The common way to deal with this difficulty is resampling. In contrast to resampling, in this paper we implement Deep Neural Network for intrusion detection varying its parameters, and analyze detection performances of minority classes in imbalanced multi-class data. The model is trained and tested on the CICIDS-2017 dataset, which contains almost 3 million records and 15 traffic classes, where some classes are in extreme minority, holding only a few records per class. Additionally, the model was evaluated on an also imbalanced CICIDS-2018 dataset. Two feature selection methods are performed on the preprocessed data, in order to obtain two different feature subsets. Our findings show that some coarse grained features are of such significance that attacks with only 3 instances can be completely and accurately detected. As a conclusion, we show the difference in feature characteristics for minority classes that are crucial for their detection.