Bridging the Last-Mile Gap in Network Security via Generating Intrusion-Specific Detection Patterns through Machine Learning

Abstract

With successful machine learning applications in many fields, researchers tried to introduce machine learning into intrusion detection systems for building classification models. Although experimental results showed that these classification models could produce higher accuracy in predicting network attacks on the offline datasets, compared with the operational intrusion detection systems, machine learning is rarely deployed in the real intrusion detection environment. This is what we call the last mile problem with the machine learning approach to network intrusion detection, the discrepancy between the strength and requirements of machine learning and network operational semantics. In this paper, we aim to bridge the aforementioned gap. In particular, an LCC-RF-RFEX feature selection approach is proposed to select optimal features of the specific type of attacks from dataset, and then, an intrusion-specific approach is introduced to convert them into detection patterns that can be used by the nonmachine-learning detector for the corresponding specific attack detection in the real-world network environment. To substantiate our approach, we take Snort, KDDCup’99 dataset, and Dos attacks as the experimental subjects to demonstrate how to close the last-mile gap. For the specific type of Dos attacks in the KDDCup’99 dataset, we use the LCC-RF-RFEX method to select optimal feature subset and utilize our intrusion-specific approach to generate new rules in Snort by using them. Comparing performance differences between the existing Snort rule set and our augmented Snort rule set with regard to Dos attacks, the experimental results showed that our approach expanded Snort’s detection capability of Dos attacks, on average, reduced up to 25.28% false-positive alerts for Teardrop attacks and Synflood attacks, and decreased up to 98.87% excessive alerts for Mail bomb attacks.