Semi-Supervised Alert Filtering for Network Security

Abstract

Network-based intrusion detection systems play a pivotal role in cybersecurity, but they generate a significant number of alerts. This leads to alert fatigue, a phenomenon where security analysts may miss true alerts hidden among false ones. To address alert fatigue, practical detection systems enable administrators to divide alerts into multiple groups by the alert name and the related Internet Protocol (IP) address. Then, some groups are deliberately ignored to conserve human resources for further analysis. However, the drawback of this approach is that the filtering basis is so coarse-grained that some true alerts are also ignored, which may cause critical security issues. In this paper, we present a new semi-supervised and fine-grained filtering method that uses not only alert names and IP addresses but also semi-supervised clustering results from the alerts. We evaluate our scheme with both a private dataset from a security operations center and a public dataset from the Internet. The experimental results demonstrate that the new filtering scheme achieves higher accuracy and saves more human resources compared to the current state-of-the-art method.