Threshold-based clustering with merging and regularization in application to network intrusion detection

Abstract

Signature-based intrusion detection systems look for known, suspicious patterns in the input data. In this paper we explore compression of labeled empirical data using threshold-based clustering with regularization. The main target of clustering is to compress training dataset to the limited number of signatures, and to minimize the number of comparisons that are necessary to determine the status of the input event as a result. Essentially, the process of clustering includes merging of the clusters which are close enough. As a consequence, we will reduce original dataset to the limited number of labeled centroids. In a complex with k-nearest-neighbor ( kNN) method, this set of centroids may be used as a multi-class classifier. The experiments on the KDD-99 intrusion detection dataset have confirmed effectiveness of the above procedure.