Fast Distance-based Outlier Detection in Data Streams based on Micro-clusters

Abstract

Continuous outlier detection in data streams is one important topic in data mining. It has many applications in public health, network intrusion detection, and fraud detection. Over the last two decades of research, many studies have been conducted on distance-based outlier detection algorithms which are viable, scalable, and parameter-free approaches. Because streaming data points arrive and expire over time, the challenge is to monitor the outlier status of data points with time and space efficiency. In this study, we propose three algorithms: O-MCOD, U-MCOD, and M-MCOD. These algorithms improve upon the state-of-the-art algorithm in distance-based outlier detection in data streams, i.e., MCOD, by relaxing the constraints of micro-clusters and using the minimal probing principal. With extensive experiments on synthetic and real-world datasets, we show that the proposed algorithms are superior in time and space efficiency. Specially, our proposed algorithms are 1.5 to 95 times faster than MCOD, require as low as 25% peak memory compared to MCOD, and outperform the most recent algorithm NETS.