Cube-based incremental outlier detection for streaming computing

Abstract

Outlier detection is one of the most critical and challenging tasks of data mining. It aims to find patterns in data that do not conform to expected behavior. Data streams in streaming computing are huge in nature and arrive continuously with changing distribution, which imposes new challenges for outlier detection algorithms in time and space efficiency. Incremental local outlier factor (ILOF) detection dynamically updates the profiles of data points, while the arrival of consecutive and massive volume data points in a streaming manner causes high local data density and leads to expensive time and space overhead. Our work is motivated by its deficiencies, and in this paper we propose a cube-based outlier detection algorithm (CB-ILOF). The data space of streaming data is divided into multiple cubes, then the outlier detection of data points is transferred into the outlier detection of cubes, which significantly reduces time and memory overhead. We also present a performance evaluation on 5 datasets. Experimental results show the superiority of the CB-ILOF over the ILOF on accuracy, memory usage, and execution time.