Investigating Outlier Detection Techniques Based on Kernel Rough Clustering

Abstract

Background: Data quality is crucial to the success of big data analytics. However, the presence of outliers affects data quality and data analysis. Employing effective outlier detection techniques to eliminate dirty data can improve data quality and garner more accurate analytical insights. Data uncertainty presents a significant challenge for outlier detection methods and warrants further refinement in the era of big data. Objective: The unsupervised outlier detection based on the integration of clustering and outlier scoring scheme is the current research hotspot. However, hard clustering fails when dealing with abnormal patterns with uncertain and unexpected behavior. Rough boundaries help identify more accurate cluster structures. Therefore, this article uses uncertainty soft clustering based on rough set theory to extend the clustering technology and designs appropriate scoring schemes to capture abnormal instances. This solves the problem of outlier detection in uncertain and nonlinear complex data. Methods: This paper proposes the flow of an outlier detection algorithm based on Kernel Rough Clustering and then compares the detection accuracy with five existing popular methods using synthetic and real-world datasets. The results show that the proposed method has higher detection accuracy. Results: The detection precision and recall of the proposed method were improved. For the detection accuracy, it is superior to popular methods, indicating that the proposed method has a good detection effect in identifying outlier. Conclusion: Compared with popular methods, the proposed method has a slight advantage in detection accuracy and is one of the effective algorithms that can be selected for outlier detection.