An Outlier Detection Approach Based on Improved Self-Organizing Feature Map Clustering Algorithm

Abstract

Local Outlier Factor (LOF) outlier detecting algorithm has good accuracy in detecting global and local outliers. However, the algorithm needs to traverse the entire dataset when calculating the local outlier factor of each data point, which adds extra time overhead and makes the algorithm execution inefficient. In addition, if the K-distance neighborhood of an outlier point P contains some outliers that are incorrectly judged by the algorithm as normal points, then P may be misidentified as normal point. To solve the above problems, this paper proposes a Neighbor Entropy Local Outlier Factor (NELOF) outlier detecting algorithm. Firstly, we improve the Self-Organizing Feature Map (SOFM) algorithm and use the optimized SOFM clustering algorithm to cluster the dataset. Therefore, the calculation of each data point's local outlier factor only needs to be performed inside the small cluster. Secondly, this paper replaces the K-distance neighborhood with relative K-distance neighborhood and utilizes the entropy of relative K neighborhood to redefine the local outlier factor, which improves the accuracy of outlier detection. Experiments results confirm that our optimized SOFM algorithm can avoid the random selection of neurons, and improve clustering effect of traditional SOFM algorithm. In addition, the proposed NELOF algorithm outperforms LOF algorithm in both accuracy and execution time of outlier detection.