How many clusters? A robust PSO-based local density model

Abstract

While most clustering methods assume that the number of data clusters is known, automatically estimating the number of clusters by algorithm itself is still a challenging problem in the data clustering field. In this paper, we aim to develop a novel local and not differentiable clustering method based on Particle Swarm Optimization, which can estimate the number of clusters automatically. In particular, the proposed approach measures the local compactness of each cluster by local density function, pushes the PSO towards maximizing such a compactness, and penalizes the whole procedure to avoid estimating quite a lot of clusters during the evolution. The compactness modeling makes the clustering robust to outliers and noise. In addition, due to the merit of PSO, although kernel trick is used in our modeling, it does not consume too much memory when more and more data are processed. The evaluation on the synthetic dataset and the five publicly available datasets shows that our algorithm can estimate the appropriate number of clusters and outperforms six related state-of-the-art clustering methods that can also estimate the number of clusters.