Effective and Optimal Clustering Based on New Clustering Validity Index

Abstract

As an important tool, clustering validity index (CVI) is usually used to evaluate the validity of the clustering results and determine the optimal clustering number $(\mathbf{K}_{\mathbf{opt}})$ . However, the existing CVIs have some shortcomings, such as too complex calculation, low computational efficiency and narrow range of applications. Aiming at these problems, an improved K-means algorithm that uses density parameters for initial centers selection is firstly proposed to make the results of traditional clustering algorithms more stable. Then, a new variance based clustering validity index (VCVI) from the perspective of spatial distribution of data sets is introduced to extend the application of the partitional clustering algorithms and better evaluate the effect of clustering results. Finally, a new algorithm integrated with the improved K-means algorithm and the new proposed VCVI is designed to effectively determine the K opt and the optimal clustering partition. The experimental results have shown that the new proposed algorithm with VCVI is effective in forming the $\mathbf{K}_{\mathbf{opt}}$ and the optimal clustering partition for the tested data sets.