Cluster validity index for adaptive clustering algorithms

Abstract

Everyday a large number of records of surfing internet are generated. In various situations when the authors are analysing internet data they do not know the cluster structure of the author's database of traffic features, such as when the border of cluster members is vague, and the clusters' partitions have different shapes, how to establish an algorithm to solve the clustering problem? Adaptive clustering algorithms can meet this challenge. Moreover, how to determinate the number of clusters when not only fuzzy cluster but also hard cluster are used? To address those problems, a new cluster validity index is proposed in this study. The proposed index focuses on the information of the geometrical structure of dataset by analysing the neighbourhood of data objects, which makes the index independent of the traditional fuzzy membership matrix. The new index consists of two parts, namely the `compactness' and `separation measure'. The compactness indicates the degree of the similarity among the data objects in the same cluster. The separation measure indicates the degree of dissimilarity among the data objects in different clusters. The performance of their proposed index is excellent underpinned by the outcomes from the experiments based on both artificial datasets and real world datasets.