A New Similarity Measure and Its Use in Determining the Number of Clusters in a Multivariate Data Set

Abstract

Krolak-Schwerdt and Eckes [Krolak-Schwerdt, S., Eckes, T. (1992). A graph theoretic criterion for determining the number of cluster in a data set. Multivariate Behav. Res. 27(4):541–565] suggested a graph theoretic criterion, named GRAPH, which can be used to decide on the number of clusters present in a data set. However, the resulting algorithm usually terminates with fewer groups than are, actually, present in the data set. To alleviate this effect we first introduce a new distance measure based on success runs theory and Andrews curves [Andrews, D. F. (1972). Plots of high dimensional data. Biometrics 28:125–136] and incorporate it in the GRAPH procedure (as an alternative to the standard Euclidean distance used there). Extensive numerical experimentation revealed that the new algorithm behaves much better than the original GRAPH algorithm.