A method for testing the distinctness of clusters: A test of the disjunction of two clusters in Euclidean space as measured by their overlap

Abstract

A method is described for testing the distinctness of two clusters in Euclidean space. One first calculates the projections, q,of the N1and N2members of the clusters onto the line joining the cluster centroids. From the distributions of qan index of disjunction, W,is calculated, which corresponds to an index of overlap, VG.The quantity W√(N1+N2)is distributed as noncentral tsubject to assumptions on the multivariate normal distribution of the clusters. This allows a test of whether the observed disjunction is significantly greater than a chosen figure, which is equivalent to testing whether the overlap of the clusters is significantly less than a corresponding value of VG.Two clusters that appear distinct may be produced simply by the partitioning of a homogeneous swarm into two contiguous regions. Provided that the clusters form a dichotomy in a dendrogram, and that the clustering method yields geometrically convex clusters, a conservative test of this situation can be derived by determining the excess of Wover the value expected for a rectangular distribution.