Hierarchical clustering of continuous variables based on the empirical copula process and permutation linkages

Abstract

The agglomerative hierarchical clustering of continuous variables is studied in the framework of the likelihood linkage analysis method proposed by Lerman. The similarity between variables is defined from the process comparing the empirical copula with the independence copula in the spirit of the test of independence proposed by Deheuvels. Unlike more classical similarity coefficients for variables based on rank statistics, the comparison measure considered in this work can also be sensitive to non-monotonic dependencies. As aggregation criteria, besides classical linkages, permutation-based linkages related to procedures for combining dependent p -values are considered. The performances of the corresponding clustering algorithms are compared through thorough simulations. In order to guide the choice of a partition, a natural probabilistic selection strategy, related to the use of the gap statistic in object clustering, is proposed and empirically compared with classical ordinal approaches. The resulting variable clustering procedure can be equivalently regarded as a potentially less computationally expensive alternative to more powerful tests of multivariate independence.