Maximum within-cluster association

Abstract

This paper addresses a new method and aspect of information-theoretic clustering where we exploit the minimum entropy principle and the quadratic distance measure between probability densities. We present a new minimum entropy objective function which leads to the maximization of within-cluster association. A simple implementation using the gradient ascent method is given. In addition, we show that the minimum entropy principle leads to the objective function of the k-means clustering, and the maximum within-cluster association is closed related to the spectral clustering which is an eigen-decomposition-based method. This information-theoretic view of spectral clustering leads us to use the kernel density estimation method in constructing an affinity matrix.