Weighted kernel deterministic annealing: A maximum-entropy principle approach for shape clustering

Abstract

Kernel k-means and spectral clustering methods have both been used extensively to cluster data that are non-linearly separable in input space. While there has been significant research since their inceptions, both the methods have some drawbacks. Similar to the basic k-means algorithm, the Kernel k-means algorithm is sensitive to initialization. On the other hand, the spectral methods are based on finding eigenvectors and can be computationally prohibitive. In this paper, we propose a novel maximum-entropy principle (MEP) based weighted-kernel deterministic annealing (WKDA) algorithm, which is independent of initialization and has ability to avoid poor local minima. Additionally, we show that the WKDA approach reduces to Kernel k-means approach as a special case. Finally, we extend the proposed algorithm to include constrained-clustering and present the results for a variety of interesting data sets.