A Bayesian non‐parametric approach for automatic clustering with feature weighting

Abstract

Despite being a well‐known problem, feature weighting and feature selection are a major predicament for clustering. Most of the algorithms, which provide weighting or selection of features, require the number of clusters to be known in advance. On the other hand, the existing automatic clustering procedures that can determine the number of clusters are computationally expensive and often do not make a room for feature weighting or selection. In this paper, we propose a Gibbs sampling‐based algorithm for the Dirichlet process mixture model, which can determine the number of clusters and can also incorporate a near‐optimal feature weighting. We show that in the limiting case, the algorithm approaches a hard clustering procedure, which resembles minimization of an underlying clustering objective similar to weighted k‐means with an additional forfeit for the number of clusters and hence retains the simplicity of the Llyod's heuristics. To avoid the trivial solution of the resulting linear program, we include an additional entropic penalty on the feature weights. The proposed algorithm is tested on several synthetic and real‐life datasets. Through a detailed experimental analysis, we demonstrate the competitiveness of our proposal against the baseline as well as state‐of‐the‐art procedures for centre‐based high‐dimensional clustering.