Deep clustering by maximizing mutual information in variational auto-encoder

Abstract

Unsupervised clustering, which is extensively employed in deep learning and computer vision as a fundamental technique, has attracted much attention in recent years. Deep embedding clustering often uses auto-encoders to learn representations for clustering. However, auto-encoders tend to corrupt the learning representations when simultaneously learning embedded representations and performing clustering. In this paper, we propose a Deep Clustering via Variational Auto-Encoder (DC-VAE) of mutual information maximization. First, we formulate the deep clustering problem as learning soft cluster assignments within the framework of variational auto-encoder. Second, we impose mutual information maximization on the observed data and the representations to prevent soft cluster assignments from distorting learning representations. Third, we derive a new generalization evidence lower bound objects related to several previous models and introduce parameters to balance learning informative representations and clustering. It is shown that the proposed model can significantly boost the performance of clustering by learning effective and reliable representations for downstream machine learning tasks. Through experimental results on several datasets, we demonstrate that the proposed model is competitive with existing state-of-the-arts on multiple performance metrics.