Online learning for the Dirichlet process mixture model via weakly conjugate approximation

Abstract

The Dirichlet process (DP) mixture model is widely used for clustering and density estimation. The use of the DP mixture model has become computationally feasible because of the development of various Markov chain Monte Carlo algorithms. However, when analyzing large data, DP mixture models are impractical owing to their high computational costs. An online learning algorithm that processes data sequentially as they arrive is an attractive way to analyze large data. Existing online learning algorithms based on variational inference are very fast; however, their performance is unsatisfactory owing to the crude approximation of the posterior distribution. We propose a novel mini-batch online learning algorithm based on assumed density filtering, which takes full advantage of available computing resources to improve performance and achieves better performances relative to existing online algorithms based on variational inference.