On a class of repulsive mixture models

Abstract

Finite or infinite mixture models are routinely used in Bayesian statistical practice for tasks such as clustering or density estimation. Such models are very attractive due to their flexibility and tractability. However, a common problem in fitting these or other discrete models to data is that they tend to produce a large number of overlapping clusters. Some attention has been given in the statistical literature to models that include a repulsive feature, i.e., that encourage separation of mixture components. We study here a method that has been shown to achieve this goal without sacrificing flexibility or model fit. The model is a special case of Gibbs measures, with a parameter that controls the level of repulsion that allows construction of d-dimensional probability densities whose coordinates tend to repel each other. This approach was successfully used for density regression in Quinlan et al. (J Stat Comput Simul 88(15):2931–2947, 2018). We detail some of the global properties of the repulsive family of distributions and offer some further insight by means of a small simulation study.