An Adaptive Exchange Algorithm for Sampling From Distributions With Intractable Normalizing Constants

Abstract

Sampling from the posterior distribution for a model whose normalizing constant is intractable is a long-standing problem in statistical research. We propose a new algorithm, adaptive auxiliary variable exchange algorithm, or, in short, adaptive exchange (AEX) algorithm, to tackle this problem. The new algorithm can be viewed as a MCMC extension of the exchange algorithm, which generates auxiliary variables via an importance sampling procedure from a Markov chain running in parallel. The convergence of the algorithm is established under mild conditions. Compared to the exchange algorithm, the new algorithm removes the requirement that the auxiliary variables must be drawn using a perfect sampler, and thus can be applied to many models for which the perfect sampler is not available or very expensive. Compared to the approximate exchange algorithms, such as the double Metropolis-Hastings sampler, the new algorithm overcomes their theoretical difficulty in convergence. The new algorithm is tested on the spatial autologistic and autonormal models. The numerical results indicate that the new algorithm is particularly useful for the problems for which the underlying system is strongly dependent. Supplementary materials for this article are available online.