First-Order Phase Transition and Bayesian Image Processing by Loopy Belief Propagation

Abstract

The framework is presented of Bayesian image restoration for multi-valued images based on the Q-state Potts model. Hyperparameters in the probabilistic model are determined so as to maximize the marginal likelihood. A practical algorithm is described for multi-valued image restoration based on loopy belief propagation. We conclude that the maximization of marginal likelihood can provide good results even if the ap rioriprobabilistic model exhibits first-order phase transition. In Bayesian image restoration, we have to determine certain hyperparameters, and in practice it is common for the hyperparameters to be determined so as to maximize a marginal likelihood. 1) In some probabilistic models treated in statistical mechanics, some types of phase transition occur at a critical point. First-order and second-order phase transitions are familiar types of phase transition. In the case of first-order phase transition, the first derivative of the free energy is discontinuous at the critical point; if the probabilistic model exhibits first-order phase transition, the free energy is not differentiable at a critical value of the hyperparameter. The marginal likelihood can be expressed in terms of the free energies of the ap riori and a posteriori probability distributions. This means that the marginal likelihood is not differentiable at a critical value of the hyperparameter present in the ap riori probability distribution. In the context of the expectation-maximization algorithm, which is a powerful method for maximizing the marginal likelihood, the marginal likelihood is differentiable at any value of the hyperparameter. In the present paper, we investigate Bayesian image restoration with hyperparameters estimated by maximizing the marginal likelihood when we adopt a Q-state Potts model 2) as the ap rioriprobability distribution. The free energy of the Q-state Potts model for Q≥3 is not differentiable at a certain value of the hyperparameter when we analyze the model by means of advanced mean-field approximations. The maximization of the marginal likelihood is achieved by using loopy belief propaga