Asymptotic behavior of free energy when optimal probability distribution is not unique

Abstract

Bayesian inference is a widely used statistical method. The free energy and the generalization loss, which are used to estimate the accuracy of Bayesian inference, are known to be small in singular models that do not have a unique optimal parameter. However, their characteristics have not yet been clarified when there are multiple optimal probability distributions. In this paper, we theoretically derive the asymptotic behaviors of the generalization loss and the free energy in the case that the optimal probability distributions are not unique and show that they contain asymptotically different terms from those of the conventional asymptotic analysis.