Two design methods of hyperparameters in variational Bayes learning for Bernoulli mixtures

Abstract

Variational Bayes learning or mean field approximation is widely used in statistical models which are made of mixtures of exponential distributions, for example, normal mixtures, binomial mixtures, and hidden Markov models. To derive variational Bayes learning algorithm, we need to determine the hyperparameters in the a priori distribution; however, the design method of hyperparameters has not yet been established. In the present paper, we propose two different design methods of hyperparameters which are applied to the different purposes. In the former method, the hyperparameter is determined for minimization of the generalization error. In the latter method, it is chosen so that candidates of hidden structure in training data are extracted. It is experimentally shown that the optimal hyperparameters for two purposes are different from each other.