Minimum Free Energies with "Data Temperature" for Parameter Learning of Bayesian Networks

Abstract

Maximum likelihood (ML) method for estimating parameters of Bayesian networks (BNs) is efficient and accurate for large samples. However, ML suffers from overfitting when the sample size is small. Bayesian methods, which are effective to avoid overfitting, have difficulties for determining optimal hyperparameters of prior distributions with good balance between theoretical and practical points of view when no prior knowledge is available. In this paper, we propose an alternative estimation method of the parameters on BNs. The method uses a principle, with roots in statistical thermal physics, of minimizing free energy. We propose an explicit model of the temperature, which should be properly estimated. We designate the model data temperature. In assessments of classification accuracy, we show that our method yields higher accuracy than that of the Bayesian method with normally recommended hyperparameters. Moreover, our method exhibits robustness for the choice of introduced hyperparameters..