"DATA TEMPERATURE" IN MINIMUM FREE ENERGIES FOR PARAMETER LEARNING OF BAYESIAN NETWORKS

Abstract

Maximum likelihood method for estimating parameters of Bayesian networks (BNs) is efficient and accurate for large samples. However, the method suffers from overfitting when the sample size is small. Bayesian methods, which are effective to avoid overfitting, present difficulties for determining optimal hyperparameters of prior distributions with good balance between theoretical and practical points of view when no prior knowledge is available. As described in this paper, we propose an alternative estimation method of the parameters on BNs. The method uses a principle, rooted in thermodynamics, of minimizing free energy (MFE). We define internal energies, entropies, and temperature, which constitute free energies. Especially for temperature, we propose a "data temperature" assumption and some explicit models. This approach can treat the maximum likelihood principle and the maximum entropy principle in a unified manner of the MFE principle. For assessments of classification accuracy, our method shows higher accuracy than that obtained using the Bayesian method with normally recommended hyperparameters. Moreover, our method exhibits robustness for the choice of introduced hyperparameters.