Learning Likelihood-Equivalence Bayesian Networks Using an Empirical Bayesian Approach

Abstract

Many studies on learning Bayesian networks have used the Dirichlet prior score metric (DPSM). Although they assume different optimum hyper-parameter values for DPSM, few studies have focused on selection of optimum hyper-parameter values. Analyses of DPSM hyper-parameters for learning Bayesian networks are presented here along with the following results: 1. DPSM has a strong consistency for any hyper-parameter values. That is, the score metric DPSM, uniform prior score metric (UPSM), likelihood-equivalence Bayesian Dirichlet score metric (BDe), and minimum description length (MDL) asymptotically converge to the same results. 2. The optimal hyper-parameter values are affected by the true network structure and the number of data. 3. Contrary to Yang and Chang (2002)’s results, BDe based on likelihood equivalence is a theoretically and actually reasonable score metric, if the optimum hyper-parameter values can be found. Using these results, this paper proposes a new learning Bayesian network method based on BDeu that uses the empirical Bayesian approach. The unique features of this method are: 1. It is able to reflect a user’s prior knowledge. 2. It has both the strong consistency and likelihood equivalence properties. 3. It finds the optimum hyperparameter value of BDeu to maximize predictive efficiency, by adapting to domain and data size. In addition, this paper presents some numerical examples using the proposed method that demonstrate the effectiveness of the proposed method.