Learning Bayesian Network Parameters With Small Data Set: A Parameter Extension under Constraints Method

Abstract

Recent advances have illustrated substantial benefits from learning Bayesian networks (BNs). However, when the available data size is small, the BN parameter learning becomes a key challenge in many intelligent applications. By integrating both sample data and expert constraints, we propose a BN parameter learning algorithm with extension method-parameter extension under constraints (PEUC) by introducing related domain expert knowledge. Knowledge is transformed into inequality constraints which candidate parameter sets arise from the relative constraints space.The maximum entropy principle helps to estimate the parameter in statistical averaging model while candidate sets of BN parameters satisfy the constrained knowledge by bootstrapping techniques. Then BN parameters are estimated based on the real available sampled data set and extension streams of candidate parameters samples from the constraints space. The sample size is also taken into account according to the contribution to the final parameters. Experimental results of benchmark BN modeling problems demonstrate that PEUC algorithm tends to the classical MLE algorithm when the modeling data size is sufficient. Furthermore, when the available data size is small, the parameters of BN can be estimated by PEUC as well, and the learned accuracy is superior to MLE, MAP or QMAP algorithm. Finally, PEUC is also applied to a real bearing fault diagnosis case. The presented approach provides a new promising BN parameter learning way for more intelligent system modeling problems, particularly when the data sets are small.