Choice of error cost function for training unobservable nodes in Bayesian networks

Abstract

In the construction of a Bayesian network from observed data, the fundamental assumption that the variables starting from the same parent are conditionally independent can be met by introduction of hidden node (C.K. Kwoh and D.F. Gillies, 1994). We show that the conditional probability matrices for the hidden node for a triplet, linking three observed nodes, can be determined by the gradient descent method. As in all operational research problems, the quality of the result depends on the ability to locate a feasible solution for the conditional probabilities. C.K. Kwoh and D.F. Gillies (1995) presented a paper in which they detailed the methodologies for estimating the initial values of unobservable variables in Bayesian networks. We present the concept of determining the best conditional matrices as an estimation problem. The discrepancies between the observed and predicted values are mapped into a monotonic function where its gradients are used for adjusting the parameters to be estimated. We present our investigation of choosing among various popular error cost functions for training the networks with hidden nodes and determined that both cross entropy and sum of squared error cost functions work equally well for our implementation.