Data-Driven Bayesian Network Learning: A Bi-Objective Approach to Address the Bias-Variance Decomposition

Vicente-Josué Aguilera-Rueda,Nicandro Cruz-Ramírez,Efrén Mezura-Montes

doi:10.3390/mca25020037

Vicente-Josué Aguilera-Rueda, Nicandro Cruz-Ramírez + Show 1 more

Open Access

https://doi.org/10.3390/mca25020037

Copy DOI

Abstract

We present a novel bi-objective approach to address the data-driven learning problem of Bayesian networks. Both the log-likelihood and the complexity of each candidate Bayesian network are considered as objectives to be optimized by our proposed algorithm named Nondominated Sorting Genetic Algorithm for learning Bayesian networks (NS2BN) which is based on the well-known NSGA-II algorithm. The core idea is to reduce the implicit selection bias-variance decomposition while identifying a set of competitive models using both objectives. Numerical results suggest that, in stark contrast to the single-objective approach, our bi-objective approach is useful to find competitive Bayesian networks especially in the complexity. Furthermore, our approach presents the end user with a set of solutions by showing different Bayesian network and their respective MDL and classification accuracy results.

Highlights

Bayesian Network (BN) [1] is a preferred formalism to represent knowledge under uncertainty using efficient reasoning
The induction of a BN from data is subsequently classified into two types (i) methods searching for conditional-dependencies, known as constraint-based methods and (ii) search and scoring based methods [2,3,4,5]
This study is based on the latter case, where the learning task is framed as a combinatorial optimization problem with two main components: (1) a metric to assess the quality of each BN candidate, and (2) a search procedure to move intelligently through the space of candidate networks

Summary

Introduction

Bayesian Network (BN) [1] is a preferred formalism to represent knowledge under uncertainty using efficient reasoning. Building a BN comes with inherent difficulties, such as deciding on the specific graph structure, and corresponding parameter values. Two traditional ways to build a BN structure are through (i) domain expertise and (ii) a data-driven inductive approach. This study is based on the latter case, where the learning task is framed as a combinatorial optimization problem with two main components: (1) a metric to assess the quality of each BN candidate, and (2) a search procedure to move intelligently through the space of candidate networks. A BN is a graphical model that represents a joint probability distribution over a set of random variables {X1, . Θ is a set of parameters which quantifies the network. The joint probability distribution can be recovered from local conditional probability distributions as is shown in Equation (1)

Methods

Results

Conclusion