Bayesian Network Graph Research Articles

Combining real data and expert knowledge to build a Bayesian Network — Application to assess multiple risk factors for fall among elderly people

Building a prediction model based on both real data and expert knowledge can be challenging. We experienced this difficulty in our context of predicting fall risk factors in the elderly, where two sources of data were available: hand-reported information from medical consultations and expert information from human specialists. As is often the case, the real data were incomplete and imbalanced, and the participation of medical experts required the selection of a kind of model they can understand. This article describes the steps followed in the construction and evaluation of the model, structured as a new framework. The proposed model is a Bayesian network (BN) built from expert knowledge and learning from a real data set, guided by both the understandability of the BN graph and the performance of the model to predict risk factors. The resulting Bayesian network model includes 90 variables to evaluate 10 actionable target risk factors. It demonstrated its potential compared to other classifiers in terms of prediction performance, whilst offering a higher degree of interpretability.

Bayesian Network structure learning algorithm for highly missing and non imputable data: Application to breast cancer radiotherapy data

It is not uncommon for real-life data produced in healthcare to have a higher proportion of missing data than in other scopes. To take into account these missing data, imputation is not always desired by healthcare experts, and complete case analysis can lead to a significant loss of data. The algorithm proposed here, allows the learning of Bayesian Network graphs when both imputation and complete case analysis are not possible. The learning process is based on a set of local bootstrap learnings performed on complete sub-datasets which are then aggregated and locally optimized. This learning method presents competitive results compared to other structure learning algorithms, whatever the mechanism of missing data.

Bayesian Techniques in Predicting Frailty among Community-Dwelling Older Adults in the Netherlands

Background Frailty is a syndrome that is defined as an accumulation of deficits in physical, psychological, and social domains. On a global scale, there is an urgent need to create frailty-ready healthcare systems due to the healthcare burden that frailty confers on systems and the increased risk of falls, healthcare utilization, disability, and premature mortality. Several studies have been conducted to develop prediction models for predicting frailty. Most studies used logistic regression as a technique to develop a prediction model. One area that has experienced significant growth is the application of Bayesian techniques, partly due to an increasing number of practitioners valuing the Bayesian paradigm as matching that of scientific discovery.Objective We compared ten different Bayesian networks as proposed by ten experts in the field of frail elderly people to predict frailty with a choice from ten dichotomized determinants for frailty.Methods We used the opinion of ten experts who could indicate, using an empty Bayesian network graph, the important predictors for frailty and the interactions between the different predictors. The candidate predictors were age, sex, marital status, ethnicity, education, income, lifestyle, multimorbidity, life events, and home living environment. The ten Bayesian network models were evaluated in terms of their ability to predict frailty. For the evaluation, we used the data of 479 participants that filled in the Tilburg Frailty indicator (TFI) questionnaire for assessing frailty among community-dwelling older people. The data set contained the aforementioned variables and the outcome “frail”. The model fit of each model was measured using the Akaike information criterion (AIC) and the predictive performance of the models was measured using the area under the curve (AUC) of the receiver operator characteristic (ROC). The AUCs of the models were validated using bootstrapping with 100 repetitions. The relative importance of the predictors in the models was calculated using the permutation feature importance algorithm (PFI).Results The ten Bayesian networks of the ten experts differed considerably regarding the predictors and the connections between the predictors and the outcome. However, all ten networks had corrected AUCs >0.700. Evaluating the importance of the predictors in each model, “diseases or chronic disorders” was the most important predictor in all models (10 times). The predictors “lifestyle” and “monthly income” were also often present in the models (both 6 times). One or more diseases or chronic disorders, an unhealthy lifestyle, and a monthly income below 1800 euro increased the likelihood of frailty.Conclusions Although the ten experts all made different graphs, the predictive performance was always satisfying (AUCs >0.700). While it is true that the predictor importance varied all the time, the top three of the predictor importance consisted of “diseases or chronic disorders”, “lifestyle” and “monthly income”. All in all, asking for the opinion of experts in the field of frail elderly to predict frailty with Bayesian networks may be more rewarding than a data-driven forecast with Bayesian networks because they have expert knowledge regarding interactions between the different predictors.

Synthetic document generator for annotation-free layout recognition

Analyzing the layout of a document to identify headers, sections, tables, figures etc. is critical to understanding its content. Deep learning based approaches for detecting the layout structure of document images have been promising. However, these methods require a large number of annotated examples during training, which are both expensive and time consuming to obtain. We describe here a synthetic document generator that automatically produces realistic documents with labels for spatial positions, extents and categories of the layout elements. The proposed generative process treats every physical component of a document as a random variable and models their intrinsic dependencies using a Bayesian Network graph. Our hierarchical formulation using stochastic templates allow parameter sharing between documents for retaining broad themes and yet the distributional characteristics produces visually unique samples, thereby capturing complex and diverse layouts. We empirically illustrate that a deep layout detection model trained purely on the synthetic documents can match the performance of a model that uses real documents.

Information graphs and their use for Bayesian network graph construction

In this paper, we present the information graph (IG) formalism, which provides a precise account of the interplay between deductive and abductive inference and causal and evidential information, where ‘deduction’ is used for defeasible ‘forward’ inference. IGs formalise analyses performed by domain experts in the informal reasoning tools they are familiar with, such as mind maps used in crime analysis. Based on principles for reasoning with causal and evidential information given the evidence, we impose constraints on the inferences that may be performed with IGs. Our IG-formalism is intended to facilitate the construction of formal representations within AI systems by serving as an intermediary formalism between analyses performed using informal reasoning tools and formalisms that allow for formal evaluation. In this paper, we investigate the use of the IG-formalism as an intermediary formalism in facilitating Bayesian network (BN) graph construction. We propose a structured approach for automatically constructing from an IG a directed BN graph, together with qualitative constraints on the probability distribution represented by the BN. Moreover, we prove a number of formal properties of our approach and identify assumptions under which the construction of an initial BN graph can be fully automated.

A fixed structure learning automata‐based optimization algorithm for structure learning of Bayesian networks

AbstractOne of the useful knowledge representation tools, which can describe the joint probability distribution between some random variables with a graphical model and can be trained by a dataset, is the Bayesian network (BN). A BN is composed of a network structure and a conditional probability distribution table for each node. Discovering an optimal BN structure is an NP‐hard optimization problem that various meta‐heuristic algorithms are applied to solve this problem by researchers. The genetic algorithms, ant colony optimization, evolutionary programming, artificial bee colony, and bacterial foraging optimization are some of the meta‐heuristic methods to solve this problem using a dataset. Most of these methods are applying a scoring metric to generate the best network structure from a set of candidates. A Fixed Structure Learning Automata‐Based (FSLA‐B) algorithm is presented in this paper to solve the structure learning problem of BNs. There is a fixed structure learning automaton for each pair of vertices in the BN's graph structure in the proposed algorithm. The action of this automaton determines the presence and direction of an edge between the vertices. The proposed algorithm performs a guided search procedure using the FSLA and escapes from local optimums. Several datasets are utilised in this paper to evaluate the performance of the proposed algorithm. By performing various experiments, multiple meta‐heuristic algorithms are compared with the introduced new one. The obtained results represented that the proposed algorithm could produce competitive results and find the near‐optimal solution for the BN structure learning problem.

Learning Bayesian networks using A* search with ancestral constraints

When using a Bayesian network to model a practical problem, weak prior knowledge projected as ancestral constraints is necessary. However, it is difficult to directly utilize these non-decomposable constraints using search strategies based on the decomposable score. In this study, we attempt to solve this problem by conducting an implicate path-space search graph and driving the A* algorithm, which is used to obtain the globally optimal solution satisfying the given constraints. We use a maximum covering principle to provide useful pruning rules based on these constraints in the new framework. Moreover, we improve the simple heuristic and the static k-cycle conflict heuristic to adapt to ancestral constraints. We theoretically prove that the new heuristic functions remain admissible and consistent. Our experiments demonstrate that the proposed framework with the new heuristic functions significantly reduces the space complexity of A* search compared with state-of-the-art frameworks, such as Bayesian network graphs and equivalent class trees, when integrating ancestral constraints.

Exploiting structured high-level knowledge for domain-specific visual classification

In the last decade, deep learning models have yielded impressive performance on visual object recognition and image classification. However these methods still rely on learning visual data distributions and show difficulties in dealing with complex scenarios where visual appearance only is not enough to effectively tackle them. This is the case, for instance, of fine-grained image classification in domain-specific applications for which it is very complex to employ data-driven models because of the lack of large amounts of samples and that, instead, can be solved by resorting to specialized human knowledge. However, encoding this specialized knowledge and injecting it into deep models is not trivial.In this paper, we address this problem by: a) employing computational ontologies to model specialized knowledge in a structured representation and, b) building a hybrid visual-semantic classification framework. The classification method performs inference over a Bayesian Network graph, whose structure depends on the knowledge encoded in an ontology and evidences are built using the outputs of deep networks.We test our approach on a fine-grained classification task, employing an extremely complex dataset containing images from several fruit varieties as well as visual and semantic annotations. Since the classification is done at the variety level (e.g., discriminating between different cherry varieties), appearance changes slightly and expert domain knowledge — making using of contextual information — is required to perform classification accurately.Experimental results show that our approach significantly outperforms standard deep learning–based classification methods over the considered scenario as well as existing methods leveraging semantic information for classification. These results demonstrate, on one hand, the difficulty of purely-visual deep methods in tackling small and highly-specialized datasets and, on the other hard, the capabilities of our approach to effectively encode and use semantic knowledge for enhanced accuracy.

Learning Bayesian Networks with the Saiyan Algorithm

Some structure learning algorithms have proven to be effective in reconstructing hypothetical Bayesian Network graphs from synthetic data. However, in their mission to maximise a scoring function, many become conservative and minimise edges discovered. While simplicity is desired, the output is often a graph that consists of multiple independent subgraphs that do not enable full propagation of evidence. While this is not a problem in theory, it can be a problem in practice. This article examines a novel unconventional associational heuristic called Saiyan, which returns a directed acyclic graph that enables full propagation of evidence. Associational heuristics are not expected to perform well relative to sophisticated constraint-based and score-based learning approaches. Moreover, forcing the algorithm to connect all data variables implies that the forced edges will not be correct at the rate of those identified unrestrictedly. Still, synthetic and real-world experiments suggest that such a heuristic can be competitive relative to some of the well-established constraint-based, score-based and hybrid learning algorithms.

Junction trees constructions in Bayesian networks

Junction trees are used as graphical structures over which propagation will be carried out through a very important property called the ruining intersection property. This paper examines an alternative method for constructing junction trees that are essential for the efficient computations of probabilities in Bayesian networks. The new proposed method converts a sequence of subsets of a Bayesian network into a junction tree, in other words, into a set of cliques that has the running intersection property. The obtained set of cliques and separators coincide with the junction trees obtained by the moralization and triangulation process, but it has the advantage of adapting to any computational task by adding links to the Bayesian network graph.

Sensing Statistical Primary Network Patterns via Bayesian Network Structure Learning

In cognitive radio (CR) technology, the trend of sensing is no longer to only detect the presence of active primary users, as a large number of applications demand for more comprehensive knowledge on primary network behaviors in spatial, temporal, and frequency domains. To satisfy such requirements, we study the statistical relationship among primary nodes by introducing a Bayesian network (BN)-based framework. How to efficiently learn such a BN structure is a long-standing issue that is not fully understood even in the statistical learning community. To address such an issue in CR, this paper proposes a BN structure learning scheme consisting of a concise directional dependence checking function and a regular BN graph, which achieves significantly lower computational complexity compared with existing approaches. With this result, cognitive users could efficiently understand the statistical behavior patterns in the primary networks, such that more efficient cognitive protocols could be designed across different network layers.

Bayesian-network-based system identification of spatial distribution of structural parameters

Structural deterioration or damage in civil infrastructures may results in severe losses of properties and human lives. To prevent such situations, structure health monitoring (SHM) technologies have been developed in recent decades. As one of the SHM technologies, so-called system identification (SI) methods aim to estimate structural parameters by minimizing an error function consisting of measured and calculated responses of a structure under the same loading condition. Such optimization-based SI algorithms may suffer from ill-posedness of the inverse problem, which may result in non-uniqueness of solutions or non-stability of the optimization process. In this paper, in order to avoid issues related to ill-posedness, an SI method based on the Bayesian Network (BN) technology is developed, especially for probabilistic identification of spatial distribution of structural parameters. Utilizing graph theories, a BN describes random variables by nodes connected by links, which represent conditional probability tables (CPT) explaining the probabilistic relationship between the linked nodes. For effective SI using BN, this study proposes a BN graph model, which employs a bi-variate Gaussian function as a shape function describing the spatial distribution of a structural parameter with a small number of parameters. The parameters of the Gaussian (shape) function are considered as nodes in the BN to describe various spatial distribution patterns using a small number of parent nodes. Using this modeling approach, the number of the BN nodes is not affected by the size of the finite element (FE) mesh. Thus, the approach prevents the BN graph from growing to an intractable size even if the FE analysis requires smaller meshes to improve the accuracy of the structural analysis. Using the constructed BN, information on applied loads, and observed structural responses, a BN inference algorithm effectively updates the prior distribution of spatial distribution parameters to the posterior distribution. The proposed method is tested and demonstrated by numerical examples. Using a variety of structural deterioration scenarios, the SI results by the proposed method are compared with those by maximum likelihood estimation (MLE), and an FE-updating method. During the test, the influences of measurement errors, and incomplete/missing data on the performance of the methods are also investigated. The results show that the proposed SI approach is more stable and robust than the other tested methods, and potentially has more merits that are worth investigating in future research.

Quantum inference on Bayesian networks

Performing exact inference on Bayesian networks is known to be $#P$-hard. Typically approximate inference techniques are used instead to sample from the distribution on query variables given the values $e$ of evidence variables. Classically, a single unbiased sample is obtained from a Bayesian network on $n$ variables with at most $m$ parents per node in time $O(nmP{(e)}^{\ensuremath{-}1})$, depending critically on $P(e)$, the probability that the evidence might occur in the first place. By implementing a quantum version of rejection sampling, we obtain a square-root speedup, taking $O(n{2}^{m}P{(e)}^{\ensuremath{-}\frac{1}{2}})$ time per sample. We exploit the Bayesian network's graph structure to efficiently construct a quantum state, a q-sample, representing the intended classical distribution, and also to efficiently apply amplitude amplification, the source of our speedup. Thus, our speedup is notable as it is unrelativized---we count primitive operations and require no blackbox oracle queries.

Iterative Message Passing for Pilot-Assisted Multiuser Detection in MC-CDMA Systems

Application of the belief propagation (BP) algorithm is proposed for pilot-assisted reception of multi-carrier code division multiple access systems where the input symbols are binary phase shift keying symbols. The proposed method utilizes a Bayesian network graph that includes channel parameters as explicit nodes representing random variables. Channel parameters in both frequency and time domains are included in the graph. The messages on the graph are obtained by making large system assumptions for the user symbols and Gaussian assumptions for the messages related to the channel parameters. Density evolution analysis is performed whose results match well with the simulation results. For the simulated scenario it is found that BP with the proposed graph outperforms channel estimation followed by graph based detection, and non-graph based iterative channel estimation.

Belief propagation with Gaussian priors for pilot-assisted communication over fading ISI channels

Application of the belief propagation (BP) algorithm is proposed for pilot-assisted reception of Gaussian distributed symbols over unknown inter-symbol interference channels whose parameters are also Gaussian distributed. The proposed Bayesian network graph for the demodulator includes the channel parameters as explicit nodes, allowing the combined estimation of the channel and information bearing symbols with the aid of pilot symbols. Since direct application of BP is not tractable for this model, the algorithm is obtained by introducing approximations of the message distributions appropriately. The proposed algorithm is parallel in nature, is suitable for sparse channels, and can be easily extended to coded systems for joint demodulation and decoding. The numerical results show near-optimal performance in the minimum mean squared error sense for a small example, and a performance better than that of iterative semi-blind equalization with batch matrix-based estimators for the general case.