Non-parametric Bayesian Networks Research Articles

Matlatzinca: A PyBANSHEE-based graphical user interface for elicitation of non-parametric Bayesian networks from experts

This article describes the development of a GUI that addresses the challenge of eliciting dependencies between uncertain quantities elicited by experts. While software for eliciting univariate uncertainties is widely available, the mathematical complexity of multivariate dependence models makes direct elicitation difficult. To overcome this, we developed Matlatzinca,11Matlatzinca may be translated from Nahuatl (the language of the Aztecs) to English as “The people that make nets”. This is the name that the Aztecs gave to the inhabitants of the Valley of Toluca in central Mexico who were well known fishermen at the time. Our first release of a GUI for PyBANSHEE is meant for people that wish to quantify (make) Non-parametric Bayesian Networks (Nets) with expert judgments. Hence we name our GUI Matlatzinca. a GUI built on top of the Python module PyBANSHEE. The GUI facilitates the elicitation process and allows experts to model dependencies using a non-parametric Bayesian network without the need for ad hoc programming. A recent practical application shows that the developed GUI is a useful tool for performing dependence elicitations, highlighting the significance of the program for dependence assessment with expert judgment.

Nonparametric Bayesian Networks as a Tool of Multiscale Time Series Analysis and Remote Sensing Data Integration

Introduction. Nonparametric Bayesian networks are a promising tool for analyzing, visualizing, interpreting and predicting the structural and dynamic characteristics of complex systems. Modern interdisciplinary research involves the complex processing of heterogeneous data obtained using sensors of various physical nature. In the study of the forest fund, both methods of direct dendrological measurements and methods of remote observation using unmanned aerial vehicles are widely used. Information obtained using these methods must be analyzed in conjunction with hydrometeorological monitoring data.Aim. Investigation of the possibility of automating the monitoring of the well-being of the forest fund based on the integration of ground survey data, remote multispectral measurements and hydrometeorological observations using the mathematical apparatus of nonparametric Bayesian networks.Materials and methods. To assess the long-term joint dynamics of natural and climatic indicators and the radial growth of trees, a modified method of multiscale cross-correlation analysis was used with the removal of the background trend described by the moving average model. Relationships between various indicators were estimated based on the unconditional and conditional nonparametric Spearman correlation coefficients, which were used to reconstruct and parameterize the nonparametric Bayesian network.Results. A multiscale nonparametric Bayesian network was constructed to characterize both unconditional and conditional statistical relationships between parameters obtained from remote sensing, hydroclimatic and dendrological measurements. The proposed model showed a good quality of the plant fund state forecasting. The correlation coefficients between the observed and predicted indicators exceed 0.6, with the correlation coefficient comprising 0.77 when predicting the growth trend of annual tree rings.Conclusion. The proposed nonparametric Bayesian network model reflects the relationship between various factors that affect the forest ecosystem. The Bayesian network can be used to assess risks and improve environmental management planning.

Version 1.3-BANSHEE—A MATLAB toolbox for Non-Parametric Bayesian Networks

This update (version 1.3) of our contribution BANSHEE—A MATLAB toolbox for Non-Parametric Bayesian Networks provides functional updates. Additions to the software package include quantifying fully parametric Bayesian Networks, drawing a user-defined sample size number of (un)conditional samples, sample-based conditioning sampling, and choosing different sample sizes for the model-validation tests based on the Hellinger distance. The use of the t copula in the Cramér–Von Mises test is also added. New detailed examples demonstrating the new features available are also provided.

PyBanshee version (1.0): A Python implementation of the MATLAB toolbox BANSHEE for Non-Parametric Bayesian Networks with updated features

In this paper we discuss PyBanshee, which is a Python-based open-source implementation of the MATLAB toolbox BANSHEE. PyBanshee constitutes the first fully open-source package to quantify, visualize and validate Non-Parametric Bayesian Networks (NPBNs). The architecture of PyBanshee is heavily based on its MATLAB predecessor. It presents the full implementation of existing tools and introduces new modules. Specifically, PyBanshee allows for: (i) choosing fully parametric one-dimensional margins, (ii) choosing different sample sizes for the model-validation tests based on the Hellinger distance, (iii) drawing user-defined sample sizes of the NPBN, (iv) sample-based conditioning sampling (similarly to the closed-source proprietary package UNINET by LightTwist Software) and (v) visualizing the comparison between the histograms of the unconditional and conditional marginal distributions. New detailed examples demonstrating new features are provided.

Decision Making Model for Municipal Wastewater Conventional Secondary Treatment with Bayesian Networks

Technical, economic, regulatory, environmental, and social and political interests make the process of selecting an appropriate wastewater treatment technology complex. Although this problem has already been addressed from the dimensioning approach, our proposal in this research, a model of decision making for conventional secondary treatment of municipal wastewater through continuous-discrete, non-parametric Bayesian networks was developed. The most suitable network was structured in unit processes, independent of each other. Validation, with data in a mostly Mexican context, provided a positive predictive power of 83.5%, an excellent kappa (0.77 > 0.75), and the criterion line was surpassed with the location of the model in a receiver operating characteristic (ROC) graph, so the model can be implemented in this region. The final configuration of the Bayesian network allows the methodology to be easily extended to other types of treatments, wastewater, and to other regions.

Applying non-parametric Bayesian networks to estimate maximum daily river discharge: potential and challenges

Abstract. Non-parametric Bayesian networks (NPBNs) are graphical tools for statistical inference widely used for reliability analysis and risk assessment and present several advantages, such as the embedded uncertainty quantification and limited computational time for the inference process. However, their implementation in hydrological studies is still scarce. Hence, to increase our understanding of their applicability and extend their use in hydrology, we explore the potential of NPBNs to reproduce catchment-scale hydrological dynamics. Long-term data from 240 river catchments with contrasting climates across the United States from the Catchment Attributes and Meteorology for Large-sample Studies (CAMELS) data set will be used as actual means to test the utility of NPBNs as descriptive models and to evaluate them as predictive models for maximum daily river discharge in any given month. We analyse the performance of three networks, one unsaturated (hereafter UN-1), one saturated (hereafter SN-1), both defined only by hydro-meteorological variables and their bivariate correlations, and one saturated network (hereafter SN-C), consisting of the SN-1 network and including physical catchments' attributes. The results indicate that the UN-1 network is suitable for catchments with a positive dependence between precipitation and discharge, while the SN-1 network can also reproduce discharge in catchments with negative dependence. The latter can reproduce statistical characteristics of discharge (tested via the Kolmogorov–Smirnov statistic) and have a Nash–Sutcliffe efficiency (NSE) ≥0.5 in ∼40 % of the catchments analysed, receiving precipitation mainly in winter and located in energy-limited regions at low to moderate elevation. Further, the SN-C network, based on similarity of the catchments, can reproduce discharge statistics in ∼10 % of the catchments analysed. We show that once a NPBN is defined, it is straightforward to infer discharge and to extend the network itself with additional variables, i.e. going from the SN-1 network to the SN-C network. However, the results also suggest considerable challenges in defining a suitable NPBN, particularly for predictions in ungauged basins. These are mainly due to the discrepancies in the timescale of the different physical processes generating discharge, the presence of a “memory” in the system, and the Gaussian-copula assumption used for modelling multivariate dependence.

A Non-parametric Bayesian Network for multivariate probabilistic modelling of Weigh-in-Motion System Data

Weigh-in-motion (WIM) systems help to collect data such as vehicular loads, individual axle loads, vehicle type, and number of axles. This is relevant in engineering because traffic load performs an essential function in the design of new bridges and in the reliability assessment of existing ones, in traffic analysis and other areas of engineering. Therefore, when WIM data is not available, computing synthetic WIM observations that adequately approximate the statistical dependence between variables is important. In this paper, WIM measurements from the Netherlands and Brazil were analysed, and a set of non-parametric Bayesian Networks (NPBNs) is presented. This paper significantly improves on previous results by allowing observations of inter-axial distance to be generated, by allowing several sources of data to be used in the modelling and by making software available to researchers and practitioners interested for generating synthetic observations based on the distribution of vehicle type. In particular, statistical models to describe the weight and length of different vehicle types are derived. Three NPBNs were quantified using data from: (i) six WIM locations of the motorway network of the Netherlands, (ii) one WIM location in one city route of Rotterdam, The Netherlands, and (iii) one WIM location of one highway in Araranguá city located in the south of Brazil. Additionally, a Graphical User Interface (GUI) for the six Dutch WIM motorways locations was developed. To illustrate a possible use of the model when WIM data is not available. The GUI was used to compute synthetic WIM observations using data collected through traffic counters gathered in Toluca city in central Mexico, as input. This paper shows that the methodology here presented is widely applicable and depends only on the assessment of vehicle type configuration.

Co-designing and building an expert-elicited non-parametric Bayesian network model: demonstrating a methodology using a Bonamia Ostreae spread risk case study.

The development and use of probabilistic models, particularly Bayesian networks (BN), to support risk‐based decision making is well established. Striking an efficient balance between satisfying model complexity and ease of development requires continuous compromise. Codesign, wherein the structural content of the model is developed hand‐in‐hand with the experts who will be accountable for the parameter estimates, shows promise, as do so‐called nonparametric Bayesian networks (NPBNs), which provide a light‐touch approach to capturing complex relationships among nodes. We describe and demonstrate the process of codesigning, building, quantifying, and validating an NPBN model for emerging risks and the consequences of potential management decisions using structured expert judgment (SEJ). We develop a case study of the local spread of a marine pathogen, namely, Bonamia ostreae. The BN was developed through a series of semistructured workshops that incorporated extensive feedback from many experts. The model was then quantified with a combination of field and expert‐elicited data. The IDEA protocol for SEJ was used in its hybrid (remote and face‐to‐face) form to elicit information about more than 100 parameters. This article focuses on the modeling and quantification process, the methodological challenges, and the way these were addressed.

Constraint-based learning for non-parametric continuous bayesian networks

Modeling high-dimensional multivariate distributions is a computationally challenging task. In the discrete case, Bayesian networks have been successfully used to reduce the complexity and to simplify the problem. However, they lack of a general model for continuous variables. In order to overcome this problem, Elidan (2010) proposed the model of copula Bayesian networks that parametrizes Bayesian networks using copula functions. We propose a new learning algorithm for this model based on a PC algorithm and a conditional independence test proposed by Bouezmarni et al. (2009). This test being non-parametric, no model assumptions are made allowing it to be as general as possible. This algorithm is compared on generated data with the parametric method proposed by Elidan (2010) and proves to have better results.

Probabilistic Characterization of the Vegetated Hydrodynamic System Using Non-Parametric Bayesian Networks

The increasing risk of flooding requires obtaining generalized knowledge for the implementation of distinct and innovative intervention strategies, such as nature-based solutions. Inclusion of ecosystems in flood risk management has proven to be an adaptive strategy that achieves multiple benefits. However, obtaining generalizable quantitative information to increase the reliability of such interventions through experiments or numerical models can be expensive, laborious, or computationally demanding. This paper presents a probabilistic model that represents interconnected elements of vegetated hydrodynamic systems using a nonparametric Bayesian network (NPBN) for seagrasses, salt marshes, and mangroves. NPBNs allow for a system-level probabilistic description of vegetated hydrodynamic systems, generate physically realistic varied boundary conditions for physical or numerical modeling, provide missing information in data-scarce environments, and reduce the amount of numerical simulations required to obtain generalized results—all of which are critically useful to pave the way for successful implementation of nature-based solutions.

VEGETATED HYDRODYNAMIC SYSTEM: PARAMETERIZATION AND STOCHASTIC DEPENDENCE MODELLING

Vegetation as a nature-based solution for increasing flood risk has convincingly shown potential for flood hazard (wave load) reduction but lacks generalized results. In this study we have introduced stochastic dependence modeling using non-parametric Bayesian networks (NPBN) for vegetated coastal systems where the system was parametrized using continuous marginal distributions, and likely (conditional) correlations among variables. The model represented a consistent joint probability distribution and hence can be used to generate physically realistic conditions in data-scare environments. It adds value to numerical modeling by reducing the number of simulations required to get meaningful generalized results. Main findings, that were derived by using a NPBN, help to pave way for implementation of nature-based solutions for a range of realistic conditions that can be found across global coastal foreshores.Recorded Presentation from the vICCE (YouTube Link): https://youtu.be/T6TP0DH0qMw

Modeling Local BES Indicators by Copula-Based Bayesian Networks

In Italy, the measure of the Equitable and Sustainable Well-being is provided by the Italian Institute of Statistics by means of a dashboard of basic and composite indicators. To investigate the dependence structure between the different domains of well-being, we propose the use of Non-Parametric Bayesian Networks based on the normal copula distribution, that allow to explore the conditional independence relationships between the composite indicators. The main advantage of the non-parametric models is that, as opposed to the parametric approach, they do not require any assumption on the marginal distributions of the variables. The proposed model is applied to the Equitable and Sustainable Well-being indicators measured at the provincial level and enriches the analysis of well-being by inspecting similarities and differences between Italian urban areas and territories.

Knowledge representation using non-parametric Bayesian networks for tunneling risk analysis

Knowledge capture and reuse are critical in the risk management of tunneling works. Bayesian networks (BNs) are promising for knowledge representation due to their ability to integrate domain knowledge, encode causal relationships, and update models when evidence is available. However, the model development based on classic BNs is challenging when expert opinions are solicited due to the discretization of variables and quantification of large conditional probability tables. This study applies non-parametric BNs, which only require the elicitation of the marginal distribution corresponding to each node and correlation coefficient associated with each edge, to develop a knowledge-based expert system for tunneling risk analysis. In particular, we propose to use the pair-wise Pearson's linear correlations to parameterize the model because the assessment is intuitive and experts in the engineering domain are more familiar and comfortable with this notion. However, when Spearman's rank correlation is given, the method can also be used by modification of the marginals. The method is illustrated with a tunnel case in the Wuhan metro project. The expert knowledge of risk assessment for common failures in shield tunneling is integrated and visualized. The developed model is validated by real documented accidents. Potential applications of the model are also explored, such as decision support for risk-based design.

Reliability analysis of reinforced concrete vehicle bridges columns using non-parametric Bayesian networks

In the bridge industry, current traffic trends have increased the likelihood of having the simultaneous presence of both extreme live loads and earthquake events. To date, their concurrent interaction has scarcely been systematically studied. Prevailing studies have investigated the isolated existence of either live loads or seismic actions.In an effort to fill this gap in the literature, a non-parametric Bayesian Network (BN) has been proposed. It is aimed at evaluating the conditional probability of failure for a reinforced concrete bridge column, subject simultaneously to the actions mentioned above. Based on actual data from a structure located in the State of Mexico, a Monte Carlo Simulation model was developed. This led to the construction of a BN with 17 variables.The set of variables included in the model can be categorized into three groups: acting loads, materials resistances and structure force-displacement behavior. Practitioners are then provided with a tool for unspecialized labor force to gather information in situ (e.g. Weight-In-Motion data and Schmidt hammer measurements), which can be included in the network, leading to an updated probability of failure. Moreover, this framework also serves as a quantitative tool for bridge column reliability assessments.Results from the theoretical model confirmed that the bridge column probability of failure was within the expected range reported in the literature. This reflects not only the appropriateness of its design but also the suitability of the proposed BN for reliability analysis.

Toward an integrated Bayesian network approach to measurement error detection and correction

ABSTRACTIn this article, the quality of data produced by national statistical institutes and by governmental institutions is considered. In particular, the problem of measurement error is analyzed and an integrated Bayesian network decision support system based on non-parametric Bayesian networks is proposed for its detection and correction. Non-parametric Bayesian networks are graphical models expressing dependence structure via bivariate copulas associated to the edges of the graph. The network structure and the misreport probability are estimated using a validation sample. The Bayesian network model is proposed to decide: (i) which records have to be corrected; (ii) the kind and amount of correction to be adopted. The proposed correction procedure is applied to the Banca d’Italia Survey on Household Income and Wealth and, specifically, the bond amounts are analyzed. Finally, the sensitivity of the conditional distribution of the true value random variable given the observed one to different evidence configurations is studied.

Large-Scale Hybrid Bayesian Network for Traffic Load Modeling from Weigh-in-Motion System Data

Traffic load plays an important role not only in the design of new bridges but also in the reliability assessment of existing structures. Weigh-in-motion systems are used to collect data to determine traffic loads. In this paper, the potential of hybrid nonparametric Bayesian networks (BNs) is demonstrated for modeling the complex data measured by the weigh-in-motion systems. The quantification process provides insight into the statistical buildup of the traffic load. The BN is shown to be a reliable traffic load model for use in bridge design. The model’s value is shown with applications for prediction of missing data and calculation of extreme loads. A simulation that includes both a dynamic BN and a static component is performed. The model is able to generate the distribution function of section forces, such as bending moments, generated by multiple vehicles in several lanes. The model presented in this paper should serve as a benchmark for further applications.

Statistical modelling of networked human-automation performance using working memory capacity

This study examines the challenging problem of modelling the interaction between individual attentional limitations and decision-making performance in networked human-automation system tasks. Analysis of real experimental data from a task involving networked supervision of multiple unmanned aerial vehicles by human participants shows that both task load and network message quality affect performance, but that these effects are modulated by individual differences in working memory (WM) capacity. These insights were used to assess three statistical approaches for modelling and making predictions with real experimental networked supervisory performance data: classical linear regression, non-parametric Gaussian processes and probabilistic Bayesian networks. It is shown that each of these approaches can help designers of networked human-automated systems cope with various uncertainties in order to accommodate future users by linking expected operating conditions and performance from real experimental data to observable cognitive traits like WM capacity.Practitioner Summary: Working memory (WM) capacity helps account for inter-individual variability in operator performance in networked unmanned aerial vehicle supervisory tasks. This is useful for reliable performance prediction near experimental conditions via linear models; robust statistical prediction beyond experimental conditions via Gaussian process models and probabilistic inference about unknown task conditions/WM capacities via Bayesian network models.

An approach to software reliability prediction based on time series modeling

Reliability is the key factor for software system quality. Several models have been introduced to estimate and predict reliability based on results of software testing activities. Software Reliability Growth Models (SRGMs) are considered the most commonly used to achieve this goal. Over the past decades, many researchers have discussed SRGMs’ assumptions, applicability, and predictability. They have concluded that SRGMs have many shortcomings related to their unrealistic assumptions, environment-dependent applicability, and questionable predictability. Several approaches based on non-parametric statistics, Bayesian networks, and machine learning methods have been proposed in the literature. Based on their theoretical nature, however, they cannot completely address the SRGMs’ limitations. Consequently, addressing these shortcomings is still a very crucial task in order to provide reliable software systems. This paper presents a well-established prediction approach based on time series ARIMA (Autoregressive Integrated Moving Average) modeling as an alternative solution to address the SRGMs’ limitations and provide more accurate reliability prediction. Using real-life data sets on software failures, the accuracy of the proposed approach is evaluated and compared to popular existing approaches.

Sign: large-scale gene network estimation environment for high performance computing.

Our research group is currently developing software for estimating large-scale gene networks from gene expression data. The software, called SiGN, is specifically designed for the Japanese flagship supercomputer "K computer" which is planned to achieve 10 petaflops in 2012, and other high performance computing environments including Human Genome Center (HGC) supercomputer system. SiGN is a collection of gene network estimation software with three different sub-programs: SiGN-BN, SiGN-SSM and SiGN-L1. In these three programs, five different models are available: static and dynamic nonparametric Bayesian networks, state space models, graphical Gaussian models, and vector autoregressive models. All these models require a huge amount of computational resources for estimating large-scale gene networks and therefore are designed to be able to exploit the speed of 10 petaflops. The software will be available freely for "K computer" and HGC supercomputer system users. The estimated networks can be viewed and analyzed by Cell Illustrator Online and SBiP (Systems Biology integrative Pipeline). The software project web site is available at http://sign.hgc.jp/ .

Estimating Genome-Wide Gene Networks Using Nonparametric Bayesian Network Models on Massively Parallel Computers

We present a novel algorithm to estimate genome-wide gene networks consisting of more than 20,000 genes from gene expression data using nonparametric Bayesian networks. Due to the difficulty of learning Bayesian network structures, existing algorithms cannot be applied to more than a few thousand genes. Our algorithm overcomes this limitation by repeatedly estimating subnetworks in parallel for genes selected by neighbor node sampling. Through numerical simulation, we confirmed that our algorithm outperformed a heuristic algorithm in a shorter time. We applied our algorithm to microarray data from human umbilical vein endothelial cells (HUVECs) treated with siRNAs, to construct a human genome-wide gene network, which we compared to a small gene network estimated for the genes extracted using a traditional bioinformatics method. The results showed that our genome-wide gene network contains many features of the small network, as well as others that could not be captured during the small network estimation. The results also revealed master-regulator genes that are not in the small network but that control many of the genes in the small network. These analyses were impossible to realize without our proposed algorithm.