Conditional Linear Gaussian Research Articles

A continuous Bayesian network regression model for estimating seismic liquefaction‐induced settlement of the free‐field ground

AbstractThe Bayesian network (BN) method has been successfully applied to evaluate earthquake liquefaction‐induced settlement of free‐field ground in recent years due to several specific advantages. However, the existing BN models need to discretize all continuous variables, and therefore, they can only predict the range value of the settlement (i.e., classification). Thus, information loss is inevitable in the process of discretization, which will largely reduce the prediction accuracy of the model. To realize the application of the BN method in the regression prediction of seismic liquefaction‐induced settlement, this study proposes a hybrid modelling method combining a hill‐climbing algorithm and domain knowledge to construct the structure of a continuous BN regression model based on historical liquefaction‐induced settlement data, and then the conditional linear Gaussian approach is used to learn the conditional probability distributions of parameters. A five‐fold cross‐validation test is used to demonstrate better generalization performance and advantages (such as considering model uncertainty and prior knowledge) of the continuous BN regression model compared with a discrete BN classification model, simplified methods such as the Tokimatsu & Seed and Ishihara & Yoshimine methods, and an artificial neural network model. Their advantages and disadvantages are discussed. In addition, the other two continuous BN models using the Arias intensity and cumulative absolute velocity instead of the peak ground acceleration perform slightly worse than the proposed continuous BN model, and the reason for this difference is discussed.

Hybrid semiparametric Bayesian networks

This paper presents a new class of Bayesian networks called hybrid semiparametric Bayesian networks, which can model hybrid data (discrete and continuous data) by mixing parametric and nonparametric estimation models. The parametric estimation models can represent a conditional linear Gaussian relationship between variables, while the nonparametric estimation model can represent other types of relationships, such as non-Gaussian and nonlinear relationships. This new class of Bayesian networks generalizes the conditional linear Gaussian Bayesian networks, including them as a special case. In addition, we describe a learning procedure for the structure and the parameters of our proposed type of Bayesian network. This learning procedure finds the best combination of parametric and nonparametric models automatically from data. This requires the definition of a cross-validated score. We also detail how new data can be sampled from a hybrid semiparametric Bayesian network, which in turn can be useful to solve other related tasks, such as inference. Furthermore, we intuitively relate our proposal with adaptive kernel density estimation models. The experimental results show that hybrid semiparametric Bayesian networks are a valuable contribution when dealing with data that do not meet the parametric assumptions that are expected for other models, such as conditional linear Gaussian Bayesian networks. We include experiments with synthetic data and real-world data from the UCI repository which demonstrate the good performance and the ability to extract useful information about the relationship between the variables in the model.

Moment-based analysis of Bayesian network properties

We use algebraic reasoning to translate Bayesian network (BN) properties into linear recurrence equations over statistical moments of BN variables. We show that this translation can always be done for various BNs, such as discrete, Gaussian, conditional linear Gaussian, and dynamic BNs. An important part of our work comes with representing BNs as while loops in probabilistic programs with polynomial assignments over random variables and parametrised distributions. We prove that closed-form summaries of probabilistic loops precisely characterize higher-order moments of BN variables. As such, we automatically solve several BN-related problems, including exact inference, sensitivity analysis, filtering, and computing the expected number of rejecting samples in sampling-based procedures. We evaluate our work on a number of BN benchmarks, using automated invariant generation within Prob-solvable loop analysis.This paper is an extended version of the “Analysis of Bayesian Networks via Prob-Solvable Loops” manuscript published at ICTAC 2020[1].

Comparing basal dendrite branches in human and mouse hippocampal CA1 pyramidal neurons with Bayesian networks

Pyramidal neurons are the most common cell type in the cerebral cortex. Understanding how they differ between species is a key challenge in neuroscience. A recent study provided a unique set of human and mouse pyramidal neurons of the CA1 region of the hippocampus, and used it to compare the morphology of apical and basal dendritic branches of the two species. The study found inter-species differences in the magnitude of the morphometrics and similarities regarding their variation with respect to morphological determinants such as branch type and branch order. We use the same data set to perform additional comparisons of basal dendrites. In order to isolate the heterogeneity due to intrinsic differences between species from the heterogeneity due to differences in morphological determinants, we fit multivariate models over the morphometrics and the determinants. In particular, we use conditional linear Gaussian Bayesian networks, which provide a concise graphical representation of the independencies and correlations among the variables. We also extend the previous study by considering additional morphometrics and by formally testing whether a morphometric increases or decreases with the distance from the soma. This study introduces a multivariate methodology for inter-species comparison of morphology.

Robust keratoconus detection with Bayesian network classifier for Placido-based corneal indices

PurposeTo evaluate in a sample of normal and keratoconic eyes a simple Bayesian network classifier for keratoconus identification that uses previously developed topographic indices, calculated directly from the digital analysis of the Placido ring images. MethodsA comparative study was performed on a total of 60 eyes from 60 patients (age 20–60 years) from the Department of keratoconus of INVISION Ophthalmology clinic (Almería, Spain). Patients were divided into two groups depending on their preliminary diagnosis based on the classical topographic criteria: a control group without topographic alteration (30 eyes) and a keratoconus group (30 eyes). The keratoconus group included all grades except grade IV with excessively distorted corneal topography. All cases were examined using the CSO topography system (CSO, Firenze, Italy), and primary corneal Placido-indices were computed, as described in literature. Finally, a classifier was built by fitting a conditional linear Gaussian Bayesian network to the data, using the 5- and 10-fold cross-validation. For comparison, the original data were perturbed with random white noise of different magnitude. ResultsThe naïve Bayes classifier showed perfect discrimination ability among normal and keratoconic corneas, with 100% of sensibility and specificity, even in the presence of a very significant noise. ConclusionsThe Bayesian network classifiers are highly accurate and proved a stable screening method to assist ophthalmologists with the detection of keratoconus, even in the presence of noise or incomplete data. This algorithm is easily implemented for any Placido topographic system.

IoT-Based Proactive Energy Supply Control for Connected Electric Vehicles

The frequent stop-and-go operations require high and fast burst driving power, which accelerates the electric vehicle batteries degradation. Hybrid electric storage system (HESS) is a promising solution, which supplements the battery with supercapacitor for rapid charging/discharging. If future power demand is available, effective power management can be done by fully exploiting the HESS benefits. Recent advances in the Internet of Things (IoT) have made the future information prediction practical, since surroundings information is obtainable. In this paper, a proactive energy management strategy is developed for the HESS with the IoT support. By analyzing the traffic data, a probabilistic graphical model, i.e., the conditional linear Gaussian (CLG), is designed for future driving information prediction. Since, the CLG prediction results are probability distributions, a scenario-tree method is developed to approximate the future power demand by sampling the possible future velocity profiles from the results. A stochastic model predictive control problem is established by incorporating the sampled trajectories. A fast dual proximal gradient method is proposed to solve the problem and facilitate real-time implementation. Simulation results demonstrate that the magnitude and fluctuation of the battery discharging power are reduced by 46.4% and 27.7%, respectively, compared with the battery only case.

Towards a Comprehensive Framework for Climate Change Multi-Risk Assessment in the Mining Industry

Changing climate conditions affect mining operations all over the world, but so far, the mining sector has focused primarily on mitigation actions. Nowadays, there exists increasing recognition of the need for planned adaptation actions. To this end, the development of a practical tool for the assessment of climate change-related risks to support the mining community is deemed necessary. In this study, a comprehensive framework is proposed for climate change multi-risk assessment at the local level customized for the needs of the mining industry. The framework estimates the climate change risks in economic terms by modeling the main activities that a mining company performs, in a probabilistic model, using Bayes’ theorem. The model permits incorporating inherent uncertainty via fuzzy logic and is implemented in two versatile ways: as a discrete Bayesian network or as a conditional linear Gaussian network. This innovative quantitative methodology produces probabilistic outcomes in monetary values estimated either as percentage of annual loss revenue or net loss/gains value. Finally, the proposed framework is the first multi-risk methodology in the mining context that considers all the relevant hazards caused by climate change extreme weather events, which offers a tool for selecting the most cost-effective action among various adaptation strategies.

Scalable importance sampling estimation of Gaussian mixture posteriors in Bayesian networks

In this paper we propose a scalable importance sampling algorithm for computing Gaussian mixture posteriors in conditional linear Gaussian Bayesian networks. Our contribution is based on using a stochastic gradient ascent procedure taking as input a stream of importance sampling weights, so that a mixture of Gaussians is dynamically updated with no need to store the full sample. The algorithm has been designed following a Map/Reduce approach and is therefore scalable with respect to computing resources. The implementation of the proposed algorithm is available as part of the AMIDST open-source toolbox for scalable probabilistic machine learning (http://www.amidsttoolbox.com).

A conditional linear Gaussian network to assess the impact of several agronomic settings on the quality of Tuscan Sangiovese grapes

Summary In this paper, a Conditional Linear Gaussian Network (CLGN) model is built for a two-year experiment on Tuscan Sangiovese grapes involving canopy management techniques (number of buds, defoliation and bunch thinning) and harvest time (technological and late harvest). We found that the impact of the considered treatments on the color of wine can be predicted still in the vegetative season of the grapevine; the best treatments to obtain wines with good structure are those with a low number of buds; the best treatments to obtain fresh wines suitable for young consumers are those with technological rather than late harvest, preferably with a high number of buds, and anyway with both defoliation and bunch thinning not performed.

Bayesian inference over ICA models: application to multibiometric score fusion with quality estimates

Bayesian networks are not well-formulated for continuous variables. The majority of recent works dealing with Bayesian inference are restricted only to special types of continuous variables such as the conditional linear Gaussian model for Gaussian variables. In this context, an exact Bayesian inference algorithm for clusters of continuous variables which may be approximated by independent component analysis models is proposed. The complexity in memory space is linear and the overfitting problem is attenuated, while the inference time is still exponential. Experiments for multibiometric score fusion with quality estimates are conducted, and it is observed that the performances are satisfactory compared to some known fusion techniques.

Risk-based water quality decision-making under small data using Bayesian network

A knowledge-based network for Section Yidong Bridge, Dongyang River, one tributary of Qiantang River, Zhejiang Province, China, is established in order to model water quality in areas under small data. Then, based on normal transformation of variables with routine monitoring data and normal assumption of variables without routine monitoring data, a conditional linear Gaussian Bayesian network is constructed. A “two-constraint selection” procedure is proposed to estimate potential parameter values under small data. Among all potential parameter values, the ones that are most probable are selected as the “representatives”. Finally, the risks of pollutant concentration exceeding national water quality standards are calculated and pollution reduction decisions for decision-making reference are proposed. The final results show that conditional linear Gaussian Bayesian network and “two-constraint selection” procedure are very useful in evaluating risks when there is limited data and can help managers to make sound decisions under small data.

Timing Foreign Exchange Markets

We take a novel approach to short-horizon exchange rate forecasting by using priced, predictable, and traded foreign exchange market factors as fundamentals. Conditional linear and Bayesian treed Gaussian process (BTGP) models with perfect foresight of these carry and dollar factors substantially outperform the random walk with respect to both squared error and profitability-related loss functions for most currencies. These results are driven primarily by information in the dollar factor. For factor timing strategies that exploit factor predictability directly, Sharpe ratios for the carry factor timing strategy are over three times those earned by timing the dollar factor, however. Simple trading strategies based on combining ex ante factor forecasts with rolling BTGP factor models for individual currencies outperform the random walk on directional accuracy, market timing, profitability, and Sharpe ratio measures for the median currency in our worldwide sample.

Two issues in using mixtures of polynomials for inference in hybrid Bayesian networks

We discuss two issues in using mixtures of polynomials (MOPs) for inference in hybrid Bayesian networks. MOPs were proposed by Shenoy and West for mitigating the problem of integration in inference in hybrid Bayesian networks. First, in defining MOP for multi-dimensional functions, one requirement is that the pieces where the polynomials are defined are hypercubes. In this paper, we discuss relaxing this condition so that each piece is defined on regions called hyper-rhombuses. This relaxation means that MOPs are closed under transformations required for multi-dimensional linear deterministic conditionals, such as Z=X+Y, etc. Also, this relaxation allows us to construct MOP approximations of the probability density functions (PDFs) of the multi-dimensional conditional linear Gaussian distributions using a MOP approximation of the PDF of the univariate standard normal distribution. Second, Shenoy and West suggest using the Taylor series expansion of differentiable functions for finding MOP approximations of PDFs. In this paper, we describe a new method for finding MOP approximations based on Lagrange interpolating polynomials (LIP) with Chebyshev points. We describe how the LIP method can be used to find efficient MOP approximations of PDFs. We illustrate our methods using conditional linear Gaussian PDFs in one, two, and three dimensions, and conditional log-normal PDFs in one and two dimensions. We compare the efficiencies of the hyper-rhombus condition with the hypercube condition. Also, we compare the LIP method with the Taylor series method.

Conditional Specification with Exponential Power Distributions

The problem of modeling Bayesian networks with continuous nodes deals with discrete approximations and conditional linear Gaussian models. In this article we have considered the possibility of using the exponential power family as conditional probability densities. It will be shown that for some platikurtic conditional distributions\\ in this family, conditional regression functions are constant. These results give conditions to avoid compatibility problems when distributions with lighter tails than the normal are used in the description of conditional densities to specify joint densities, like in Bayesian networks.

Belief update in CLG Bayesian networks with lazy propagation

In recent years, Bayesian networks with a mixture of continuous and discrete variables have received an increasing level of attention. In this paper, we focus on the restricted class of mixture Bayesian networks known as conditional linear Gaussian Bayesian networks (CLG Bayesian networks) and present an architecture for exact belief update for this class of mixture networks. The proposed architecture is an extension of lazy propagation using operations of Lauritzen and Jensen [S.L. Lauritzen, F. Jensen, Stable local computation with mixed Gaussian distributions, Statistics and Computing 11(2) (2001) 191–203] and Cowell [R.G. Cowell, Local propagation in conditional Gaussian Bayesian networks, Journal of Machine Learning Research 6 (2005) 1517–1550]. By decomposing clique and separator potentials into sets of factors, the proposed architecture takes advantage of independence and irrelevance properties induced by the structure of the graph and the evidence. The resulting benefits are illustrated by examples and assessed by experiments. The performance of the proposed architecture has been evaluated using a set of randomly generated networks. The results indicate a significant potential of the proposed architecture.

Operations for inference in continuous Bayesian networks with linear deterministic variables

An important class of continuous Bayesian networks are those that have linear conditionally deterministic variables (a variable that is a linear deterministic function of its parents). In this case, the joint density function for the variables in the network does not exist. Conditional linear Gaussian (CLG) distributions can handle such cases when all variables are normally distributed. In this paper, we develop operations required for performing inference with linear conditionally deterministic variables in continuous Bayesian networks using relationships derived from joint cumulative distribution functions. These methods allow inference in networks with linear deterministic variables and non-Gaussian distributions.

Inference in hybrid Bayesian networks with mixtures of truncated exponentials

Mixtures of truncated exponentials (MTE) potentials are an alternative to discretization for solving hybrid Bayesian networks. Any probability density function (PDF) can be approximated with an MTE potential, which can always be marginalized in closed form. This allows propagation to be done exactly using the Shenoy–Shafer architecture for computing marginals, with no restrictions on the construction of a join tree. This paper presents MTE potentials that approximate an arbitrary normal PDF with any mean and a positive variance. The properties of these MTE potentials are presented, along with examples that demonstrate their use in solving hybrid Bayesian networks. Assuming that the joint density exists, MTE potentials can be used for inference in hybrid Bayesian networks that do not fit the restrictive assumptions of the conditional linear Gaussian (CLG) model, such as networks containing discrete nodes with continuous parents.

A sequential monte carlo filter for joint linear/nonlinear state estimation with application to DS-CDMA

A sequential Monte Carlo filter is considered which combines previously developed sequential importance sampling (SIS) techniques for conditional linear Gaussian models with measurement linearization for construction of approximate simulation densities. The resulting sequential Monte Carlo Kalman filter (SMC-KF) consists of a bank of conventional Kalman filters individually tuned to sampled trajectories of the nonlinear state variables. Sampling is according to a Gaussian distribution, with mean and covariance determined by extended Kalman filter-type equations. The SMC-KF is then applied to joint delay and multipath channel estimation in direct-sequence code-division multiple access (DS-CDMA). A combined analytical/simulation technique is employed to compare performance of the SMC-KF and a previously derived extended Kalman filter (EKF)-based DS-CDMA channel estimator.

Experiments on the application of IOHMMs to model financial returns series

Input-output hidden Markov models (IOHMM) are conditional hidden Markov models in which the emission (and possibly the transition) probabilities can be conditioned on an input sequence. For example, these conditional distributions can be linear, logistic, or nonlinear (using for example multilayer neural networks). We compare the generalization performance of several models which are special cases of input-output hidden Markov models on financial time-series prediction tasks: an unconditional Gaussian, a conditional linear Gaussian, a mixture of Gaussians, a mixture of conditional linear Gaussians, a hidden Markov model, and various IOHMMs. The experiments compare these models on predicting the conditional density of returns of market and sector indices. Note that the unconditional Gaussian estimates the first moment with the historical average. The results show that, although for the first moment the historical average gives the best results, for the higher moments, the IOHMMs yielded significantly better performance, as estimated by the out-of-sample likelihood.