Conditional Gaussian Research Articles

Computationally efficient and error aware surrogate construction for numerical solutions of subsurface flow through porous media

Limiting the injection rate to restrict the pressure below a threshold at a critical location can be an important goal of simulations that model the subsurface pressure between injection and extraction wells. The pressure is approximated by the solution of Darcy’s partial differential equation for a given permeability field. The subsurface permeability is modeled as a random field since it is known only up to statistical properties. This induces uncertainty in the computed pressure. Solving the partial differential equation for an ensemble of random permeability simulations enables estimating a probability distribution for the pressure at the critical location. These simulations are computationally expensive, and practitioners often need rapid online guidance for real-time pressure management. An ensemble of numerical partial differential equation solutions is used to construct a Gaussian process regression model that can quickly predict the pressure at the critical location as a function of the extraction rate and permeability realization. The Gaussian process surrogate analyzes the ensemble of numerical pressure solutions at the critical location as noisy observations of the true pressure solution, enabling robust inference using the conditional Gaussian process distribution.Our first novel contribution is to identify a sampling methodology for the random environment and matching kernel technology for which fitting the Gaussian process regression model scales as O(nlogn) instead of the typical O(n3) rate in the number of samples n used to fit the surrogate. The surrogate model allows almost instantaneous predictions for the pressure at the critical location as a function of the extraction rate and permeability realization. Our second contribution is a novel algorithm to calibrate the uncertainty in the surrogate model to the discrepancy between the true pressure solution of Darcy’s equation and the numerical solution. Although our method is derived for building a surrogate for the solution of Darcy’s equation with a random permeability field, the framework broadly applies to solutions of other partial differential equations with random coefficients.

A Barndorff-Nielsen and Shephard model with leverage in Hilbert space for commodity forward markets

We propose an extension of the model introduced by Barndorff-Nielsen and Shephard, based on stochastic processes of Ornstein–Uhlenbeck type taking values in Hilbert spaces and including the leverage effect. We compute explicitly the characteristic function of the log-return and the volatility processes. By introducing a measure change of Esscher type, we provide a relation between the dynamics described with respect to the historical and the risk-neutral measures. We discuss in detail the application of the proposed model to describe the commodity forward curve dynamics in a Heath–Jarrow–Morton framework, including the modelling of forwards with delivery period occurring in energy markets and the pricing of options. For the latter, we show that a Fourier approach can be applied in this infinite-dimensional setting, relying on the attractive property of conditional Gaussianity of our stochastic volatility model. In our analysis, we study both arithmetic and geometric models of forward prices and provide appropriate martingale conditions in order to ensure arbitrage-free dynamics.

Imaging Spectroscopy‐Based Estimation of Aboveground Biomass in Louisiana's Coastal Wetlands: Toward Consistent Spectroscopic Retrievals Across Atmospheric States

AbstractDeveloping accurate landscape‐scale aboveground biomass (AGB) maps is critical to understanding coastal deltaic wetland resilience, as AGB influences stability and elevation dynamics in herbaceous wetlands. Here we used AVIRIS‐NG imaging spectrometer (or “hyperspectral”) data from NASA's 2021 Delta‐X mission in coastal Louisiana to map seasonal changes in herbaceous AGB across two deltaic basins with contrasting sediment delivery and hydrologic regimes: the Atchafalaya (active) and Terrebonne (inactive). We assessed the impact of atmospheric effects on our retrievals, as high water vapor content in August 2021 caused significant noise in the 880–1,000 and 1,080–1,200 nm near‐infrared (NIR) wavelengths. We hypothesized that correcting these wavelengths with our conditional Gaussian interpolation algorithm would improve AGB estimates due to their association with plant canopy water content. We empirically assessed the performance of the corrected spectra on AGB estimates using Partial Least Squares Regression (PLSR), finding that the corrected NIR bands attained high variable importance and reduced estimation errors. Our Random Forest regression approach based on the corrected spectra attained equivalent error metrics via leave‐one‐out‐cross‐validation as the PLSR models (R2 = 0.43, mean absolute error = 257.3 g/m2) while greatly improving the AGB maps' visual quality, having better captured variability while reducing noise and discontinuities in AGB estimates across flightlines. The maps show differing seasonal growth, with the Atchafalaya and Terrebonne Basins' AGB increasing from means of 4.3–9.4 and 4.6–8.9 Mg/ha, respectively. We demonstrated that imaging spectroscopy can be applied to assess herbaceous biomass stocks, growth patterns, and resilience in coastal ecosystems.

Software application profile: tpc and micd-R packages for causal discovery with incomplete cohort data.

The Peter Clark (PC) algorithm is a popular causal discovery method to learn causal graphs in a data-driven way. Until recently, existing PC algorithm implementations in R had important limitations regarding missing values, temporal structure or mixed measurement scales (categorical/continuous), which are all common features of cohort data. The new R packages presented here, micd and tpc, fill these gaps. micd and tpc packages are R packages. The micd package provides add-on functionality for dealing with missing values to the existing pcalg R package, including methods for multiple imputations relying on the Missing At Random assumption. Also, micd allows for mixed measurement scales assuming conditional Gaussianity. The tpc package efficiently exploits temporal information in a way that results in a more informative output that is less prone to statistical errors. The tpc and micd packages are freely available on the Comprehensive R Archive Network (CRAN). Their source code is also available on GitHub (https://github.com/bips-hb/micd; https://github.com/bips-hb/tpc).

Additive dynamic Bayesian networks for enhanced feature learning in soft sensor modeling

Due to the advantages of indicating variable structure and efficient reasoning, Bayesian Networks (BN) have been widely used in data-driven soft sensor applications. However, restricted to linear and conditional Gaussian property, BN-based soft sensors rarely achieve high prediction accuracy. In this paper, an ensemble learning framework – additive dynamic Bayesian networks (ADBN) is proposed for enhanced feature learning, in which Dynamic Bayesian networks are used to learn the conditional independencies among variables and construct feature sets for the following base learners. Additional DBNs are constructed upon the residual information from the past model, to carry out feature learning to fit the residuals. The procedure is repeated and a termination rule from the perspective of feature learning is proposed to end this process, and thus the model complexity can be well restricted. The proposed method is validated on two actual industrial cases. It reveals that the ADBN feature learning method has obtained great improvements. Compared to the single DBN feature engineering method, the root mean square error (RMSE) performance has been improved by 20% and 13% on the two cases, respectively.

CGNSDE: Conditional Gaussian neural stochastic differential equation for modeling complex systems and data assimilation

A new knowledge-based and machine learning hybrid modeling approach, called conditional Gaussian neural stochastic differential equation (CGNSDE), is developed to facilitate modeling complex dynamical systems and implementing analytic formulae of the associated data assimilation (DA). In contrast to the standard neural network predictive models, the CGNSDE is designed to effectively tackle both forward prediction tasks and inverse state estimation problems. The CGNSDE starts by exploiting a systematic causal inference via information theory to build a simple knowledge-based nonlinear model that nevertheless captures as much explainable physics as possible. Then, neural networks are supplemented to the knowledge-based model in a specific way, which not only characterizes the remaining features that are challenging to model with simple forms but also advances the use of analytic formulae to efficiently compute the nonlinear DA solution. These analytic formulae are used as an additional computationally affordable loss to train the neural networks that directly improve the DA accuracy. This DA loss function promotes the CGNSDE to capture the interactions between state variables and thus advances its modeling skills. With the DA loss, the CGNSDE is more capable of estimating extreme events and quantifying the associated uncertainty. Furthermore, crucial physical properties in many complex systems, such as the translate-invariant local dependence of state variables, can significantly simplify the neural network structures and facilitate the CGNSDE to be applied to high-dimensional systems. Numerical experiments based on chaotic systems with intermittency and strong non-Gaussian features indicate that the CGNSDE outperforms knowledge-based regression models, and the DA loss further enhances the modeling skills of the CGNSDE.

Contingency analysis with warm starter using probabilistic graphical model

Cyberthreats are an increasingly common risk to the power grid and can thwart secure grid operations. We propose to extend contingency analysis to include cyberthreat evaluations. However, unlike the traditional N-1 or N-2 contingencies, cyberthreats (e.g., MadIoT) require simulating hard-to-solve N-k (with k ≫ 2) contingencies in a practical amount of time. Purely physics-based power flow solvers, while being accurate, are slow and may not solve N-k contingencies in a timely manner, whereas the emerging data-driven alternatives are fast but not sufficiently generalizable, interpretable, and scalable. To address these challenges, we propose a novel conditional Gaussian Random Field-based data-driven method that performs fast and accurate evaluation of cyberthreats. It achieves speedup of contingency analysis by warm-starting simulations, i.e., improving starting points, for the physical solvers. To improve the physical interpretability and generalizability, the proposed method incorporates domain knowledge by considering the graphical nature of the grid topology. To improve scalability, the method applies physics-informed regularization that reduces model complexity. Experiments validate that simulating MadIoT-induced attacks with our warm starter becomes approximately 5x faster on a realistic 2000-bus system.

A novel fault diagnosis method for Bayesian networks fusing models and data

Data-driven methods are the mainstream methods applied in current fault diagnosis research; however, in natural mechanical systems, fault samples are difficult or even impossible to obtain, and physical model-based methods cannot easily model the faults of complex systems. To address the respective limitations of data-driven and model diagnosis methods in fault diagnosis, in this paper, a Bayesian network fault diagnosis method based on a joint model and data modelling is proposed. Conditional Gaussian Bayesian networks are used to fuse discrete and continuous information, and the discrete information in the residuals is used to compensate for the insufficient feature information problem caused by small amounts of data. To address the exponential modeling complexity growth in the case of multiple fault types cases, a single classification method is chosen for fault classification. Meanwhile, the T2 statistics distance suppression method is introduced into the Bayesian network to improve the diagnostic capabilities for new types of faults. By comparing the mainstream methods that address small sample diagnosis and the existing fusion methods, the experimental results show that in cases with small numbers of samples, the method in this paper has good fault diagnosis capabilities by effectively utilizing the model’s diagnostic results to compensate for the diagnostic errors caused by a lack of data, and it has an improved ability to recognize unknown faults.

Bayesian inference of graph-based dependencies from mixed-type data

Mixed data comprise measurements of different types, with both categorical and continuous variables, and can be found in various areas, such as in life science or industrial processes. Inferring conditional independencies from the data is crucial to understand how these variables relate to each other. To this end, graphical models provide an effective framework, which adopts a graph-based representation of the joint distribution to encode such dependence relations. This framework has been extensively studied in the Gaussian and categorical settings separately; on the other hand, the literature addressing this problem in presence of mixed data is still narrow. We propose a Bayesian model for the analysis of mixed data based on the notion of Conditional Gaussian (CG) distribution. Our method is based on a canonical parameterization of the CG distribution, which allows for posterior inference of parameters indexing the (marginal) distributions of continuous and categorical variables, as well as expressing the interactions between the two types of variables. We derive the limiting Gaussian distributions, centered on the correct unknown value and with vanishing variance, for the Bayesian estimators of the canonical parameters expressing continuous, discrete and mixed interactions. In addition, we implement the proposed method for structure learning purposes, namely to infer the underlying graph of conditional independencies. When compared to alternative frequentist methods, our approach shows favorable results both in a simulation setting and in real-data applications, besides allowing for a coherent uncertainty quantification around parameter estimates.

Ordinal Unsupervised Domain Adaptation With Recursively Conditional Gaussian Imposed Variational Disentanglement.

There has been a growing interest in unsupervised domain adaptation (UDA) to alleviate the data scalability issue, while the existing works usually focus on classifying independently discrete labels. However, in many tasks (e.g., medical diagnosis), the labels are discrete and successively distributed. The UDA for ordinal classification requires inducing non-trivial ordinal distribution prior to the latent space. Target for this, the partially ordered set (poset) is defined for constraining the latent vector. Instead of the typically i.i.d. Gaussian latent prior, in this work, a recursively conditional Gaussian (RCG) set is proposed for ordered constraint modeling, which admits a tractable joint distribution prior. Furthermore, we are able to control the density of content vectors that violate the poset constraint by a simple "three-sigma rule". We explicitly disentangle the cross-domain images into a shared ordinal prior induced ordinal content space and two separate source/target ordinal-unrelated spaces, and the self-training is worked on the shared space exclusively for ordinal-aware domain alignment. Extensive experiments on UDA medical diagnoses and facial age estimation demonstrate its effectiveness.

VaR calculation by binary response models

<p>The original Risk-Metrics method is underpinned by the assumption that daily asset returns are conditional Gaussian independently identically distributed (iid) random variables with a mean of zero. In this paper, a new method to calculate Value at Risk (VaR) was suggested to overcome the shortcoming of Risk-Metrics by employing binary response models to compute probability forecasts of the portfolio return by exceeding a grid of candidate quantile values. From those values, the VaR quantile value was selected. The proposed model was called BRV (Binary Response VaR method). Consistent application of BRV to the Dow Jones Industrial Average (INDEXDJX: DJI) and Dow Jones U.S. Marine Transportation Index (DJUSMT) time series proved that it was more accurate than the Risk-Metric system. This method not only worked similar to quantile regression but had the advantage that conventional maximum likelihood methods could be used for parameter estimation and inference. The BRV method was the best performing method for computing the daily VaR at both the 95% and 99% confidence levels over the period 02/01/06–31/12/08. The BRV and the QR (quantile regression) methods performed similarly, but the BRV method had the practical advantage that conventional maximum likelihood (ML) technique could be used for parameter estimation and robust inference.</p>

Uncertainty in Land Value Modeling of the San José Metropolitan Region, Costa Rica

Land value patterns show very distinct spatial associations with accessibility to urban centralities and physical factors in a territory. However, predictions based on models of this structure can be highly uncertain, as the underlying data also may show clustering (thus allowing for better predictions in more densely sampled areas). An assessment of this uncertainty for land value extrapolations in the the San José Metropolitan Region of Costa Rica is presented, via conditional Gaussian simulation, and the determinants of this uncertainty were explored, to find spatial strengths and weaknesses in the modeling efforts. The E-Type prediction from the conditional Gaussian simulation was found to marginally improve on ordinary kriging methods and it also provided explicit uncertainty patterns, which are the inverse of the land value prediction. The estimated uncertainty was found to decrease with characteristics that identify suitability for urban land use (and thus higher land values).

A physics-informed data-driven algorithm for ensemble forecast of complex turbulent systems

A new ensemble forecast algorithm, named the physics-informed data-driven algorithm with conditional Gaussian statistics (PIDD-CG), is developed to predict the probability density functions (PDFs) of complex turbulent systems with partial observations. The PIDD-CG algorithm integrates a unique multiscale statistical closure modeling strategy with a highly efficient nonlinear data assimilation scheme to create a mixture of conditional statistics. An effective data-driven modeling method is integrated with the dominant conditional statistics to serve as the forecast ensemble members that can significantly reduce the high computational cost of recovering high-dimensional PDFs. The multiscale features in the time evolution of these conditional statistics ensembles are effectively predicted by an appropriate combination of physics-informed analytic formulae and recurrent neural networks. An information metric is adopted as the loss function for the latter to more accurately capture the desirable turbulent features. The proposed algorithm displays effective forecasting performance in both the transient and statistical equilibrium non-Gaussian PDFs of strongly turbulent systems with intermittency, regime switching, and extreme events. It also facilitates the development of efficient statistical reduced-order models in recovering and predicting the large-scale coherent structures of a large group of multiscale complex systems.

A DYNAMIC SPATIAL FILTERING APPROACH TO MITIGATE UNDERESTIMATION BIAS IN FIELD CALIBRATED LOW-COST SENSOR AIR POLLUTION DATA.

Low-cost air pollution sensors, offering hyper-local characterization of pollutant concentrations, are becoming increasingly prevalent in environmental and public health research. However, low-cost air pollution data can be noisy, biased by environmental conditions, and usually need to be field-calibrated by collocating low-cost sensors with reference-grade instruments. We show, theoretically and empirically, that the common procedure of regression-based calibration using collocated data systematically underestimates high air pollution concentrations, which are critical to diagnose from a health perspective. Current calibration practices also often fail to utilize the spatial correlation in pollutant concentrations. We propose a novel spatial filtering approach to collocation-based calibration of low-cost networks that mitigates the underestimation issue by using an inverse regression. The inverse-regression also allows for incorporating spatial correlations by a second-stage model for the true pollutant concentrations using a conditional Gaussian Process. Our approach works with one or more collocated sites in the network and is dynamic, leveraging spatial correlation with the latest available reference data. Through extensive simulations, we demonstrate how the spatial filtering substantially improves estimation of pollutant concentrations, and measures peak concentrations with greater accuracy. We apply the methodology for calibration of a low-cost PM2.5 network in Baltimore, Maryland, and diagnose air pollution peaks that are missed by the regression-calibration.

Bayesian network analysis of BIN1 risk allele and other risk factors and biomarkers of Alzheimer’s disease

AbstractBackgroundThe Bridging Integrator I (BIN1) is a significant genetic risk gene, second only to the established risk gene apolipoprotein E (APOE4), for late‐onset Alzheimer’s disease (LOAD, AD) (Carrasquillo, 2011). BIN1 rs744733 risk allele was shown to affect tau pathology and cognitive decline (Franzmeier, 2021).In this study, we compared BIN1 risk allele rs6733839 with APOE4 on interdependency among AD risk factors and biomarkers by applying Bayesian Network (BN) modeling (Scutari, 2017) to data from Lilly AD clinical trials.MethodData from A05C (n = 142, NCT01565382), A05E (n = 217, NCT02016560), and untreated arms of AZES (n = 206, NCT02245737), LLCF (n = 229, NCT02791191), and LZAX (n = 194, NCT01900665) trial participants were used for the analysis (table 1). We constructed graphs representing the dependencies among the variables, including demographics, genetic risk genes, brain amyloid and tau measured by 18F‐Florbetapir or 18F‐Flortaucipir positron emission tomography (PET) imaging quantified globally and regionally as a standardized uptake value ratio. Longitudinal tau changes from baseline were computed for a sub cohort (n = 234). Conditional Gaussian Bayesian Network (CGBN) was used to model discrete (gender, genotype) and continuous (age, PET quantifications) variables. Directed Acyclic Graphs (DAG) structure was learned using constraint‐based Incremental Association (iamb) algorithm and score‐based Hill‐Climbing (hc) algorithm with one‐sided type I error rate at 10%. The connection strength was estimated by bootstrap resampling and computing the probability of any variable influencing another variable. Model parameters were estimated using the maximum likelihood estimation method.ResultThe result shows that age influences baseline amyloid (Strength s = 1.0) and parietal tau (s = 0.83). APOE4 influences baseline amyloid (s = 0.75) and tau at four lopes with strength ranging from 0.84 to 0.31 in the order of frontal, parietal, occipital, and temporal lobes. BIN1 influences baseline tau at the parietal lobe (s = 0.42) and other lobes with strength below 0.3. We have not observed consistent dependency between variables and change in tau, likely due to the small sample size (n = 234).ConclusionOur study result is consistent with current knowledge of APOE4 influencing amyloid and tau. The finding that BIN1 influences tau supports the observation of BIN1 rs744373, in linkage disequilibrium with rs6733839, in ADNI and BioFINDER studies (Franzmeier, 2021).

Healthcare Cost Prediction Based on Hybrid Machine Learning Algorithms

Healthcare cost is an issue of concern right now. While many complex machine learning algorithms have been proposed to analyze healthcare cost and address the shortcomings of linear regression and reliance on expert analyses, these algorithms do not take into account whether each characteristic variable contained in the healthcare data has a positive effect on predicting healthcare cost. This paper uses hybrid machine learning algorithms to predict healthcare cost. First, network structure learning algorithms (a score-based algorithm, constraint-based algorithm, and hybrid algorithm) for a Conditional Gaussian Bayesian Network (CGBN) are used to learn the isolated characteristic variables in healthcare data without changing the data properties (i.e., discrete or continuous). Then, the isolated characteristic variables are removed from the original data and the remaining data used to train regression algorithms. Two public healthcare datasets are used to test the performance of the proposed hybrid machine learning algorithm model. Experiments show that when compared to popular single machine learning algorithms (Long Short Term Memory, Random Forest, etc.) the proposed scheme can obtain similar or higher prediction accuracy with a reduced amount of data.

Open-set lung sound recognition model based on conditional Gaussian capsule network and variational time–frequency feature reconstruction

Lung sound auscultation is an essential method for diagnosing lung diseases; however, most existing lung sound recognition methods fail to identify classes that are unknown in training. Thus, we proposed an open-set lung sound recognition model based on the conditional Gaussian capsule network and variational time–frequency feature reconstruction. The proposed model incorporates an inference network, a cubic encoder, an attention module, a classifier, a cubic decoder, and a generative network. First, the inference network is employed to extract the time–frequency features of lung sounds at a single time step. Then, the variational distribution of lung sounds is computed using the capsule network and optimized to approximate the Gaussian model of the class to which the sample belongs according to the labels. Time–frequency synchronized feature extraction and reconstruction are performed on the entire lung sound sample using the cubic encoder and cubic decoder. Finally, we utilize the generative network to refactor the lung sound features for open-set recognition. The proposed model was evaluated experimentally on a combined dataset using two different category assignment schemes. The results demonstrate that the proposed model achieved accuracies of 82.31% and 88.47%, respectively, thereby outperforming existing methods.

Collaborative and Adaptive Bayesian Optimization for bounding variances and probabilities under hybrid uncertainties

Uncertainty quantification (UQ) has been widely recognized as of vital importance for reliability-oriented analysis and design of engineering structures, and three groups of mathematical models, i.e., the probability models, the imprecise probability models and the non-probabilistic models, have been developed for characterizing uncertainties of different forms. The propagation of these three groups of models through expensive-to-evaluate simulators to quantify the uncertainties of outputs is then one of the core, yet highly challenging task in reliability engineering, as it involves a demanding double-loop numerical dilemma. For addressing this issue, the Collaborative and Adaptive Bayesian Optimization (CABO) has been developed in our previous work, but it only applies to imprecise probability models and is only capable of bounding the output expectation. We present a substantial improvement of CABO to incorporate all three categories of uncertainty models and to bound arbitrary probabilistic measures such as output variance and failure probability. The algorithm is based on a collaborative active learning mechanism, that is, jointly performing Bayesian optimization in the epistemic uncertainty subspace and Bayesian cubature in the aleatory uncertainty subspace, thus allowing to adaptively produce training samples in the joint uncertainty space. An efficient conditional Gaussian process simulation algorithm is embedded in CABO for acquiring training points and Bayesian inference in both uncertain subspaces. Benchmark studies show that CABO exhibits a remarkable performance in terms of numerical efficiency, accuracy, and global convergence.

The conditional Gaussian multiplicative chaos structure underlying a critical continuum random polymer model on a diamond fractal

The conditional Gaussian multiplicative chaos structure underlying a critical continuum random polymer model on a diamond fractal

Improved environmental mapping and validation using bagging models with spatially clustered data

Spatially clustered sampling may result in non-independent data that pose challenges for environmental mapping applications. Two outstanding challenges resulting from the use of spatially clustered data for predictive geospatial modelling with machine learning approaches are biased model training and validation. These issues can be severe for popular bagging models such as Random Forest, yet one or both are often ignored or are handled using sub-optimal approaches. We propose to address these challenges using information on both the spatial autocorrelation of map errors and the spatial sampling intensity. This is achieved by applying the residual spatial covariance as a weighting function for the bagging procedure and for the calculation of weighted validation statistics. Using this approach, the full feature space of the sample data is retained during model training and validation. The utility of covariance weighting for these purposes is investigated through extensive simulation with a range of sample clustering configurations. Results are benchmarked against existing approaches. Covariance weighting improved model performance across a range of clustering scenarios but appeared to produce the greatest improvements for highly clustered data. Covariance-weighted validation demonstrated low bias across a broad range of clustering scenarios compared to existing spatial methods. Findings also suggest, though, that conditional Gaussian simulation approaches may perform well when the proportion of clustered data is very high. Covariance weighting is straightforward to implement, computationally efficient, and scales to different sample sizes and spatial extents.