Conditional Gaussian Research Articles

A Conditional Multiscale Locally Gaussian Texture Synthesis Algorithm

Exemplar-based texture synthesis is defined as the process of generating, from an input texture sample, new texture images that are perceptually equivalent to the input. In the present work, we model texture self-similarity with conditional Gaussian distributions in the patch space in order to extend the use of stitching techniques. Then, a multiscale texture synthesis algorithm is introduced, where texture patches are modeled at each scale as spatially variable Gaussian vectors in the patch space. The Gaussian distribution for each patch is inferred from the set of its nearest neighbors in the patch space obtained from the input sample. This approach is tested over several real and synthetic texture images, and its results show the effectiveness of the proposed technique for a wide range of textures.

SMURC: High-Dimension Small-Sample Multivariate Regression With Covariance Estimation.

We consider a high-dimension low sample-size multivariate regression problem that accounts for correlation of the response variables. The system is underdetermined as there are more parameters than samples. We show that the maximum likelihood approach with covariance estimation is senseless because the likelihood diverges. We subsequently propose a normalization of the likelihood function that guarantees convergence. We call this method small-sample multivariate regression with covariance (SMURC) estimation. We derive an optimization problem and its convex approximation to compute SMURC. Simulation results show that the proposed algorithm outperforms the regularized likelihood estimator with known covariance matrix and the sparse conditional Gaussian graphical model. We also apply SMURC to the inference of the wing-muscle gene network of the Drosophila melanogaster (fruit fly).

Gradient Scan Gibbs Sampler: An Efficient Algorithm for High-Dimensional Gaussian Distributions

This paper deals with Gibbs samplers that include high dimensional conditional Gaussian distributions. It proposes an efficient algorithm that avoids the high dimensional Gaussian sampling and relies on a random excursion along a small set of directions. The algorithm is proved to converge, i.e. the drawn samples are asymptotically distributed according to the target distribution. Our main motivation is in inverse problems related to general linear observation models and their solution in a hierarchical Bayesian framework implemented through sampling algorithms. It finds direct applications in semi-blind/unsupervised methods as well as in some non-Gaussian methods. The paper provides an illustration focused on the unsupervised estimation for super-resolution methods.

Conditional Fractional Gaussian Fields with the Package FieldSim

Conditional Fractional Gaussian Fields with the Package FieldSim

Belief propagation in genotype-phenotype networks.

Graphical models have proven to be a valuable tool for connecting genotypes and phenotypes. Structural learning of phenotype-genotype networks has received considerable attention in the post-genome era. In recent years, a dozen different methods have emerged for network inference, which leverage natural variation that arises in certain genetic populations. The structure of the network itself can be used to form hypotheses based on the inferred direct and indirect network relationships, but represents a premature endpoint to the graphical analyses. In this work, we extend this endpoint. We examine the unexplored problem of perturbing a given network structure, and quantifying the system-wide effects on the network in a node-wise manner. The perturbation is achieved through the setting of values of phenotype node(s), which may reflect an inhibition or activation, and propagating this information through the entire network. We leverage belief propagation methods in Conditional Gaussian Bayesian Networks (CG-BNs), in order to absorb and propagate phenotypic evidence through the network. We show that the modeling assumptions adopted for genotype-phenotype networks represent an important sub-class of CG-BNs, which possess properties that ensure exact inference in the propagation scheme. The system-wide effects of the perturbation are quantified in a node-wise manner through the comparison of perturbed and unperturbed marginal distributions using a symmetric Kullback-Leibler divergence. Applications to kidney and skin cancer expression quantitative trait loci (eQTL) data from different mus musculus populations are presented. System-wide effects in the network were predicted and visualized across a spectrum of evidence. Sub-pathways and regions of the network responded in concert, suggesting co-regulation and coordination throughout the network in response to phenotypic changes. We demonstrate how these predicted system-wide effects can be examined in connection with estimated class probabilities for covariates of interest, e.g. cancer status. Despite the uncertainty in the network structure, we demonstrate the system-wide predictions are stable across an ensemble of highly likely networks. A software package, geneNetBP, which implements our approach, was developed in the R programming language.

Intermittency in turbulent diffusion models with a mean gradient

This paper provides a rigorous, self contained analysis of the intermittent behavior of turbulent diffusion models with a mean gradient. The intermittency can be described as large spikes randomly occurring in the time sequence of a passive tracer or exponential like fat tails in the probability density function. This type of passive tracer intermittency is subtle and occurs without any positive Lyapunov exponents in the system. Observations of such passive tracers in nature also show such intermittency. By exploiting an intrinsic conditional Gaussian structure, the enormous fluctuation in conditional variance of the passive tracer is found to be the source of intermittency in these models. An intuitive physical interpretation of such enormous fluctuation can be described through the random resonance between Fourier modes of the turbulent velocity field and the passive tracer. This intuition can be rigorously proved in a long time slow varying limit, where the limiting distribution of the passive tracer is computed through an integral formula. This leads to rigorous predictions of various types of intermittency. Numerical experiments are conducted in different dynamical regimes to verify and supplement all the theoretical results. All the proofs in this paper are elementary and essentially self contained.

Fault detection with Conditional Gaussian Network

The main interest of this paper is to illustrate a new representation of the Principal Component Analysis (PCA) for fault detection under a Conditional Gaussian Network (CGN), a special case of Bayesian networks. PCA and its associated quadratic statistics such as T2 and SPE are integrated under a sole CGN. The proposed framework projects a new observation into an orthogonal space and gives probabilities on the state of the system. It could do so even when some data in the sample test are missing. This paper also gives the probabilities thresholds to use in order to match quadratic statistics decisions. The proposed network is validated and compared to the standard PCA scheme for fault detection on the Tennessee Eastman Process and the Hot Forming Process.

Sparse inverse covariance learning of conditional Gaussian mixtures for multiple-output regression

We consider the task of multiple-output regression where both input and output are high-dimensional. Due to the limited amount of training samples compared to data dimensions, properly imposing loose statistical dependency in learning a regression model is crucial for reliable prediction accuracy. The sparse inverse covariance learning of conditional Gaussian random fields has been recently emerging to achieve this goal, shown to exhibit superior performance to non-sparse approaches. However, one of its main drawbacks is the strong assumption of linear Gaussianity in modeling the input-output relationship. For certain application domains, the assumption might be too restricted and less powerful in representation, and consequently, prediction based on the wrong models can result in suboptimal performance. In this paper, we extend the idea of sparse learning to a non-Gaussian model, especially the powerful conditional Gaussian mixture. For this latent-variable model, we propose a novel sparse inverse covariance learning algorithm based on the expectation-maximization lower-bound optimization technique. It is shown that each M-step reduces to solving the regular sparse inverse covariance estimation of linear Gaussian models, in conjunction with estimating sparse logistic regression. We demonstrate the improved prediction performance of the proposed algorithm over exisitng methods on several datasets.

Exploring Multimodal Data Fusion Through Joint Decompositions with Flexible Couplings

A Bayesian framework is proposed to define flexible coupling models for joint tensor decompositions of multiple data sets. Under this framework, a natural formulation of the data fusion problem is to cast it in terms of a joint maximum a posteriori (MAP) estimator. Data driven scenarios of joint posterior distributions are provided, including general Gaussian priors and non Gaussian coupling priors. We present and discuss implementation issues of algorithms used to obtain the joint MAP estimator. We also show how this framework can be adapted to tackle the problem of joint decompositions of large datasets. In the case of a conditional Gaussian coupling with a linear transformation, we give theoretical bounds on the data fusion performance using the Bayesian Cramer-Rao bound. Simulations are reported for hybrid coupling models ranging from simple additive Gaussian models, to Gamma-type models with positive variables and to the coupling of data sets which are inherently of different size due to different resolution of the measurement devices.

Causality between inflation and inflation uncertainty in South Africa: Evidence from a Markov-switching vector autoregressive model

This study investigates the asymmetric and time-varying causalities between inflation and inflation uncertainty in South Africa within a conditional Gaussian Markov switching vector autoregressive (MS-VAR) model framework. The MS-VAR model is capable of determining both the sign and direction of causality. We account for the nonlinear, long memory and seasonal features of the inflation series simultaneously by measuring inflation uncertainty as the conditional variance of inflation generated by recursive estimation of a Seasonal Fractionally Integrated Smooth Transition Autoregressive Asymmetric Power GARCH (SEA-FISTAR-APGARCH) model using monthly data for the period 1921:01 to 2012:12. The recursive, rather than full-sample, estimation allows us to obtain a time-varying measure of uncertainty and better mimics the real-time scenario faced by economic agents and/or policy makers. The inferred probabilities from the four-state MS-VAR model show evidence of a time-varying relationship. The conditional (i.e. lead–lag) and regime-prediction Granger causality provide evidence in favor of Friedman's hypothesis. This implies that past information on inflation can help improve the one-step-ahead prediction of inflation uncertainty but not vice versa. Our results have some important policy implications.

The metabolomics of asthma control: a promising link between genetics and disease.

Short-acting β agonists (e.g., albuterol) are the most commonly used medications for asthma, a disease that affects over 300 million people in the world. Metabolomic profiling of asthmatics taking β agonists presents a new and promising resource for identifying the molecular determinants of asthma control. The objective is to identify novel genetic and biochemical predictors of asthma control using an integrative “omics” approach. We generated lipidomic data by liquid chromatography tandem mass spectrometry (LC-MS), ­ using plasma samples from 20 individuals with asthma. The outcome of interest was a binary indicator of asthma control defined by the use of albuterol inhalers in the preceding week. We integrated metabolomic data with genome-wide genotype, gene expression, and methylation data of this cohort to identify genomic and molecular indicators of asthma control. A Conditional Gaussian Bayesian Network (CGBN) was generated using the strongest predictors from each of these analyses. Integrative and metabolic pathway over-representation analyses (ORA) identified enrichment of known biological pathways within the strongest molecular determinants. Of the 64 metabolites measured, 32 had known identities. The CGBN model based on four SNPs (rs9522789, rs7147228, rs2701423, rs759582) and two metabolites—monoHETE_0863 and sphingosine-1-phosphate (S1P) could predict asthma control with an AUC of 95%. Integrative ORA identified 17 significantly enriched pathways related to cellular immune response, interferon signaling, and cytokine-related signaling, for which arachidonic acid, PGE2 and S1P, in addition to six genes (CHN1, PRKCE, GNA12, OASL, OAS1, and IFIT3) appeared to drive the pathway results. Of these predictors, S1P, GNA12, and PRKCE were enriched in the results from integrative and metabolic ORAs. Through an integrative analysis of metabolomic, genomic, and methylation data from a small cohort of asthmatics, we implicate altered metabolic pathways, related to sphingolipid metabolism, in asthma control. These results provide insight into the pathophysiology of asthma control.

A Bayesian network dealing with measurements and residuals for system monitoring

The purpose of this paper is to present an original method for system monitoring with Bayesian networks. Our proposal is to associate a data-driven method to another model-based under a common tool. The two methods are first modeled under a Bayesian network (conditional Gaussian network), and then combined to evaluate the system state. In the proposed framework the residuals and measures coexist under a probabilistic framework. This approach is tested on a simulation of a water heater process under some various circumstances and shows better results than the two methods used alone.

Probabilistic Solution of a Duffing-Type Energy Harvester System Under Gaussian White Noise

This paper proposes a solution procedure to formulate an approximate joint probability density function (PDF) of a Duffing-type energy harvester system under Gaussian white noise. The joint PDF solution of displacement, velocity, and an electrical variable is governed by the Fokker-Planck (FP) equation. First, the FP equation is reduced to a lower-dimensional FP equation only about displacement and velocity by a state-space-split (SSS) method. The stationary joint PDF of displacement and velocity can be solved exactly. Then, the joint PDF of displacement, velocity, and the electrical variable can be approximated by the product of the obtained exact PDF and the conditional Gaussian PDF of the electrical variable. A parametric study is further conducted to show the effectiveness of the proposed solution procedure. The study considers weak nonlinearity, strong nonlinearity, high excitation level, and a bistable oscillator. Comparison with the simulated results shows that the proposed solution procedure is effective in obtaining the joint PDF of the energy harvester system in the examined examples.

Blended particle methods with adaptive subspaces for filtering turbulent dynamical systems

It is a major challenge throughout science and engineering to improve uncertain model predictions by utilizing noisy data sets from nature. Hybrid methods combining the advantages of traditional particle filters and the Kalman filter offer a promising direction for filtering or data assimilation in high dimensional turbulent dynamical systems. In this paper, blended particle filtering methods that exploit the physical structure of turbulent dynamical systems are developed. Non-Gaussian features of the dynamical system are captured adaptively in an evolving-in-time low dimensional subspace through particle methods, while at the same time statistics in the remaining portion of the phase space are amended by conditional Gaussian mixtures interacting with the particles. The importance of both using the adaptively evolving subspace and introducing conditional Gaussian statistics in the orthogonal part is illustrated here by simple examples. For practical implementation of the algorithms, finding the most probable distributions that characterize the statistics in the phase space as well as effective resampling strategies is discussed to handle realizability and stability issues. To test the performance of the blended algorithms, the forty dimensional Lorenz 96 system is utilized with a five dimensional subspace to run particles. The filters are tested extensively in various turbulent regimes with distinct statistics and with changing observation time frequency and both dense and sparse spatial observations. In real applications perfect dynamical models are always inaccessible considering the complexities in both modeling and computation of high dimensional turbulent system. The effects of model errors from imperfect modeling of the systems are also checked for these methods. The blended methods show uniformly high skill in both capturing non-Gaussian statistics and achieving accurate filtering results in various dynamical regimes with and without model errors.

Fault Detection and Diagnosis in a Bayesian Network classifier incorporating probabilistic boundary1

The purpose of this article is to present a method for Fault Detection and Diagnosis (FDD) with Bayesian network, and more particularly with Conditional Gaussian Network (CGN). A classical problem of FDD with supervised classification is when Normal Operating Conditions class is integrated, the false alarm rate is not guarantee by the classifier. The interest of the proposed method is to introduce a probabilistic limit on the Normal Operating Conditions class, allowing to respect a given false alarm rate. Performances of this method are evaluated on data of a benchmark example: the Tennessee Eastman Process. Three kinds of faults are taken into account on this complex process.

Clustering financial time series: New insights from an extended hidden Markov model

In recent years, large amounts of financial data have become available for analysis. We propose exploring returns from 21 European stock markets by model-based clustering of regime switching models. These econometric models identify clusters of time series with similar dynamic patterns and moreover allow relaxing assumptions of existing approaches, such as the assumption of conditional Gaussian returns. The proposed model handles simultaneously the heterogeneity across stock markets and over time, i.e., time-constant and time-varying discrete latent variables capture unobserved heterogeneity between and within stock markets, respectively. The results show a clear distinction between two groups of stock markets, each one characterized by different regime switching dynamics that correspond to different expected return-risk patterns. We identify three regimes: the so-called bull and bear regimes, as well as a stable regime with returns close to 0, which turns out to be the most frequently occurring regime. This is consistent with stylized facts in financial econometrics.

On the conditional distributions and the efficient simulations of exponential integrals of Gaussian random fields

In this paper, we consider the extreme behavior of a Gaussian random field $f(t)$ living on a compact set $T$. In particular, we are interested in tail events associated with the integral $\int_{T}e^{f(t)}\,dt$. We construct a (non-Gaussian) random field whose distribution can be explicitly stated. This field approximates the conditional Gaussian random field $f$ (given that $\int_{T}e^{f(t)}\,dt$ exceeds a large value) in total variation. Based on this approximation, we show that the tail event of $\int_{T}e^{f(t)}\,dt$ is asymptotically equivalent to the tail event of $\sup_{T}\gamma(t)$ where $\gamma(t)$ is a Gaussian process and it is an affine function of $f(t)$ and its derivative field. In addition to the asymptotic description of the conditional field, we construct an efficient Monte Carlo estimator that runs in polynomial time of $\log b$ to compute the probability $P(\int_{T}e^{f(t)}\,dt>b)$ with a prescribed relative accuracy.

Estimate for the doppler shift of a non-Gaussian signal upon coherent detection of scattered optical radiation in a turbulent atmosphere

A non-Gaussian model for estimating the radial velocity of turbulent flows in the atmosphere for coherent detection of scattered optical radiation is proposed. The model was obtained based on a theoretical approach that includes results of the statistical analysis of a pulse Doppler lidar signal in a turbulent medium, as well as on the perturbation-theory methods that have been developed in the theory of probability and mathematical statistics. It is shown that the estimate of the Doppler shift in the first-order perturbation theory is a sum of a regular component and two conditional fluctuation components—Gaussian and non-Gaussian ones. In the case of a homogeneous and isotropic turbulence, the estimate of the radial wind velocity is approximately equal to its true average value. The statistical uncertainty in measurements of the average radial wind velocity is determined by the behavior of conditional Gaussian and non-Gaussian components and significantly depends on the state of atmospheric turbulence. It is shown that basic equations of the non-Gaussian model in the limit case coincide with formulas of the local and nonlocal models, as well as with those of the Gaussian model.

A deformable atlas of the laboratory mouse.

This paper presents a deformable mouse atlas of the laboratory mouse anatomy. This atlas is fully articulated and can be positioned into arbitrary body poses. The atlas can also adapt body weight by changing body length and fat amount. A training set of 103 micro-CT images was used to construct the atlas. A cage-based deformation method was applied to realize the articulated pose change. The weight-related body deformation was learned from the training set using a linear regression method. A conditional Gaussian model and thin-plate spline mapping were used to deform the internal organs following the changes of pose and weight. The atlas was deformed into different body poses and weights, and the deformation results were more realistic compared to the results achieved with other mouse atlases. The organ weights of this atlas matched well with the measurements of real mouse organ weights. This atlas can also be converted into voxelized images with labeled organs, pseudo CT images and tetrahedral mesh for phantom studies. With the unique ability of articulated pose and weight changes, the deformable laboratory mouse atlas can become a valuable tool for preclinical image analysis.

CGBayesNets: conditional Gaussian Bayesian network learning and inference with mixed discrete and continuous data.

Bayesian Networks (BN) have been a popular predictive modeling formalism in bioinformatics, but their application in modern genomics has been slowed by an inability to cleanly handle domains with mixed discrete and continuous variables. Existing free BN software packages either discretize continuous variables, which can lead to information loss, or do not include inference routines, which makes prediction with the BN impossible. We present CGBayesNets, a BN package focused around prediction of a clinical phenotype from mixed discrete and continuous variables, which fills these gaps. CGBayesNets implements Bayesian likelihood and inference algorithms for the conditional Gaussian Bayesian network (CGBNs) formalism, one appropriate for predicting an outcome of interest from, e.g., multimodal genomic data. We provide four different network learning algorithms, each making a different tradeoff between computational cost and network likelihood. CGBayesNets provides a full suite of functions for model exploration and verification, including cross validation, bootstrapping, and AUC manipulation. We highlight several results obtained previously with CGBayesNets, including predictive models of wood properties from tree genomics, leukemia subtype classification from mixed genomic data, and robust prediction of intensive care unit mortality outcomes from metabolomic profiles. We also provide detailed example analysis on public metabolomic and gene expression datasets. CGBayesNets is implemented in MATLAB and available as MATLAB source code, under an Open Source license and anonymous download at http://www.cgbayesnets.com.