Bayesian Inference Scheme Research Articles

Source reconstruction: a statistical mechanics perspective

A Bayesian inference scheme is used to address the problem of general source reconstruction. To this purpose, starting from a simple, physically motivated structure-based representation for a general source distribution, Bayesian probability theory is used to formulate the full joint posterior probability density function for the parameters in this representation. The exploration of the high-dimensional parameter space using simulated annealing and a Markov chain Monte Carlo sampler is described. The explicit linkage between the computational tools used for posterior sampling and concepts in statistical mechanics is emphasised. The proposed methodology is validated using a real dispersion experiment involving a multiple source release.

Stochastic identification of composite material properties from limited experimental databases, Part II: Uncertainty modelling

The objective of this work is to characterise stochastic macroscopic material properties of heterogeneous composite fabrics from limited-size macro-scale experimental measurements. The work is presented in a sequence of two papers. In the first paper (Part I), a database consisting of observations of heterogeneous Young's modulus fields is obtained by a set of deterministic inverse problems. In this paper (Part II), a data assimilation framework is considered to identify a stochastic random field model of the Young's modulus. Such a model is set up to account for both aleatory uncertainties, related to sample inter-variabilities, as well as epistemic uncertainties due to insufficiency of the available data. This uncertainty characterisation is achieved by discretising the random field using a spectral decomposition procedure known as the Karhunen–Loève expansion. Random variables of this representation are expanded in a Hermite Polynomial Chaos (PC) basis whose coefficients themselves are considered as random variables. While the Gaussian variables of the PC basis model the aleatory uncertainty, the PC coefficients represent the epistemic uncertainty. A Bayesian inference scheme with Markov Chain Monte Carlo sampler is implemented to characterise the PC coefficients according to the Maximum A posteriori Probability (MAP) estimator.

Probability hazard map for future vent opening at the Campi Flegrei caldera, Italy

The Campi Flegrei caldera is a restless structure affected by general subsidence and ongoing resurgence of its central part. The persistent activity of the system and the explosive character of the volcanism lead to a very high volcanic hazard that, combined with intense urbanization, corresponds to a very high volcanic risk. One of the largest sources of uncertainty in volcanic hazard/risk assessment for Campi Flegrei is the spatial location of the future volcanic activity. This paper presents and discusses a long-term probability hazard map for vent opening in case of renewal of volcanism at the Campi Flegrei caldera, which shows the spatial conditional probability for the next vent opening, given that an eruption occurs. The map has been constructed by building a Bayesian inference scheme merging prior information and past data. The method allows both aleatory and epistemic uncertainties to be evaluated. The probability map of vent opening shows that two areas of relatively high probability are present within the active portion of the caldera, with a probability approximately doubled with respect to the rest of the caldera. The map has an immediate use in evaluating the areas of the caldera prone to the highest volcanic hazard. Furthermore, it represents an important ingredient in addressing the more general problem of quantitative volcanic hazards assessment at the Campi Flegrei caldera.

Bayesian Inference With Adaptive Fuzzy Priors and Likelihoods

Fuzzy rule-based systems can approximate prior and likelihood probabilities in Bayesian inference and thereby approximate posterior probabilities. This fuzzy approximation technique allows users to apply a much wider and more flexible range of prior and likelihood probability density functions than found in most Bayesian inference schemes. The technique does not restrict the user to the few known closed-form conjugacy relations between the prior and likelihood. It allows the user in many cases to describe the densities with words and just two rules can absorb any bounded closed-form probability density directly into the rulebase. Learning algorithms can tune the expert rules as well as grow them from sample data. The learning laws and fuzzy approximators have a tractable form because of the convex-sum structure of additive fuzzy systems. This convex-sum structure carries over to the fuzzy posterior approximator. We prove a uniform approximation theorem for Bayesian posteriors: An additive fuzzy posterior uniformly approximates the posterior probability density if the prior or likelihood densities are continuous and bounded and if separate additive fuzzy systems approximate the prior and likelihood densities. Simulations demonstrate this fuzzy approximation of priors and posteriors for the three most common conjugate priors (as when a beta prior combines with a binomial likelihood to give a beta posterior). Adaptive fuzzy systems can also approximate non-conjugate priors and likelihoods as well as approximate hyperpriors in hierarchical Bayesian inference. The number of fuzzy rules can grow exponentially in iterative Bayesian inference if the previous posterior approximator becomes the new prior approximator.

Global transcription regulation of RK2 plasmids: a case study in the combined use of dynamical mathematical models and statistical inference for integration of experimental data and hypothesis exploration

BackgroundIncP-1 plasmids are broad host range plasmids that have been found in clinical and environmental bacteria. They often carry genes for antibiotic resistance or catabolic pathways. The archetypal IncP-1 plasmid RK2 is a well-characterized biological system, with a fully sequenced and annotated genome and wide range of experimental measurements. Its central control operon, encoding two global regulators KorA and KorB, is a natural example of a negatively self-regulated operon. To increase our understanding of the regulation of this operon, we have constructed a dynamical mathematical model using Ordinary Differential Equations, and employed a Bayesian inference scheme, Markov Chain Monte Carlo (MCMC) using the Metropolis-Hastings algorithm, as a way of integrating experimental measurements and a priori knowledge. We also compared MCMC and Metabolic Control Analysis (MCA) as approaches for determining the sensitivity of model parameters.ResultsWe identified two distinct sets of parameter values, with different biological interpretations, that fit and explain the experimental data. This allowed us to highlight the proportion of repressor protein as dimers as a key experimental measurement defining the dynamics of the system. Analysis of joint posterior distributions led to the identification of correlations between parameters for protein synthesis and partial repression by KorA or KorB dimers, indicating the necessary use of joint posteriors for correct parameter estimation. Using MCA, we demonstrated that the system is highly sensitive to the growth rate but insensitive to repressor monomerization rates in their selected value regions; the latter outcome was also confirmed by MCMC. Finally, by examining a series of different model refinements for partial repression by KorA or KorB dimers alone, we showed that a model including partial repression by KorA and KorB was most compatible with existing experimental data.ConclusionsWe have demonstrated that the combination of dynamical mathematical models with Bayesian inference is valuable in integrating diverse experimental data and identifying key determinants and parameters for the IncP-1 central control operon. Moreover, we have shown that Bayesian inference and MCA are complementary methods for identification of sensitive parameters. We propose that this demonstrates generic value in applying this combination of approaches to systems biology dynamical modelling.

Improvements in the reconstruction of time-varying gene regulatory networks: dynamic programming and regularization by information sharing among genes

Dynamic Bayesian networks (DBNs) have been applied widely to reconstruct the structure of regulatory processes from time series data, and they have established themselves as a standard modelling tool in computational systems biology. The conventional approach is based on the assumption of a homogeneous Markov chain, and many recent research efforts have focused on relaxing this restriction. An approach that enjoys particular popularity is based on a combination of a DBN with a multiple changepoint process, and the application of a Bayesian inference scheme via reversible jump Markov chain Monte Carlo (RJMCMC). In the present article, we expand this approach in two ways. First, we show that a dynamic programming scheme allows the changepoints to be sampled from the correct conditional distribution, which results in improved convergence over RJMCMC. Second, we introduce a novel Bayesian clustering and information sharing scheme among nodes, which provides a mechanism for automatic model complexity tuning. We evaluate the dynamic programming scheme on expression time series for Arabidopsis thaliana genes involved in circadian regulation. In a simulation study we demonstrate that the regularization scheme improves the network reconstruction accuracy over that obtained with recently proposed inhomogeneous DBNs. For gene expression profiles from a synthetically designed Saccharomyces cerevisiae strain under switching carbon metabolism we show that the combination of both: dynamic programming and regularization yields an inference procedure that outperforms two alternative established network reconstruction methods from the biology literature. A MATLAB implementation of the algorithm and a supplementary paper with algorithmic details and further results for the Arabidopsis data can be downloaded from: http://www.statistik.tu-dortmund.de/bio2010.html.

Bayesian priors and multisensory integration at multiple levels of visual processing: Reply to comments on “Crossmodal influences on visual perception”

We would like to thank all commentators for their insightful and thought-provoking commentaries. We find itgratifying that the commentators represent a diverse array of expertise, as they have enriched the discussion with theirdifferent perspectives on this topic. The commentaries have also identified and highlighted important open questionsin this field; an important step in advancing the field. Here, we discuss some of the important theoretical issues andobservations raised by de Haas and Rees [1], Spence [2], Alais [3], Vroomen [4], and Barone [5].de Haas and Rees [1] underscore that crossmodal interactions appear to occur at multiple levels of processing, andargue that any theory of multisensory perception should account for this phenomenon. We agree with this assessment.As discussed in the target paper [6], crossmodal interactions have been reported for a variety of visual tasks rangingfrom low-level perceptual tasks such as detection and motion perception, to high-level tasks such as object recognition.Even for a simple low-level task, there appear to be interactions between modalities at multiple levels of processing.For example, as noted by de Haas and Rees [1], in the numerosity judgment task (discussed in Sections 3 and 4 ofthe target paper), fMRI revealed interactions in areas ranging from superior colliculus, V1, and STS [7,8], and MEGrevealed early-onset interactions in occipital regions followed by later interactions in parietal and frontal areas [9].Interactions at multiple levels of processing are indeed consistent with a Bayesian inference scheme in which bothlikelihoods (sensory representations) and priors (expectations) play a role in the perceptual process. While the combi-nation of sensory information (likelihood functions) can occur at an early neural level of processing (e.g., V1, superiorcolliculus, or thalamus), the priors may involve interactions at a range of neural processing levels. For example, in

Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors

Meaningful quantification of data and structural uncertainties in conceptual rainfall‐runoff modeling is a major scientific and engineering challenge. This paper focuses on the total predictive uncertainty and its decomposition into input and structural components under different inference scenarios. Several Bayesian inference schemes are investigated, differing in the treatment of rainfall and structural uncertainties, and in the precision of the priors describing rainfall uncertainty. Compared with traditional lumped additive error approaches, the quantification of the total predictive uncertainty in the runoff is improved when rainfall and/or structural errors are characterized explicitly. However, the decomposition of the total uncertainty into individual sources is more challenging. In particular, poor identifiability may arise when the inference scheme represents rainfall and structural errors using separate probabilistic models. The inference becomes ill‐posed unless sufficiently precise prior knowledge of data uncertainty is supplied; this ill‐posedness can often be detected from the behavior of the Monte Carlo sampling algorithm. Moreover, the priors on the data quality must also be sufficiently accurate if the inference is to be reliable and support meaningful uncertainty decomposition. Our findings highlight the inherent limitations of inferring inaccurate hydrologic models using rainfall‐runoff data with large unknown errors. Bayesian total error analysis can overcome these problems using independent prior information. The need for deriving independent descriptions of the uncertainties in the input and output data is clearly demonstrated.

Modelling the evolution of a bipartite network—Peer referral in interlocking directorates

A central part of relational ties between social actors is constituted by shared affiliations and events. The action of joint participation reinforces personal ties between social actors as well as mutually shared values and norms that in turn perpetuate the patterns of social action that define groups. Therefore the study of bipartite networks is central to social science. Furthermore, the dynamics of these processes suggests that bipartite networks should not be considered static structures but rather be studied over time. In order to model the evolution of bipartite networks empirically we introduce a class of models and a Bayesian inference scheme that extends previous stochastic actor-oriented models for unimodal graphs. Contemporary research on interlocking directorates provides an area of research in which it seems reasonable to apply the model. Specifically, we address the question of how tie formation, i.e. director recruitment, contributes to the structural properties of the interlocking directorate network. For boards of directors on the Stockholm stock exchange we propose that a prolific mechanism in tie formation is that of peer referral. The results indicate that such a mechanism is present, generating multiple interlocks between boards.

A bayesian generative model for surface template estimation.

3D surfaces are important geometric models for many objects of interest in image analysis and Computational Anatomy. In this paper, we describe a Bayesian inference scheme for estimating a template surface from a set of observed surface data. In order to achieve this, we use the geodesic shooting approach to construct a statistical model for the generation and the observations of random surfaces. We develop a mode approximation EM algorithm to infer the maximum a posteriori estimation of initial momentum μ, which determines the template surface. Experimental results of caudate, thalamus, and hippocampus data are presented.

Probabilistic Error Modeling for Nano-Domain Logic Circuits

In nano-domain logic circuits, errors generated are transient in nature and will arise due to the uncertainty or the unreliability of the computing element itself. This type of errors - which we refer to as dynamic errors - are to be distinguished from traditional faults and radiation related errors. Due to these highly likely dynamic errors, it is more appropriate to model nano-domain computing as probabilistic rather than deterministic. We propose a probabilistic error model based on Bayesian networks to estimate this expected output error probability, given dynamic error probabilities in each device since this estimate is crucial for nano-domain circuit designers to be able to compare and rank designs based on the expected output error. We estimate the overall output error probability by comparing the outputs of a dynamic error-encoded model with an ideal logic model. We prove that this probabilistic framework is a compact and minimal representation of the overall effect of dynamic errors in a circuit. We use both exact and approximate Bayesian inference schemes for propagation of probabilities. The exact inference shows better time performance than the state-of-the art by exploiting conditional independencies exhibited in the underlying probabilistic framework. However, exact inference is worst case NP-hard and can handle only small circuits. Hence, we use two approximate inference schemes for medium size benchmarks. We demonstrate the efficiency and accuracy of these approximate inference schemes by comparing estimated results with logic simulation results. We have performed our experiments on <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LGSynth'93</i> and <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ISCAS'85</i> benchmark circuits. We explore our probabilistic model to calculate: 1) error sensitivity of individual gates in a circuit; 2) compute overall exact error probabilities for small circuits; 3) compute approximate error probabilities for medium sized benchmarks using two stochastic sampling schemes; 4) compare and vet design with respect to dynamic errors; 5) characterize the input space for desired output characteristics by utilizing the unique backtracking capability of Bayesian networks (inverse problem); and 6) to apply selective redundancy to highly sensitive nodes for error tolerant designs.

3D structure inference by integrating segmentation and reconstruction from a single image

The authors present a hierarchical Bayesian method for inferring the 3D structure of polyhedral man-made objects from a single image by integrating 2D image parsing and 3D reconstruction. In the first stage, the image is parsed into its constituent components - arbitrary shape regions and polygonal shape regions. In the second stage, polygonal shape regions are grouped into man-made polyhedral objects. The 3D structures of these polyhedral objects are further inferred using geometric priors. These two stages are integrated into a Bayesian inference scheme and cooperate to compute the optimal solutions. This method enables the model to correct possible errors and explain ambiguities in the lower level with the help of information from the higher level. The algorithm is applied to the images of indoor scenes, and the experimental results demonstrate satisfactory performance.

Multi-source data fusion and super-resolution from astronomical images

Virtual observatories give us access to huge amounts of image data that are often redundant. Our goal is to take advantage of this redundancy by combining images of the same field of view into a single model. To achieve this goal, we propose to develop a multi-source data fusion method that relies on probability and band-limited signal theory. The target object is an image to be inferred from a number of blurred and noisy sources, possibly from different sensors under various conditions (i.e. resolution, shift, orientation, blur, noise...). We aim at the recovery of a compound model “image + uncertainties” that best relates to the observations and contains a maximum of useful information from the initial data set. Thus, in some cases, spatial super-resolution may be required in order to preserve the information. We propose to use a Bayesian inference scheme to invert a forward model, which describes the image formation process for each observation and takes into account some a priori knowledge (e.g. stars as point sources). This involves both automatic registration and spatial resampling, which are ill-posed inverse problems that are addressed within a rigorous Bayesian framework. The originality of the work is in devising a new technique of multi-image data fusion that provides us with super-resolution, self-calibration and possibly model selection capabilities. This approach should outperform existing methods such as resample-and-add or drizzling since it can handle different instrument characteristics for each input image and compute uncertainty estimates as well. Moreover, it is designed to also work in a recursive way, so that the model can be updated when new data become available.

Bayesian estimation of synaptic physiology from the spectral responses of neural masses

We describe a Bayesian inference scheme for quantifying the active physiology of neuronal ensembles using local field recordings of synaptic potentials. This entails the inversion of a generative neural mass model of steady-state spectral activity. The inversion uses Expectation Maximization (EM) to furnish the posterior probability of key synaptic parameters and the marginal likelihood of the model itself. The neural mass model embeds prior knowledge pertaining to both the anatomical [synaptic] circuitry and plausible trajectories of neuronal dynamics. This model comprises a population of excitatory pyramidal cells, under local interneuron inhibition and driving excitation from layer IV stellate cells. Under quasi-stationary assumptions, the model can predict the spectral profile of local field potentials (LFP). This means model parameters can be optimised given real electrophysiological observations. The validity of inferences about synaptic parameters is demonstrated using simulated data and experimental recordings from the medial prefrontal cortex of control and isolation-reared Wistar rats. Specifically, we examined the maximum a posteriori estimates of parameters describing synaptic function in the two groups and tested predictions derived from concomitant microdialysis measures. The modelling of the LFP recordings revealed (i) a sensitization of post-synaptic excitatory responses, particularly marked in pyramidal cells, in the medial prefrontal cortex of socially isolated rats and (ii) increased neuronal adaptation. These inferences were consistent with predictions derived from experimental microdialysis measures of extracellular glutamate levels.

On the construction and analysis of stochastic models: Characterization and propagation of the errors associated with limited data

This paper investigates the predictive accuracy of stochastic models. In particular, a formulation is presented for the impact of data limitations associated with the calibration of parameters for these models, on their overall predictive accuracy. In the course of this development, a new method for the characterization of stochastic processes from corresponding experimental observations is obtained. Specifically, polynomial chaos representations of these processes are estimated that are consistent, in some useful sense, with the data. The estimated polynomial chaos coefficients are themselves characterized as random variables with known probability density function, thus permitting the analysis of the dependence of their values on further experimental evidence. Moreover, the error in these coefficients, associated with limited data, is propagated through a physical system characterized by a stochastic partial differential equation (SPDE). This formalism permits the rational allocation of resources in view of studying the possibility of validating a particular predictive model. A Bayesian inference scheme is relied upon as the logic for parameter estimation, with its computational engine provided by a Metropolis-Hastings Markov chain Monte Carlo procedure.

Inferring CO2 sources and sinks from satellite observations: Method and application to TOVS data

Properly handling satellite data to constrain the inversion of CO2 sources and sinks at the Earth surface is a challenge motivated by the limitations of the current surface observation network. In this paper we present a Bayesian inference scheme to tackle this issue. It is based on the same theoretical principles as most inversions of the flask network but uses a variational formulation rather than a pure matrix‐based one in order to cope with the large amount of satellite data. The minimization algorithm iteratively computes the optimum solution to the inference problem as well as an estimation of its error characteristics and some quantitative measures of the observation information content. A global climate model, guided by analyzed winds, provides information about the atmospheric transport to the inversion scheme. A surface flux climatology regularizes the inference problem. This new system has been applied to 1 year's worth of retrievals of vertically integrated CO2 concentrations from the Television Infrared Observation Satellite Operational Vertical Sounder (TOVS). Consistent with a recent study that identified regional biases in the TOVS retrievals, the inferred fluxes are not useful for biogeochemical analyses. In addition to the detrimental impact of these biases, we find a sensitivity of the results to the formulation of the prior uncertainty and to the accuracy of the transport model. Notwithstanding these difficulties, four‐dimensional inversion schemes of the type presented here could form the basis of multisensor data assimilation systems for the estimation of the surface fluxes of key atmospheric compounds.

TEST STATISTICS FOR SYSTEM DESIGN FAILURE

A structural model for the estimation of software failure rates is proposed which is based on a partition of the code into components. Systematic failure is assumed to be induced by the interaction of these components. A Bayesian inference scheme is used to perform failure rate estimation on the basis of N failure free test runs. The approach splits the problem of constructing a system prior up into the smaller, conceivably simpler problems of constructing subtask priors. Thereby a wider range of prior information is used in the process of assessing system safety. The work in this paper constitutes a first step towards a formal statistical understanding of the relationship between system complexity and system testing for reliability. Long-term implications are the achievement of more informative reliability estimates and guidelines on cost-effective testing.

AN ITERATIVE INFERENCE MECHANISM FOR THE PROBABILISTIC EXPERT SYSTEM PES

The paper suggests a new inference mechanism based on iterative use of the Bayesian inference scheme. The procedure iteratively computes optimal component weights of a distribution mixture from a class called generalized knowledge base. It is proved that the iterative process converges to a unique limit whereby the resulting probability distribution can be defined as the information-divergence projection of input distribution on the generalized knowledge base. The iterative inference mechanism resembles natural process of cognition as iteratively improving understanding of the input information.

Reconstruction of Quantum States of Spin Systems: From Quantum Bayesian Inference to Quantum Tomography

We study in detail the reconstruction of spin-1/2 states and analyze the connection between (1) quantum Bayesian inference, (2) reconstruction via the Jaynes principle of maximum entropy, and (3) complete reconstruction schemes such asdiscrete quantum tomography. We derive an expression for a density operator estimated via Bayesian quantum inference in the limit of an infinite number of measurements. This expression is derived under the assumption that the reconstructed system is in a pure state. In this case the estimation corresponds to averaging over a microcanonical ensemble of pure states satisfying a set of constraints imposed by the measured mean values of the observables under consideration. We show that via a “purification” ansatz, statistical mixtures can also be consistently reconstructed via the quantum Bayesian inference scheme. In this case the estimation corresponds to averaging over the generalized grand canonical ensemble of states satisfying the given constraints, and in the limit of large number of measurements this density operator is equal to the generalized canonical density operator, which can be obtained with the help of the Jaynes principle of the maximum entropy. We also discuss inseparability of reconstructed density operators of two spins-1/2.

The estimation of failure rates for low probability events

In many cases the individual components in an engineering system may have suffered no or perhaps only a single failure during their operating history. Such a history may be that of continuous operation over a period of time or it may be over a number of discrete demands. In spite of the apparent lack, or paucity of failure rate data, it is still important to be able to estimate a failure rate or failure probability. In order to do this, it is necessary for the engineer to decide whether there is available some source of information which, while not precisely related to the problem of interest, has some material bearing on it. If this is the case, it may be possible to construct a probability distribution for the failure rate defined by a most probable value and some measure of the variance. If no such information is available then it will be necessary to use a so-called non-informative prior within a Bayesian inference scheme. We describe here the essential results of such a scheme. In some cases, experience with similar systems or elicitation by expert judgement may lead to the construction of a prior which gives some guidance as to the most likely value of a failure rate, even though no failures have actually been observed in the system of interest. The choice of prior is not a ‘mechanical’ process and requires some experience as to the type of data of interest and the amount and form of the information available. Ideally, a complete probability distribution would be desirable, obtained by eliciting information from a large number of experts. This situation is rarely practical for logistical and financial reasons. More commonly, one has a most probable value and a standard deviation, or possibly upper and lower confidence limits. Thus, two pieces of information are available and this restricts the algebraic form of the prior to one which contains two arbitrary parameters. The following functions are therefore suitable candidates and have been used in the past to represent the variability of failure rates in various situations. 1.Gamma distribution, 2.Log-normal distribution, 3.Hat function, 4.Log-uniform distribution. The log-normal distribution is useful to represent data which varies in inverse powers of 10 and is symmetrical about the most probable value on a logarithmic scale. The Gamma distribution, on the other hand, will describe data that is skewed about the most probable value on a logarithmic scale with particular emphasis on the suppression of large values of the failure rate. For cases in which little is known about the data except mean values, standard deviations and confidence limits, then the hat function or log-uniform are appropriate. The log-uniform distribution is particularly useful since it represents probabilities that are of equal likelihood on a logarithmic scale between an upper and lower bound. A new form of prior, based on the Cauchy distribution, is found to be useful to represent situations where information about the system is particularly sparse. This paper explores these issues by first reviewing some of the seminal work in the area and then applying Bayesian methods to obtain quantitative results of direct use for practicing engineers. In particular, the paper gives results for the zero failure case for continuous and discrete systems. An important conclusion of the work is that the most appropriate failure rate to use in prospective studies, for components that have been operating continuously for a time T with no failures, is 0.55/T.