Bayesian Inference Scheme Research Articles

A likelihood-free approach towards Bayesian modeling of degradation growths using mixed-effects regression

Mixed-effects regression models are widely applicable for predicting degradation growths in structural components. The Bayesian inference method is used to estimate the regression parameters when the degradation data are confounded by measurement and parameter uncertainties. The Gibbs sampler (GS), commonly used for this purpose, works when the regression errors are assumed as normally distributed that allows for the analytical formulation of the likelihood function. In case of a more general regression error distribution (e.g., mixture models), the likelihood becomes analytically intractable and computationally expensive to a degree that any likelihood-based Bayesian inference scheme (e.g., GS, Metropolis-Hastings sampler) can no longer be used for solving a practical problem.This paper proposes a practical likelihood-free approach for parameter estimation based on the approximate Bayesian computation (ABC) method. The ABC method implements forward simulation coupled with a rejection mechanism to sample from a target posterior distribution thereby eliminating the need to evaluate the likelihood function. The advantages of the proposed method are illustrated by analyzing degradation data obtained from a Canadian nuclear power plant.

Max-and-Smooth: A Two-Step Approach for Approximate Bayesian Inference in Latent Gaussian Models

With modern high-dimensional data, complex statistical models are necessary, requiring computationally feasible inference schemes. We introduce Max-and-Smooth, an approximate Bayesian inference scheme for a flexible class of latent Gaussian models (LGMs) where one or more of the likelihood parameters are modeled by latent additive Gaussian processes. Our proposed inference scheme is a two-step approach. In the first step (Max), the likelihood function is approximated by a Gaussian density with mean and covariance equal to either (a) the maximum likelihood estimate and the inverse observed information, respectively, or (b) the mean and covariance of the normalized likelihood function. In the second step (Smooth), the latent parameters and hyperparameters are inferred and smoothed with the approximated likelihood function. The proposed method ensures that the uncertainty from the first step is correctly propagated to the second step. Because the prior density for the latent parameters is assumed to be Gaussian and the approximated likelihood function is Gaussian, the approximate posterior density of the latent parameters (conditional on the hyperparameters) is also Gaussian, thus facilitating efficient posterior inference in high dimensions. Furthermore, the approximate marginal posterior distribution of the hyperparameters is tractable, and as a result, the hyperparameters can be sampled independently of the latent parameters. We show that the computational cost of Max-and-Smooth is close to being insensitive to the number of independent data replicates, and that it scales well with increased dimension of the latent parameter vector provided that its Gaussian prior density is specified with a sparse precision matrix. In the case of a large number of independent data replicates, sparse precision matrices, and high-dimensional latent vectors, the speedup is substantial in comparison to an MCMC scheme that infers the posterior density from the exact likelihood function. The accuracy of the Gaussian approximation to the likelihood function increases with the number of data replicates per latent model parameter. The proposed inference scheme is demonstrated on one spatially referenced real dataset and on simulated data mimicking spatial, temporal, and spatio-temporal inference problems. Our results show that Max-and-Smooth is accurate and fast.

A fast Bayesian inference scheme for identification of local structural properties of layered composites based on wave and finite element-assisted metamodeling strategy and ultrasound measurements

Reliable verification and evaluation of the mechanical properties of a layered composite ensemble are critical for industrially relevant applications, however it still remains an open engineering challenge. In this study, a fast Bayesian inference scheme based on multi-frequency single shot measurements of wave propagation characteristics is developed to overcome the limitations of ill-conditioning and non-uniqueness associated with the conventional approaches. A Transitional Markov chain Monte Carlo (TMCMC) algorithm is employed for the sampling process. A Wave and Finite Element (WFE)-assisted metamodeling scheme in lieu of expensive-to-evaluate explicit FE analysis is proposed to cope with the high computational cost involved in TMCMC sampling. For this, the Kriging predictor providing a surrogate mapping between the probability spaces of the model predictions for the wave characteristics and the mechanical properties in the likelihood evaluations is established based on the training outputs computed using a WFE forward solver, coupling periodic structure theory to conventional FE. The valuable uncertainty information of the prediction variance introduced by the use of a surrogate model is also properly taken into account when estimating the parameters’ posterior probability distribution by TMCMC. A numerical study as well as an experimental study are conducted to verify the computational efficiency and accuracy of the proposed methodology. Results show that the TMCMC algorithm in conjunction with the WFE forward solver-aided metamodeling can sample the posterior Probability Density Function (PDF) of the updated parameters at a very reasonable cost. This approach is capable of quantifying the uncertainties of recovered independent characteristics for each layer of the composite structure under investigation through fast and inexpensive experimental measurements on localized portions of the structure.

Hierarchical Bayesian myocardial perfusion quantification.

Myocardial blood flow can be quantified from dynamic contrast-enhanced magnetic resonance (MR) images through the fitting of tracer-kinetic models to the observed imaging data. The use of multi-compartment exchange models is desirable as they are physiologically motivated and resolve directly for both blood flow and microvascular function. However, the parameter estimates obtained with such models can be unreliable. This is due to the complexity of the models relative to the observed data which is limited by the low signal-to-noise ratio, the temporal resolution, the length of the acquisitions and other complex imaging artefacts. In this work, a Bayesian inference scheme is proposed which allows the reliable estimation of the parameters of the two-compartment exchange model from myocardial perfusion MR data. The Bayesian scheme allows the incorporation of prior knowledge on the physiological ranges of the model parameters and facilitates the use of the additional information that neighbouring voxels are likely to have similar kinetic parameter values. Hierarchical priors are used to avoid making a priori assumptions on the health of the patients. We provide both a theoretical introduction to Bayesian inference for tracer-kinetic modelling and specific implementation details for this application. This approach is validated in both in silico and in vivo settings. In silico, there was a significant reduction in mean-squared error with the ground-truth parameters using Bayesian inference as compared to using the standard non-linear least squares fitting. When applied to patient data the Bayesian inference scheme returns parameter values that are in-line with those previously reported in the literature, as well as giving parameter maps that match the independant clinical diagnosis of those patients.

Novel statistical analysis method for determining shear strength of C/C composite pin

The discreteness of strength seriously restricts the wider use of Carbon/Carbon (C/C) composites. The strength of C/C composites is studied by using statistical analysis methods. Here, a novel statistical analysis method named B-distribution is proposed to examine the shear strength of C/C composite pins. The method is implemented using a Bayesian inference scheme with a Markov Chain Monte Carlo sampler. Then, the shear-strength distributions of the C/C composites in different directions are calculated by the B-distribution and four traditional statistical models (three-parameter Weibull, two-parameter Weibull, log-normal, and gamma). These are critically analysed by a Kolmogorov-Smirnov (K–S) test and a maximum likelihood test. Next, the effect of the loading direction on the shear strength of C/C composite pin is statistically analysed. The results show that the B-distribution is the most appropriate statistical distribution for the experimental data. Additionally, it is found that the high reliability shear strengths of C/C composite pin in the 30° and 45° directions are relatively higher, and the high reliability shear strengths of C/C composite pin are relatively lower in the 75° and 90° directions.

Sequential Bayesian Detection of Spike Activities From Fluorescence Observations

Extracting and detecting spike activities from the fluorescence observations is an important step in understanding how neuron systems work. The main challenge lies in that the combination of the ambient noise with dynamic baseline fluctuation, often contaminates the observations, thereby deteriorating the reliability of spike detection. This may be even worse in the face of the nonlinear biological process, the coupling interactions between spikes and baseline, and the unknown critical parameters of an underlying physiological model, in which erroneous estimations of parameters will affect the detection of spikes causing further error propagation. In this paper, we propose a random finite set (RFS) based Bayesian approach. The dynamic behaviors of spike sequence, fluctuated baseline and unknown parameters are formulated as one RFS. This RFS state is capable of distinguishing the hidden active/silent states induced by spike and non-spike activities respectively, thereby \emph{negating the interaction role} played by spikes and other factors. Then, premised on the RFS states, a Bayesian inference scheme is designed to simultaneously estimate the model parameters, baseline, and crucial spike activities. Our results demonstrate that the proposed scheme can gain an extra $12\%$ detection accuracy in comparison with the state-of-the-art MLSpike method.

A Sequential Bayesian Partitioning Approach for Online Steady-State Detection of Multivariate Systems

The steady-state detection is critically important in many engineering fields, such as fault detection and diagnosis and process monitoring and control. However, most of the existing methods were designed for univariate signals and, thus, are not effective in handling multivariate signals. In this paper, we propose an efficient online steady-state detection method for multivariate systems through a sequential Bayesian partitioning approach. The signal is modeled by a Bayesian piecewise constant mean and covariance model, and a recursive updating method is developed to calculate the posterior distributions analytically. The duration of the current segment is utilized for steady-state testing. Insightful guidance is also provided for hyperparameter selection. The effectiveness of the proposed method is demonstrated through thorough numerical and real case studies. Note to Practitioners —This paper addresses the problem of online steady-state detection of systems captured by multivariate signals. Existing approaches often monitor each signal independently, and the system is claimed steady when all signals reach steady state. These methods have many shortcomings, such as failing to consider the correlations among signals and suffering the multiple testing problems. In this paper, we propose a novel joint monitoring approach, where the multivariate signal is sequentially partitioned into segments of constant mean and covariance through an online Bayesian inference scheme, and once the current segment duration is sufficiently large, the signal is considered steady. We also provide several insightful guidelines to select appropriate hyperparameters under different scenarios. The proposed approach is much more accurate and robust than existing ones. However, this method may face prohibitive computational cost and ill-posed covariance inversion problem when there are hundreds or even thousands of variables in the system. In future research, we will develop efficient distributed monitoring and data fusion techniques to overcome these challenges.

Constraints on the neutron star equation of state from GW170817

The first detection of gravitational waves from a neutron star-neutron star merger, GW170817, has opened up a new avenue for constraining the ultradense-matter equation of state (EOS). The deviation of the observed waveform from a point-particle waveform is a sensitive probe of the EOS controlling the merging neutron stars' structure. In this topical review, I discuss the various constraints that have been made on the EOS in the year following the discovery of GW170817. In particular, I review the surprising relationship that has emerged between the effective tidal deformability of the binary system and the neutron star radius. I also report new results that make use of this relationship, finding that the radius inferred from GW170817 lies between 9.8 and 13.2 km at 90% confidence, with distinct likelihood peaks at 10.8 and 12.3 km. I compare these radii, as well as those inferred in the literature, to X-ray measurements of the neutron star radius. I also summarize the various maximum mass constraints, which point towards a maximum mass < 2.3 M_sun, depending on the fate of the remnant, and which can be used to additionally constrain the high-density EOS. I review the constraints on the EOS that have been performed directly, through Bayesian inference schemes. Finally, I comment on the importance of disentangling thermal effects in future EOS constraints from neutron star mergers.

Joint boundary-aware method and Bayesian Inference for object proposal

Abstract As a fundamental pre-processing procedure, object proposal focuses on selecting a small group of object candidates possibly containing most of the category-independent objects to simplify the search extent. Generally, traditional handcrafted features and CNN-based features are applied to generate object candidates. However, it is challengeable for the existing methods to achieve balance between the proposal accuracy and the computational efficiency. In this paper, we propose an effective low-level feature extractor for object proposal by combing the boundary-preserving property of super-pixels together with Bayesian Inference framework. We first compute prior probability of edge segment which is derived from clustering operation of the edge map. Then, the posterior probability of edge segment is calculated by the boundary-preserving property of super-pixel and Bayesian Inference scheme. Finally, based on the enhanced edge segment map, we apply the same mechanism used in Edge Boxes to obtain the candidate bounding boxes. Extensive experiments on PASCAL VOC2007 dataset demonstrate that the proposed method achieves good balance between proposal quality and computational efficiency.

Effects of stream nitrate data frequency on watershed model performance and prediction uncertainty

High-frequency water quality monitoring is increasingly used in examining the nutrient fluxes within catchments. Despite this, no studies have assessed the impact of monitoring frequency on the uncertainty of nitrate estimates obtained from distributed or semi-distributed catchment models. This study aims to evaluate the impacts of two different frequencies of nitrate sampling on the performance of a catchment hydrology model, including the uncertainty in both predictions and calibrated parameters. The investigation uses the HYPE model to simulate streamflow and nitrate concentrations (2010–2015) in the Selke catchment, a heterogeneous mesoscale catchment in central Germany. The Bayesian inference scheme of the DREAM code was employed for calibration and uncertainty analysis, and to explore differences between fortnightly and daily nitrate sampling strategies. The results indicate that: (a) the posterior uncertainty intervals of nitrogen-export process parameters were narrower when the model was calibrated to daily nitrate measurements, while similar maximum likelihood parameter values were obtained regardless of the sampling frequency; (b) the model calibrated using daily nitrate data better represented both daily and fortnightly nitrate measurements relative to that obtained using fortnightly sampling; (c) the daily nitrate dataset produced significantly smaller parametric prediction uncertainty, but only modest reduction in total prediction uncertainty, relative to the fortnightly nitrate dataset; (d) model structural error and measurement errors are the primary sources of total prediction uncertainty, and these combine to inhibit the benefits of high-frequency monitoring. We conclude that the adequacy of sampling frequency is dependent on model structure and measurement errors, such that higher-frequency nitrate monitoring may not markedly reduce the uncertainty of nutrient predictions, depending on other levels and sources of uncertainty.

Using environmental tracers to determine the relative importance of travel times in the unsaturated and saturated zones for the delay of nitrate reduction measures

Groundwater quality in many regions with intense agriculture has deteriorated due to the leaching of nitrate and other agricultural pollutants. Modified agricultural practices can reduce the input of nitrate to groundwater bodies, but it is crucial to determine the time span over which these measures become effective at reducing nitrate levels in pumping wells. Such estimates can be obtained from hydrogeological modeling or lumped-parameter models (LPM) in combination with environmental tracer data. Two challenges in such tracer-based estimates are (i) accounting for the different modes of transport in the unsaturated zone (USZ), and (ii) assessing uncertainties. Here we extend a recently published Bayesian inference scheme for simple LPMs to include an explicit USZ model and apply it to the Dünnerngäu aquifer, Switzerland. Compared to a previous estimate of travel times in the aquifer based on a 2D hydrogeological model, our approach provides a more accurate assessment of the dynamics of nitrate concentrations in the aquifer. We find that including tracer measurements (3H/3He, 85Kr, 39Ar, 4He) reduces uncertainty in nitrate predictions if nitrate time series at wells are not available or short, but does not necessarily lead to better predictions if long nitrate time series are available. Additionally, the combination of tracer data with nitrate time series allows for a separation of the travel times in the unsaturated and saturated zone.

Vector Autoregressive Hierarchical Hidden Markov Models for Extracting Finger Movements Using Multichannel Surface EMG Signals

We present a novel computational technique intended for the robust and adaptable control of a multifunctional prosthetic hand using multichannel surface electromyography. The initial processing of the input data was oriented towards extracting relevant time domain features of the EMG signal. Following the feature calculation, a piecewise modeling of the multidimensional EMG feature dynamics using vector autoregressive models was performed. The next step included the implementation of hierarchical hidden semi‐Markov models to capture transitions between piecewise segments of movements and between different movements. Lastly, inversion of the model using an approximate Bayesian inference scheme served as the classifier. The effectiveness of the novel algorithms was assessed versus methods commonly used for real‐time classification of EMGs in a prosthesis control application. The obtained results show that using hidden semi‐Markov models as the top layer, instead of the hidden Markov models, ranks top in all the relevant metrics among the tested combinations. The choice of the presented methodology for the control of prosthetic hand is also supported by the equal or lower computational complexity required, compared to other algorithms, which enables the implementation on low‐power microcontrollers, and the ability to adapt to user preferences of executing individual movements during activities of daily living.

Time-dependent propagators for stochastic models of gene expression: an analytical method

The inherent stochasticity of gene expression in the context of regulatory networks profoundly influences the dynamics of the involved species. Mathematically speaking, the propagators which describe the evolution of such networks in time are typically defined as solutions of the corresponding chemical master equation (CME). However, it is not possible in general to obtain exact solutions to the CME in closed form, which is due largely to its high dimensionality. In the present article, we propose an analytical method for the efficient approximation of these propagators. We illustrate our method on the basis of two categories of stochastic models for gene expression that have been discussed in the literature. The requisite procedure consists of three steps: a probability-generating function is introduced which transforms the CME into (a system of) partial differential equations (PDEs); application of the method of characteristics then yields (a system of) ordinary differential equations (ODEs) which can be solved using dynamical systems techniques, giving closed-form expressions for the generating function; finally, propagator probabilities can be reconstructed numerically from these expressions via the Cauchy integral formula. The resulting ‘library’ of propagators lends itself naturally to implementation in a Bayesian parameter inference scheme, and can be generalised systematically to related categories of stochastic models beyond the ones considered here.

A model of adaptive decision-making from representation of information environment by quantum fields.

We present the mathematical model of decision-making (DM) of agents acting in a complex and uncertain environment (combining huge variety of economical, financial, behavioural and geopolitical factors). To describe interaction of agents with it, we apply the formalism of quantum field theory (QTF). Quantum fields are a purely informational nature. The QFT model can be treated as a far relative of the expected utility theory, where the role of utility is played by adaptivity to an environment (bath). However, this sort of utility-adaptivity cannot be represented simply as a numerical function. The operator representation in Hilbert space is used and adaptivity is described as in quantum dynamics. We are especially interested in stabilization of solutions for sufficiently large time. The outputs of this stabilization process, probabilities for possible choices, are treated in the framework of classical DM. To connect classical and quantum DM, we appeal to Quantum Bayesianism. We demonstrate the quantum-like interference effect in DM, which is exhibited as a violation of the formula of total probability, and hence the classical Bayesian inference scheme.This article is part of the themed issue 'Second quantum revolution: foundational questions'.

Reconstructing promoter activity from Lux bioluminescent reporters.

The bacterial Lux system is used as a gene expression reporter. It is fast, sensitive and non-destructive, enabling high frequency measurements. Originally developed for bacterial cells, it has also been adapted for eukaryotic cells, and can be used for whole cell biosensors, or in real time with live animals without the need for euthanasia. However, correct interpretation of bioluminescent data is limited: the bioluminescence is different from gene expression because of nonlinear molecular and enzyme dynamics of the Lux system. We have developed a computational approach that, for the first time, allows users of Lux assays to infer gene transcription levels from the light output. This approach is based upon a new mathematical model for Lux activity, that includes the actions of LuxAB, LuxEC and Fre, with improved mechanisms for all reactions, as well as synthesis and turn-over of Lux proteins. The model is calibrated with new experimental data for the LuxAB and Fre reactions from Photorhabdus luminescens—the source of modern Lux reporters—while literature data has been used for LuxEC. Importantly, the data show clear evidence for previously unreported product inhibition for the LuxAB reaction. Model simulations show that predicted bioluminescent profiles can be very different from changes in gene expression, with transient peaks of light output, very similar to light output seen in some experimental data sets. By incorporating the calibrated model into a Bayesian inference scheme, we can reverse engineer promoter activity from the bioluminescence. We show examples where a decrease in bioluminescence would be better interpreted as a switching off of the promoter, or where an increase in bioluminescence would be better interpreted as a longer period of gene expression. This approach could benefit all users of Lux technology.

Structured Bayesian compressive sensing with spatial location dependence via variational Bayesian inference

In this paper, a novel non-parametric Bayesian compressive sensing algorithm is proposed to enhance reconstruction performance of sparse entries with a continuous structure by exploiting the location dependence of entries. An approach is proposed which involves the logistic model and location-dependent Gaussian kernel. The variational Bayesian inference scheme is used to perform the posterior distributions and acquire an approximately analytical solution. Compared to the conventional clustered based methods, which only exploit the information of neighboring pixels, the proposed approach takes the relationship between the pixels of the entire image into account to enable the utilization of the underlying sparse signal structure. It significantly reduces the required number of observations for sparse reconstruction. Both real-valued signal applications, including one-dimension signal and two-dimension image, and complex-valued signal applications, including single-snapshot direction-of-arrival (DOA) estimation of distributed sources and inverse synthetic aperture radar (ISAR) imaging with a limited number of pluses, demonstrate the superiority of the proposed algorithm.

Social Information Is Integrated into Value and Confidence Judgments According to Its Reliability

How much we like something, whether it be a bottle of wine or a new film, is affected by the opinions of others. However, the social information that we receive can be contradictory and vary in its reliability. Here, we tested whether the brain incorporates these statistics when judging value and confidence. Participants provided value judgments about consumer goods in the presence of online reviews. We found that participants updated their initial value and confidence judgments in a Bayesian fashion, taking into account both the uncertainty of their initial beliefs and the reliability of the social information. Activity in dorsomedial prefrontal cortex tracked the degree of belief update. Analogous to how lower-level perceptual information is integrated, we found that the human brain integrates social information according to its reliability when judging value and confidence.SIGNIFICANCE STATEMENT The field of perceptual decision making has shown that the sensory system integrates different sources of information according to their respective reliability, as predicted by a Bayesian inference scheme. In this work, we hypothesized that a similar coding scheme is implemented by the human brain to process social signals and guide complex, value-based decisions. We provide experimental evidence that the human prefrontal cortex's activity is consistent with a Bayesian computation that integrates social information that differs in reliability and that this integration affects the neural representation of value and confidence.

Uncertainty analysis of force coefficients during micromilling of titanium alloy

Predicting process forces in micromilling is difficult due to complex interaction between the cutting edge and the work material, size effect, and process dynamics. This study describes the application of Bayesian inference to identify force coefficients in the micromilling process. The Metropolis-Hastings (MH) algorithm Markov chain Monte Carlo (MCMC) approach has been used to identify probability distributions of cutting, edge, and ploughing force coefficients based on experimental measurements and a mechanistic model of micromilling. The Bayesian inference scheme allows for predicting the upper and lower limits of micromilling forces, providing useful information about stability boundary calculations and robust process optimization. In the first part of the paper, micromilling experiments are performed to investigate the influence of micromilling process parameters on machining forces, tool edge condition, and surface texture. Under the experimental conditions used in this study, built-up edge formation is observed to have a significant influence on the process outputs in micromilling of titanium alloy Ti6Al4V. In the second part, Bayesian inference was explained in detail and applied to model micromilling force prediction. The force predictions are validated with the experimental measurements. The paper concludes with a discussion of the effectiveness of employing Bayesian inference in micromilling force modeling considering special machining cases.

A virtual sensing based augmented particle filter for tool condition prognosis

Timely evaluation and prediction of tool condition is critical to establish optimized maintenance plans in order to enhance production, minimize costly downtime. This paper presents an augmented particle filter based on virtual sensing technique with support vector regression (SVR) model to account for uncertainties in the tool condition degradation process. Tool condition is predicted by recursively updating a physics-based tool condition degradation model with virtual measurement approximately estimating tool degradation condition through virtual sensing technique, following a Bayesian inference scheme. Additionally, in order to improve estimation accuracy of virtual sensing model, different state-of-the-art dimension reduction techniques including principal component analysis (PCA) and its kernel version (KPCA), locality preserving projection (LPP) method have been investigated for feature fusion in a virtual sensing model, and the KPCA method performs best in terms of sensing accuracy. Afterwards, virtual measurement is then incorporated into particle filter. The effectiveness of the developed method is experimentally validated in a set of machining tool run-to-failure tests on a computer numerical control (CNC) milling machine.

Improving modeled air pollution concentration maps by residual interpolation

Models that are used to map air pollutant concentrations are not free of errors. A possible approach for improving the final concentration map is to interpolate the residuals of the initial model concentration estimates. Due to the possible spatial autocorrelation of the residuals of the initial model estimates, Bayesian inference schemes were suggested for this task, since they can correctly adjust the level of fitting of the residuals to the random measurement errors. However, the complexity of Bayesian methods often discourages their use. Here, we present an alternative and simpler approach, using a leave-one-out cross-validation to determine the optimal level of fitting of the residual correction. We show that the optimal correction level is related to the extent of the spatial autocorrelation of the cross-validated residuals. Namely, when the residuals are not autocorrelated residual correction is unnecessary, and if employed may actually degrade the quality of the final concentration map. Moreover, our approach enables to optimize the residual correction based on different target performance measures, with a possibly different optimal correction depending on the performance measure used. Hence, different target performance measures can be chosen to fit best the specific application of interest. The method is demonstrated using output of three different models used for estimating NOx and NO2 concentrations over Israel. We show that our approach can be used as an exploratory step, for assessing the potential benefit of residual correction, and as a simple alternative to Bayesian schemes.