Metropolis-Hastings Sampling Algorithm Research Articles

Analysis of the effect of surface coating and texturization on the estimation of effective thermophysical properties

This article aims to investigate the influence of surface modifications on the effective thermophysical properties of textured carbon steel and coated hard metal. The novelty of this work lies in the simultaneous estimation of thermal diffusivity and thermal conductivity via measurements using one active heating surface and Bayesian inference to solve the inverse heat conduction problem. To achieve this, an experimental setup was developed where samples were partially heated on an active surface, and the temperature was measured at two different points on the surface to estimate the thermal properties. The method was applied to two different thermal models using the same set of experimental data. Thus, the thermophysical properties were determined with and without the Markov Chain Monte Carlo technique using the Metropolis-Hastings sampling algorithm. Both techniques proved suitable for estimating the thermophysical properties. It is noteworthy that texturization and coating did not significantly modify the effective thermal diffusivity of the analyzed samples. However, there were considerable changes in the effective thermal conductivity, with an increase of approximately 13 % in the textured carbon steel sample and a reduction of approximately 11 % in the coated hard metal sample. The expanded uncertainty of the estimated effective thermophysical properties was less than 14 %. These results reflect low dispersion and good accuracy of the experimental technique used.

Bayesian calibration for Lamb wave propagation on a composite plate using a machine learning surrogate model

This paper presents a new framework for stochastic updating of a finite element model for a composite plate, considering the influence of temperature on Lamb wave propagation. The framework involves deterministic updating to optimize mechanical properties and stochastic updating to derive probability density functions for key parameters. It utilizes sensitivity analysis and Bayesian inference with Markov-Chain Monte Carlo simulations and the Metropolis–Hastings sampling algorithm. This paper proposes a machine learning surrogate model based on artificial neural networks to improve computational efficiency. This surrogate modeling approach allows parallelized Monte Carlo simulations, reducing updating time significantly without compromising the accuracy of the resulting probability density functions for model parameters. These advancements show a promising way to enhance composite plate modeling and Lamb wave propagation studies, providing a more efficient and accurate approach to verify and validate finite element models with potential applications in engineering simulations.

Kinetic samplers for neural quantum states

Neural quantum states are a recently introduced class of variational many-body wave functions that are very flexible in approximating diverse quantum states. Optimization of an NQS ansatz requires sampling from the corresponding probability distribution defined by squared wave function amplitude. For this purpose, we propose to use kinetic sampling protocols and demonstrate that in many important cases such methods lead to much smaller autocorrelation times than the Metropolis-Hastings sampling algorithm while still allowing to easily implement lattice symmetries (unlike autoregressive models). We also use uniform manifold approximation and projection algorithm to construct two-dimensional isometric embedding of Markov chains and show that kinetic sampling helps attain a more homogeneous and ergodic coverage of the Hilbert space basis.

Bayesian inversion for electromyography using low-rank tensor formats

The reconstruction of the structure of biological tissue using electromyographic (EMG) data is a non-invasive imaging method with diverse medical applications. Mathematically, this process is an inverse problem. Furthermore, EMG data are highly sensitive to changes in the electrical conductivity that describes the structure of the tissue. Modeling the inevitable measurement error as a stochastic quantity leads to a Bayesian approach. Solving the discretized Bayesian inverse problem means drawing samples from the posterior distribution of parameters, e.g., the conductivity, given measurement data. Using, e.g., a Metropolis–Hastings algorithm for this purpose involves solving the forward problem for different parameter combinations which requires a high computational effort. Low-rank tensor formats can reduce this effort by providing a data-sparse representation of all occurring linear systems of equations simultaneously and allow for their efficient solution. The application of Bayes’ theorem proves the well-posedness of the Bayesian inverse problem. The derivation and proof of a low-rank representation of the forward problem allow for the precomputation of all solutions of this problem under certain assumptions, resulting in an efficient and theory-based sampling algorithm. Numerical experiments support the theoretical results, but also indicate that a high number of samples is needed to obtain reliable estimates for the parameters. The Metropolis–Hastings sampling algorithm, using the precomputed forward solution in a tensor format, draws this high number of samples and therefore enables solving problems which are infeasible using classical methods.

A Process Model of Causal Reasoning.

How do we make causal judgments? Many studies have demonstrated that people are capable causal reasoners, achieving success on tasks from reasoning to categorization to interventions. However, less is known about the mental processes used to achieve such sophisticated judgments. We propose a new process model-the mutation sampler-that models causal judgments as based on a sample of possible states of the causal system generated using the Metropolis-Hastings sampling algorithm. Across a diverse array of tasks and conditions encompassing over 1,700 participants, we found that our model provided a consistently closer fit to participant judgments than standard causal graphical models. In particular, we found that the biases introduced by mutation sampling accounted for people's consistent, predictable errors that the normative model by definition could not. Moreover, using a novel experimental methodology, we found that those biases appeared in the samples that participants explicitly judged to be representative of a causal system. We conclude by advocating sampling methods as plausible process-level accounts of the computations specified by the causal graphical model framework and highlight opportunities for future research to identify not just what reasoners compute when drawing causal inferences, but also how they compute it.

X-DMM: Fast and Scalable Model Based Text Clustering

Text clustering is a widely studied problem in the text mining domain. The Dirichlet Multinomial Mixture (DMM) model based clustering algorithms have shown good performance to cope with high dimensional sparse text data, obtaining reasonable results in both clustering accuracy and computational efficiency. However, the time complexity of DMM model training is proportional to the average document length and the number of clusters, making it inefficient for scaling up to long text and large corpora, which is common in realworld applications such as documents organization, retrieval and recommendation. In this paper, we leverage a symmetric prior setting for Dirichlet distribution, and build indices to decrease the time complexity of the sampling-based training for DMM from O(K∗L) to O(K∗U), where K is the number of clusters, L the average length of document, and U the average number of unique words in each document. We introduce a Metropolis-Hastings sampling algorithm, which further reduces the sampling time complexity from O(K∗U) to O(U) in the nearly-to-convergence training stages. Moreover, we also parallelize the DMM model training to obtain a further acceleration by using an uncollapsed Gibbs sampler. We combine all these optimizations into a highly efficient implementation, called X-DMM, which enables the DMM model to scale up for long and large-scale text clustering. We evaluate the performance of X-DMM on several real world datasets, and the experimental results show that XDMM achieves substantial speed up compared with existing state-of-the-art algorithms without clustering accuracy degradation.

A Bayesian inference approach: estimation of heat flux from fin for perturbed temperature data

This paper reports the estimation of the unknown boundary heat flux from a fin using the Bayesian inference method. The setup consists of a rectangular mild steel fin of dimensions 250×150×6 mm3 and an aluminium base plate of dimensions 250×150×8 mm3. The fin is subjected to constant heat flux at the base and the fin setup is modelled using ANSYS14.5. The problem considered is a conjugate heat transfer from the fin, and the Navier–Stokes equation is solved to obtain the flow parameters. Grid independence study is carried out to fix the number of grids for the study considered. To reduce the computational cost, computational fluid dynamics (CFD) is replaced with artificial neural network (ANN) as the forward model. The Markov Chain Monte Carlo (MCMC) powered by Metropolis–Hastings sampling algorithm along with the Bayesian framework is used to explore the estimation space. The sensitivity analysis of the estimated temperature with respect to the unknown parameter is discussed to know the dependency of the temperature with the parameter. This paper signifies the effect of a prior model on the execution of the inverse algorithm at different noise levels. The unknown heat flux is estimated for the surrogated temperature and the estimates are reported as mean, Maximum a Posteriori (MAP) and standard deviation. The effect of a-priori information on the estimated parameter is also addressed. The standard deviation in the estimation process is referred to as the uncertainty associated with the estimated parameters.

Granger Causality and Regime Inference in Bayesian Markov-Switching VARs

We derive restrictions for Granger noncausality in Markov-switching vector autoregressive models and also show under which conditions a variable does not affect the forecast of the hidden Markov process. Based on Bayesian approach to evaluating the hypotheses, the computational tools for posterior inference include a novel block Metropolis-Hastings sampling algorithm for the estimation of the restricted models. We analyze a system of monthly US data on money and income. The test results in MS-VARs contradict those in linear VARs: the money aggregate M1 is useful for forecasting income and for predicting the next period’s state.

Estimating Correlated Jumps and Stochastic Volatilities

We formulate a bivariate stochastic volatility jump-diffusion model with correlated jumps and volatilities. An MCMC Metropolis-Hastings sampling algorithm is proposed to estimate the model's parameters and latent state variables (jumps and stochastic volatilities) given observed returns. The methodology is successfully tested on several artificially generated bivariate time series and then on the two most important Czech domestic financial market time series of the FX (CZK/EUR) and stock (PX index) returns. Four bivariate models with and without jumps and/or stochastic volatility are compared using the deviance information criterion (DIC) confirming importance of incorporation of jumps and stochastic volatility into the model.

Combining Integral Transforms and Bayesian Inference in the Simultaneous Identification of Variable Thermal Conductivity and Thermal Capacity in Heterogeneous Media

This work presents the combined use of the integral transform method, for the direct problem solution, and of Bayesian inference, for the inverse problem analysis, in the simultaneous estimation of spatially variable thermal conductivity and thermal capacity for one-dimensional heat conduction within heterogeneous media. The direct problem solution is analytically obtained via integral transforms and the related eigenvalue problem is solved by the generalized integral transform technique (GITT), offering a fast, precise, and robust solution for the transient temperature field. The inverse problem analysis employs a Markov chain Monte Carlo (MCMC) method, through the implementation of the Metropolis-Hastings sampling algorithm. Instead of seeking the functions estimation in the form of local values for the thermal conductivity and capacity, an alternative approach is employed based on the eigenfunction expansion of the thermophysical properties themselves. Then, the unknown parameters become the corresponding series coefficients for the properties eigenfunction expansions. Simulated temperatures obtained via integral transforms are used in the inverse analysis, for a prescribed concentration distribution of the dispersed phase in a heterogeneous media such as particle filled composites. Available correlations for the thermal conductivity and theory of mixtures relations for the thermal capacity are employed to produce the simulated results with high precision in the direct problem solution, while eigenfunction expansions with reduced number of terms are employed in the inverse analysis itself, in order to avoid the inverse crime. Gaussian distributions were used as priors for the parameter estimation procedure. In addition, simulated results with different randomly generated errors were employed in order to test the inverse analysis robustness.

Estimating Correlated Jumps and Stochastic Volatilities

We formulate a bivariate stochastic volatility jump-diffusion model with correlated jumps and volatilities. An MCMC Metropolis-Hastings sampling algorithm is proposed to estimate the model’s parameters and latent state variables (jumps and stochastic volatilities) given observed returns. The methodology is successfully tested on several artificially generated bivariate time series and then on the two most important Czech domestic financial market time series of the FX (CZK/EUR) and stock (PX index) returns. Four bivariate models with and without jumps and/or stochastic volatility are compared using the deviance information criterion (DIC) confirming importance of incorporation of jumps and stochastic volatility into the model.

Source Reconstruction of Unknown Model Parameters in Atmospheric Dispersion Using Dynamic Bayesian Inference

A source reconstruction capability driven by time-evolving data is tried out by coupling the observed data and predictive model through dynamic Bayesian inference to obtain solutions to inverse problem. Solutions are determined by posterior probability distributions describing unknown model parameters and the Markov Chain Monte Carlo method with the Metropolis-Hastings sampling algorithm is employed to obtain the solutions. The posterior distributions of model parameters are obtained by performing stochastic sampling by the likelihood function test indicating the agreements between the measurements and the predictions from Gaussian plume model. The Yonggwang atmospheric tracer experiment in Korea is selected to testify the source reconstruction algorithm. The simulation has shown that the posterior modes of the release point and the released source rate for this experiment obviously converged to their true values.

Characterization of Petroleum-Hydrocarbon Fate and Transport in Homogeneous and Heterogeneous Aquifers Using a Generalized Uncertainty Estimation Method

This paper presents a generalized uncertainty estimation method (GUEM) to characterize the fate and transport of petroleum-hydrocarbon contaminants in subsurface environments. Compared to the conventional methods, GUEM employs the Metropolis-Hastings sampling algorithm to replace the conventional Monte Carlo algorithm; this algorithm has the advantage of reducing computational cost in obtaining realizations of uncertain parameters. The GUEM method is applied to a hypothetical homogenous and a real-world heterogonous site for demonstration studies. The first demonstration shows that GUEM generates outputs with wider bounds than conventional uncertainty characterization methods. This avoids overestimation of lower bounds and underestimation of upper bounds of resulting modeling outputs. Results from the second demonstration implied that the groundwater would not be suitable for the drinking water source after three years of natural attenuation and remediation actions should be taken to guarantee the groundwater safety. Even after taking a three-year pump-and-treat remediation action, it would pose risks as one can conclude that the benzene concentrations would violate the regulated environmental guideline with a possibility of over 0.9, and enhanced remediation actions are still desired to be taken for improving the groundwater quality. Despite the advantages in characterizing the fate and transport of contaminants, GUEM could be improved by introducing two correlated random variables in the sampling process for enhancing simulation accuracy. It is also expected to be integrated with traditional methods such as Monte Carlo method and generalized likelihood uncertainty estimation method to deal with hybrid fuzzy-random and interval-random inputs, respectively.

Analyzing lateral seabed variability with Bayesian inference of seabed reflection data

This paper considers Bayesian inversion of seabed reflection-coefficient data for multi-layer geoacoustic models at several sites, with the goal of studying lateral variability of the seabed. Rigorous uncertainty estimation is carried out to resolve lateral variability of the sediments from inherent inversion uncertainties. The uncertainty analysis includes Bayesian model selection, comprehensive quantification of data error statistics, and a Markov-chain Monte Carlo approach to transforming data uncertainties to model uncertainties. Model selection is addressed using the Bayesian information criterion to ensure parsimony of the parametrizations. Data error statistics are quantified by estimating full covariance matrices from data residuals, with posterior statistical validation. A Metropolis-Hastings sampling algorithm is used to compute posterior probability densities. Four experiment sites are considered along a track located on the Malta Plateau, Mediterranean Sea, and the inversion results are compared to cores taken at each site. Differences between profile marginal-probability distributions at adjacent sites are quantified using the Bhattacharyya coefficient. Differences that exceed the estimated geoacoustic uncertainties are interpreted as spatial variability of the seabed. The results are compared to an interpretation of geologic features evident in a chirp sub-bottom-profiler section.

Estimation of parameters in multi-mode heat transfer problems using Bayesian inference – Effect of noise and a priori

Parameter estimation problems and heat source/flux reconstruction problems are some of the most frequently encountered inverse heat transfer problems. These problems find their application in many areas of science and engineering. The primary focus of this paper is on the heat transfer parameter estimation for a two-dimensional unsteady heat conduction problem with (a) convection boundary condition and (b) convection and radiation boundary condition. The paper demonstrates the effect of a priori model on the performance of the algorithm at different noise levels in the measured data. The inverse problem is solved using three different a priori models namely normal, log normal and uniform. The posterior PDF is sampled using the Metropolis–Hastings sampling algorithm. Both single-parameter estimation and multi-parameter estimation problems are addressed and the effects of corresponding a priori models are studied. It was found that the mean and maximum a posteriori estimates for thermal conductivity and the convection heat transfer coefficient were insensitive to the a priori model at all the considered noise levels for the single-parameter estimation problem. At high noise levels in the two-parameter estimation problem, the estimates for thermal conductivity and convection coefficient were sensitive to the a priori model. It was also found that the standard deviation of the samples was correlated to the error in estimation in the single-parameter estimation case. In three parameter estimation case, alternate solutions to the same problem were retrieved due to a strong correlation between the convection coefficient and the emissivity. However, a more informative a priori model could address this issue.

Bayesian Prediction Analysis for Growth Curve Model Using Noninformative Priors

We apply a Bayesian approach to the problem of prediction in an unbalanced growth curve model using noninformative priors. Due to the complexity of the model, no analytic forms of the predictive densities are available. We propose both approximations and a prediction-oriented Metropolis-Hastings sampling algorithm for two types of prediction, namely the prediction of future observations for a new subject and the prediction of future values for a partially observed subject. They are illustrated and compared through real data and simulation studies. Two of the approximations compare favorably with the approximation in Fearn (1975, Biometrika, 62, 89–100) and are very comparable to the more accurate Rao-Blackwellization from Metropolis-Hastings sampling algorithm.