Adaptive Markov Chain Monte Carlo Algorithms Research Articles

Uncertainty Quantification of Water Level Predictions from Radar‐based Areal Rainfall Using an Adaptive MCMC Algorithm

This study proposes an approach for the uncertainty quantification at each stage of a single hydrological process of water level predictions based on different sources of mean areal precipitation (MAP) forecasts by using an adaptive Bayesian Markov chain Monte Carlo (MCMC) approach. The MAP forecasts are derived from the McGill Algorithm for Precipitation Nowcasting by Lagrangian Extrapolation (MAPLE) system and a long short-term memory (LSTM) network. The predicted water levels at two stations in the Gangnam catchment, Seoul, South Korea, are processed with a coupled 1D/2D urban hydrological model (1D/2D-UHM) forced by MAPLE MAP forecasts and LSTM-corrected MAP forecasts of five heavy rainfall events. The proposed Bayesian approach using the delayed rejection and adaptive Metropolis (DRAM) algorithm was compared with the Metropolis-Hastings (MH) algorithm in the uncertainty estimation of Weibull distribution parameters. The uncertainty contributions of the stages and sources in the related process were analyzed, including quantitative precipitation estimation (QPE) inputs, MAP inputs and 1D/2D-UHM. The results indicate that the uncertainty contribution of the MAPLE MAP forecasting is the highest in the 3-hour forecasting time. The uncertainty contribution of the QPE input for MAPLE MAP forecasting is the smallest and that of two sources, including the LSTM-corrected MAP source, and MAP and the coupled model is more significant than that of the QPE input. This research showed that the adaptive Bayesian MCMC method using the DRAM algorithm might be a robust option in quantitative uncertainty analyses of a single hydrological process, especially for urban flood management.

Path differential-informed stratified MCMC and adaptive forward path sampling

Markov Chain Monte Carlo (MCMC) rendering is extensively studied, yet it remains largely unused in practice. We propose solutions to several practicability issues, opening up path space MCMC to become an adaptive sampling framework around established Monte Carlo (MC) techniques. We address non-uniform image quality by deriving an analytic target function for imagespace sample stratification. The function is based on a novel connection between variance and path differentials, allowing analytic variance estimates for MC samples, with potential uses in other adaptive algorithms outside MCMC. We simplify these estimates down to simple expressions using only quantities known in any MC renderer. We also address the issue that most existing MCMC renderers rely on bi-directional path tracing and reciprocal transport, which can be too costly and/or too complex in practice. Instead, we apply our theoretical framework to optimize an adaptive MCMC algorithm that only uses forward path construction. Notably, we construct our algorithm by adapting (with minimal changes) a full-featured path tracer into a single-path state space Markov Chain, bridging another gap between MCMC and existing MC techniques.

Randomized reduced forward models for efficient Metropolis–Hastings MCMC, with application to subsurface fluid flow and capacitance tomography

Bayesian modelling and computational inference by Markov chain Monte Carlo (MCMC) is a principled framework for large-scale uncertainty quantification, though is limited in practice by computational cost when implemented in the simplest form that requires simulating an accurate computer model at each iteration of the MCMC. The delayed acceptance Metropolis–Hastings MCMC leverages a reduced model for the forward map to lower the compute cost per iteration, though necessarily reduces statistical efficiency that can, without care, lead to no reduction in the computational cost of computing estimates to a desired accuracy. Randomizing the reduced model for the forward map can dramatically improve computational efficiency, by maintaining the low cost per iteration but also avoiding appreciable loss of statistical efficiency. Randomized maps are constructed by a posteriori adaptive tuning of a randomized and locally-corrected deterministic reduced model. Equivalently, the approximated posterior distribution may be viewed as induced by a modified likelihood function for use with the reduced map, with parameters tuned to optimize the quality of the approximation to the correct posterior distribution. Conditions for adaptive MCMC algorithms allow practical approximations and algorithms that have guaranteed ergodicity for the target distribution. Good statistical and computational efficiencies are demonstrated in examples of calibration of large-scale numerical models of geothermal reservoirs and electrical capacitance tomography.

A Bayesian Approach to Real-Time Dynamic Parameter Estimation Using Phasor Measurement Unit Measurement

In this work, we develop a polynomial-chaos-expa- nsion (PCE)-based approach for decentralized dynamic parameter estimation through Bayesian inference. Using this approach, the non-Gaussian distribution of the inverted parameters is obtained. More specifically, we first represent the decentralized generator model with the PCE-based surrogate. This surrogate allows us to efficiently evaluate the time-consuming dynamic solver at parameter values through Metropolis-Hastings (M-H)-based Markov chain Monte Carlo (MCMC). Then, we propose a two-stage hybrid Markov chain Monte Carlo (MCMC) to recover a posteriori distribution of the decentralized generator model parameters. In the first stage, we use the gradient-enhanced Langevin MCMC algorithm to characterize an intermediate posterior parameter distribution. This algorithm is computationally scalable to the high-dimensional parameter space. Based on the intermediate posterior distribution, during the second stage, we use the adaptive MCMC algorithm to fine-tune the strong correlations between the parameters. Finally, the fully recovered a posterior distribution is obtained in the end. The simulation results show that the proposed PCE-based hybrid MCMC algorithm can accurately and efficiently estimate the high-dimensional generator dynamic model parameters with full probabilistic distribution provided.

A low-complexity algorithm for the joint antenna selection and user scheduling in multi-cell multi-user downlink massive MIMO systems

The massive MIMO (multiple-input multiple-output) technology plays a key role in the next-generation (5G) wireless communication systems, which are equipped with a large number of antennas at the base station (BS) of a network to improve cell capacity for network communication systems. However, activating a large number of BS antennas needs a large number of radio-frequency (RF) chains that introduce the high cost of the hardware and high power consumption. Our objective is to achieve the optimal combination subset of BS antennas and users to approach the maximum cell capacity, simultaneously. However, the optimal solution to this problem can be achieved by using an exhaustive search (ES) algorithm by considering all possible combinations of BS antennas and users, which leads to the exponential growth of the combinatorial complexity with the increasing of the number of BS antennas and active users. Thus, the ES algorithm cannot be used in massive MIMO systems because of its high computational complexity. Hence, considering the trade-off between network performance and computational complexity, we proposed a low-complexity joint antenna selection and user scheduling (JASUS) method based on Adaptive Markov Chain Monte Carlo (AMCMC) algorithm for multi-cell multi-user massive MIMO downlink systems. AMCMC algorithm is helpful for selecting combination subset of antennas and users to approach the maximum cell capacity with consideration of the multi-cell interference. Performance analysis and simulation results show that AMCMC algorithm performs extremely closely to ES-based JASUS algorithm. Compared with other algorithms in our experiments, the higher cell capacity and near-optimal system performance can be obtained by using the AMCMC algorithm. At the same time, the computational complexity is reduced significantly by combining with AMCMC.

Adaptive Markov chain Monte Carlo algorithms for Bayesian inference: recent advances and comparative study

Condition assessments of structures require prediction models such as empirical model and numerical simulation model. Generally, these prediction models have model parameters to be estimated from experimental data. Bayesian inference is the formal statistical framework to estimate the model parameters and their uncertainties. As a result, uncertainties associated with the model and measurement can be accounted for decision making. Markov Chain Monte Carlo (MCMC) algorithms have been widely employed. However, there still remain some implementation issues from the inappropriate selection of the proposal mechanism in Markov chain. Since the posterior density for a given problem is often problem-dependent and unknown, users require a trial-and-error approach to select and tune optimal proposal mechanism. To relieve this difficulty, various adaptive MCMC algorithms have been recently appeared. Users must understand their mechanism and limitations before applying the algorithms to their problems. However, there is no comprehensive work to provide detailed exposition and their performance comparison together. This study aims to bring together different adaptive MCMC algorithms with the goal of providing their mechanisms and evaluating their performances through comparative study. Three algorithms are chosen as the representative proposal mechanism. From comparative studies, the discussions were drawn in terms of performances, simplicity and computational costs for less-experienced users.

Robust and Fast Markov Chain Monte Carlo Sampling of Diffusion MRI Microstructure Models.

In diffusion MRI analysis, advances in biophysical multi-compartment modeling have gained popularity over the conventional Diffusion Tensor Imaging (DTI), because they can obtain a greater specificity in relating the dMRI signal to underlying cellular microstructure. Biophysical multi-compartment models require a parameter estimation, typically performed using either the Maximum Likelihood Estimation (MLE) or the Markov Chain Monte Carlo (MCMC) sampling. Whereas, the MLE provides only a point estimate of the fitted model parameters, the MCMC recovers the entire posterior distribution of the model parameters given in the data, providing additional information such as parameter uncertainty and correlations. MCMC sampling is currently not routinely applied in dMRI microstructure modeling, as it requires adjustment and tuning, specific to each model, particularly in the choice of proposal distributions, burn-in length, thinning, and the number of samples to store. In addition, sampling often takes at least an order of magnitude, more time than non-linear optimization. Here we investigate the performance of the MCMC algorithm variations over multiple popular diffusion microstructure models, to examine whether a single, well performing variation could be applied efficiently and robustly to many models. Using an efficient GPU-based implementation, we showed that run times can be removed as a prohibitive constraint for the sampling of diffusion multi-compartment models. Using this implementation, we investigated the effectiveness of different adaptive MCMC algorithms, burn-in, initialization, and thinning. Finally we applied the theory of the Effective Sample Size, to the diffusion multi-compartment models, as a way of determining a relatively general target for the number of samples needed to characterize parameter distributions for different models and data sets. We conclude that adaptive Metropolis methods increase MCMC performance and select the Adaptive Metropolis-Within-Gibbs (AMWG) algorithm as the primary method. We furthermore advise to initialize the sampling with an MLE point estimate, in which case 100 to 200 samples are sufficient as a burn-in. Finally, we advise against thinning in most use-cases and as a relatively general target for the number of samples, we recommend a multivariate Effective Sample Size of 2,200.

Efficient Adaptive MCMC Through Precision Estimation

ABSTRACTThe performance of Markov chain Monte Carlo (MCMC) algorithms like the Metropolis Hastings Random Walk (MHRW) is highly dependent on the choice of scaling matrix for the proposal distributions. A popular choice of scaling matrix in adaptive MCMC methods is to use the empirical covariance matrix (ECM) of previous samples. However, this choice is problematic if the dimension of the target distribution is large, since the ECM then converges slowly and is computationally expensive to use. We propose two algorithms to improve convergence and decrease computational cost of adaptive MCMC methods in cases when the precision (inverse covariance) matrix of the target density can be well-approximated by a sparse matrix. The first is an algorithm for online estimation of the Cholesky factor of a sparse precision matrix. The second estimates the sparsity structure of the precision matrix. Combining the two algorithms allows us to construct precision-based adaptive MCMC algorithms that can be used as black-box methods for densities with unknown dependency structures. We construct precision-based versions of the adaptive MHRW and the adaptive Metropolis adjusted Langevin algorithm and demonstrate the performance of the methods in two examples. Supplementary materials for this article are available online.

Adaptive independent sticky MCMC algorithms

Monte Carlo methods have become essential tools to solve complex Bayesian inference problems in different fields, such as computational statistics, machine learning, and statistical signal processing. In this work, we introduce a novel class of adaptive Monte Carlo methods, called adaptive independent sticky Markov Chain Monte Carlo (MCMC) algorithms, to sample efficiently from any bounded target probability density function (pdf). The new class of algorithms employs adaptive non-parametric proposal densities, which become closer and closer to the target as the number of iterations increases. The proposal pdf is built using interpolation procedures based on a set of support points which is constructed iteratively from previously drawn samples. The algorithm’s efficiency is ensured by a test that supervises the evolution of the set of support points. This extra stage controls the computational cost and the convergence of the proposal density to the target. Each part of the novel family of algorithms is discussed and several examples of specific methods are provided. Although the novel algorithms are presented for univariate target densities, we show how they can be easily extended to the multivariate context by embedding them within a Gibbs-type sampler or the hit and run algorithm. The ergodicity is ensured and discussed. An overview of the related works in the literature is also provided, emphasizing that several well-known existing methods (like the adaptive rejection Metropolis sampling (ARMS) scheme) are encompassed by the new class of algorithms proposed here. Eight numerical examples (including the inference of the hyper-parameters of Gaussian processes, widely used in machine learning for signal processing applications) illustrate the efficiency of sticky schemes, both as stand-alone methods to sample from complicated one-dimensional pdfs and within Gibbs samplers in order to draw from multi-dimensional target distributions.

Adaptive MCMC for multiple changepoint analysis with applications to large datasets

We consider the problem of Bayesian inference for changepoints where the number and position of the changepoints are both unknown. In particular, we consider product partition models where it is possible to integrate out model parameters for the regime between each changepoint, leaving a posterior distribution over a latent vector indicating the presence or not of a changepoint at each observation. The same problem setting has been considered by Fearnhead (2006) where one can use filtering recursions to make exact inference. However, the complexity of this filtering recursions algorithm is quadratic in the number of observations. Our approach relies on an adaptive Markov Chain Monte Carlo (MCMC) method for finite discrete state spaces. We develop an adaptive algorithm which can learn from the past states of the Markov chain in order to build proposal distributions which can quickly discover where changepoint are likely to be located. We prove that our algorithm leaves the posterior distribution ergodic. Crucially, we demonstrate that our adaptive MCMC algorithm is viable for large datasets for which the filtering recursions approach is not. Moreover, we show that inference is possible in a reasonable time thus making Bayesian changepoint detection computationally efficient.

A Hybrid Adaptive MCMC Algorithm in Function Spaces

The preconditioned Crank--Nicolson (pCN) method is a Markov chain Monte Carlo (MCMC) scheme, specifically designed to perform Bayesian inferences in function spaces. Unlike many standard MCMC algorithms, the pCN method can preserve the sampling efficiency under the mesh refinement, a property referred to as being dimension independent. In this work we consider an adaptive strategy to further improve the efficiency of pCN. In particular we develop a hybrid adaptive MCMC method: the algorithm performs an adaptive Metropolis scheme in a chosen finite dimensional subspace and a standard pCN algorithm in the complement space of the chosen subspace. We show that the proposed algorithm satisfies certain important ergodicity conditions. Finally with numerical examples we demonstrate that the proposed method has competitive performance with existing adaptive algorithms.

Automatically tuned general-purpose MCMC via new adaptive diagnostics

Adaptive Markov Chain Monte Carlo (MCMC) algorithms attempt to `learn' from the results of past iterations so the Markov chain can converge quicker. Unfortunately, adaptive MCMC algorithms are no longer Markovian, so their convergence is difficult to guarantee. In this paper, we develop new diagnostics to determine whether the adaption is still improving the convergence. We present an algorithm which automatically stops adapting once it determines further adaption will not increase the convergence speed. Our algorithm allows the computer to tune a `good' Markov chain through multiple phases of adaption, and then run conventional non-adaptive MCMC. In this way, the efficiency gains of adaptive MCMC can be obtained while still ensuring convergence to the target distribution.

Adaptive MCMC with online relabeling

When targeting a distribution that is artificially invariant under some permutations, Markov chain Monte Carlo (MCMC) algorithms face the label-switching problem, rendering marginal inference particularly cumbersome. Such a situation arises, for example, in the Bayesian analysis of finite mixture models. Adaptive MCMC algorithms such as adaptive Metropolis (AM), which self-calibrates its proposal distribution using an online estimate of the covariance matrix of the target, are no exception. To address the label-switching issue, relabeling algorithms associate a permutation to each MCMC sample, trying to obtain reasonable marginals. In the case of adaptive Metropolis (Bernoulli 7 (2001) 223-242), an online relabeling strategy is required. This paper is devoted to the AMOR algorithm, a provably consistent variant of AM that can cope with the label-switching problem. The idea is to nest relabeling steps within the MCMC algorithm based on the estimation of a single covariance matrix that is used both for adapting the covariance of the proposal distribution in the Metropolis algorithm step and for online relabeling. We compare the behavior of AMOR to similar relabeling methods. In the case of compactly supported target distributions, we prove a strong law of large numbers for AMOR and its ergodicity. These are the first results on the consistency of an online relabeling algorithm to our knowledge. The proof underlines latent relations between relabeling and vector quantization.

Adaptive Metropolis algorithm using variational Bayesian adaptive Kalman filter

Markov chain Monte Carlo (MCMC) methods are powerful computational tools for analysis of complex statistical problems. However, their computational efficiency is highly dependent on the chosen proposal distribution, which is generally difficult to find. One way to solve this problem is to use adaptive MCMC algorithms which automatically tune the statistics of a proposal distribution during the MCMC run. A new adaptive MCMC algorithm, called the variational Bayesian adaptive Metropolis (VBAM) algorithm, is developed. The VBAM algorithm updates the proposal covariance matrix using the variational Bayesian adaptive Kalman filter (VB-AKF). A strong law of large numbers for the VBAM algorithm is proven. The empirical convergence results for three simulated examples and for two real data examples are also provided.

Randomize-Then-Optimize: A Method for Sampling from Posterior Distributions in Nonlinear Inverse Problems

High-dimensional inverse problems present a challenge for Markov chain Monte Carlo (MCMC)-type sampling schemes. Typically, they rely on finding an efficient proposal distribution, which can be difficult for large-scale problems, even with adaptive approaches. Moreover, the autocorrelations of the samples typically increase with dimension, which leads to the need for long sample chains. We present an alternative method for sampling from posterior distributions in nonlinear inverse problems, when the measurement error and prior are both Gaussian. The approach computes a candidate sample by solving a stochastic optimization problem. In the linear case, these samples are directly from the posterior density, but this is not so in the nonlinear case. We derive the form of the sample density in the nonlinear case, and then show how to use it within both a Metropolis--Hastings and importance sampling framework to obtain samples from the posterior distribution of the parameters. We demonstrate, with various small- and medium-scale problems, that randomize-then-optimize can be efficient compared to standard adaptive MCMC algorithms.

An Adaptive Sequential Monte Carlo Sampler

Sequential Monte Carlo (SMC) methods are not only a popular tool in the analysis of state–space models, but offer an alternative to Markov chain Monte Carlo (MCMC) in situations where Bayesian inference must proceed via simulation. This paper introduces a new SMC method that uses adaptive MCMC kernels for particle dynamics. The proposed algorithm features an online stochastic optimization procedure to select the best MCMC kernel and simultaneously learn optimal tuning parameters. Theoretical results are presented that justify the approach and give guidance on how it should be implemented. Empirical results, based on analysing data from mixture models, show that the new adaptive SMC algorithm (ASMC) can both choose the best MCMC kernel, and learn an appropriate scaling for it. ASMC with a choice between kernels outperformed the adaptive MCMC algorithm of Haario et al. (1998) in 5 out of the 6 cases considered.

Efficient MCMC for Climate Model Parameter Estimation: Parallel Adaptive Chains and Early Rejection

The emergence of Markov chain Monte Carlo (MCMC) methods has opened a way for Bayesian analysis of complex models. Running MCMC samplers typically requires thousands of model evaluations, which can exceed available computer resources when this evaluation is computationally intensive. We will discuss two generally applicable techniques to improve the efficiency of MCMC. First, we consider a parallel version of the adaptive MCMC algorithm of Haario et al. (2001), implementing the idea of inter-chain adaptation introduced by Craiu et al. (2009). Second, we present an early rejection (ER) approach, where model simulation is stopped as soon as one can conclude that the proposed parameter value will be rejected by the MCMC algorithm. This work is motivated by practical needs in estimating parameters of climate and Earth system models. These computationally intensive models involve non-linear expressions of the geophysical and biogeochemical processes of the Earth system. Modeling of these processes, especially those operating in scales smaller than the model grid, involves a number of specified parameters, or ‘tunables’. MCMC methods are applicable for estimation of these parameters, but they are computationally very demanding. Efficient MCMC variants are thus needed to obtain reliable results in reasonable time. Here we evaluate the computational gains attainable through parallel adaptive MCMC and Early Rejection using both simple examples and a realistic climate model.

Bayesian calibration of a large‐scale geothermal reservoir model by a new adaptive delayed acceptance Metropolis Hastings algorithm

The aim of this research is to estimate the parameters of a large‐scale numerical model of a geothermal reservoir using Markov chain Monte Carlo (MCMC) sampling, within the framework of Bayesian inference. All feasible parameters that are consistent with the measured data are summarized by the posterior distribution, and hence parameter estimation and uncertainty quantification are both given by calculating expected values of statistics of interest over the posterior distribution. It appears to be computationally infeasible to use the standard Metropolis‐Hastings algorithm (MH) to sample the high dimensional computationally expensive posterior distribution. To improve the sampling efficiency, a new adaptive delayed‐acceptance MH algorithm (ADAMH) is implemented to adaptively build a stochastic model of the error introduced by the use of a reduced‐order model. This use of adaptivity differs from existing adaptive MCMC algorithms that tune proposal distributions of the Metropolis‐Hastings algorithm (MH), though ADAMH also implements that technique. For the 3‐D geothermal reservoir model we present here, ADAMH shows a great improvement in the computational efficiency of the MCMC sampling, and promising results for parameter estimation and uncertainty quantification are obtained. This algorithm could offer significant improvement in computational efficiency when implementing sample‐based inference in other large‐scale inverse problems.

Divide and Conquer: A Mixture-Based Approach to Regional Adaptation for MCMC

The efficiency of Markov chain Monte Carlo (MCMC) algorithms can vary dramatically with the choice of simulation parameters. Adaptive MCMC (AMCMC) algorithms allow the automatic tuning of the parameters while the simulation is in progress. A multimodal target distribution may call for regional adaptation of Metropolis–Hastings samplers so that the proposal distribution varies across regions in the sample space. Establishing such a partition is not straightforward and, in many instances, the learning required for its specification takes place gradually, as the simulation proceeds. In the case in which the target distribution is approximated by a mixture of Gaussians, we propose an adaptation process for the partition. It involves fitting the mixture using the available samples via an online EM algorithm and, based on the current mixture parameters, constructing the regional adaptive algorithm with online recursion (RAPTOR). The method is compared with other regional AMCMC samplers and is tested on simulated as well as real data examples.Relevant theoretical proofs, code and datasets are posted as an online supplement.

Adaptive evolutionary Monte Carlo algorithm for optimization with applications to sensor placement problems

In this paper, we present an adaptive evolutionary Monte Carlo algorithm (AEMC), which combines a tree-based predictive model with an evolutionary Monte Carlo sampling procedure for the purpose of global optimization. Our development is motivated by sensor placement applications in engineering, which requires optimizing certain complicated "black-box" objective function. The proposed method is able to enhance the optimization efficiency and effectiveness as compared to a few alternative strategies. AEMC falls into the category of adaptive Markov chain Monte Carlo (MCMC) algorithms and is the first adaptive MCMC algorithm that simulates multiple Markov chains in parallel. A theorem about the ergodicity property of the AEMC algorithm is stated and proven. We demonstrate the advantages of the proposed method by applying it to a sensor placement problem in a manufacturing process, as well as to a standard Griewank test function.