Stochastic Approximation Monte Carlo Research Articles

Stochastic approximation Monte Carlo with a dynamic update factor.

We present a Monte Carlo algorithm based on the stochastic approximation Monte Carlo (SAMC) algorithm for directly calculating the density of states. The proposed method is stochastic approximation with a dynamic update factor (SAD), which dynamically adjusts the update factor γ_{t} during the course of the simulation. We test this method on a square-well fluid and a 31-atom Lennard-Jones cluster and compare the convergence behavior of several related Monte Carlo methods. We find that both the SAD and 1/t-Wang-Landau (1/t-WL) methods rapidly converge to the correct density of states without the need for the user to specify an arbitrary tunable parameter t_{0} as in the case of SAMC. SAD requires as input the temperature range of interest, in contrast with 1/t-WL, which requires that the user identify the interesting range of energies. The convergence of the 1/t-WL method is very sensitive to the energy range chosen for the low-temperature heat capacity of the Lennard-Jones cluster. Thus, SAD is more powerful in the common case in which the range of energies is not known in advance.

Crystallisation in Melts of Short, Semi-Flexible Hard-Sphere Polymer Chains: The Role of the Non-Bonded Interaction Range

A melt of short semi-flexible polymers with hard-sphere-type non-bonded interaction undergoes a first-order crystallisation transition at lower density than a melt of hard-sphere monomers or a flexible hard-sphere chain. In contrast to the flexible hard-sphere chains, the semi-flexible ones have an intrinsic stiffness energy scale, which determines the natural temperature scale of the system. In this paper, we investigate the effect of weak additional non-bonded interaction on the phase transition temperature. We study the system using the stochastic approximation Monte Carlo (SAMC) method to estimate the micro-canonical entropy of the system. Since the density of states in the purely hard-sphere non-bonded interaction case already covers 5600 orders of magnitude, we consider the effect of weak interactions as a perturbation. In this case, the system undergoes the same ordering transition with a temperature shift non-uniformly depending on the additional interaction. Short-range attractions impede ordering of the melt of semi-flexible polymers and decrease the transition temperature, whereas relatively long-range attractions assist ordering and shift the transition temperature to higher values, whereas weak repulsive interactions demonstrate an opposite effect on the transition temperature.

Estimation of oil reservoir parameters from temperature data for water injection

Fluids and oil flow rate, oil saturation and reservoir porosity as well as their measure method play a key role in petroleum engineering. Based on analysis of heat and mass transfer for hot water injection reservoir, this study presents a novel method to estimate the above parameters of oil reservoir simultaneously just from temperature data. The proposed method is an inversion method coupling with Monte Carlo stochastic approximation method. Firstly, a two-phase flow and heat transfer model in core flooding for hot water injection reservoir was built to simulate the reservoir temperature. Then, based on the built model, the concerned parameters were estimated sequentially by the inversion method from the reservoir temperature data which was obtained in a built experiment. Moreover, for determining sequence of parameters inversion, sensitivity analysis was performed to investigate correlation between the estimated parameters and reservoir temperature. Lastly, applying the presented method to two experimental water injection reservoirs, the transient flow rate, oil saturation and porosity during oil displacement were obtained. The prediction errors of oil flow rate and oil saturation for were less than 10% at all times, especially, the mean prediction accuracy for oil flow rate could reach 1.9%, and the mean accuracy for the oil saturation was about 2.0%.

Bayesian estimation of ordinary differential equation models when the likelihood has multiple local modes

Abstract Ordinary differential equations (ODEs) are popularly used to model complex dynamic systems by scientists; however, the parameters in ODE models are often unknown and have to be inferred from noisy measurements of the dynamic system. One conventional method is to maximize the likelihood function, but the likelihood function often has many local modes due to the complexity of ODEs, which makes the optimizing algorithm be vulnerable to trap in local modes. In this paper, we solve the global optimization issue of ODE parameters with the help of the Stochastic Approximation Monte Carlo (SAMC) algorithm which is shown to be self-adjusted and escape efficiently from the “local-trapping” problem. Our simulation studies indicate that the SAMC method is a powerful tool to estimate ODE parameters globally. The efficiency of SAMC method is demonstrated by estimating a predator-prey ODEs model from real experimental data.

Model-Based Proposal Learning for Monte Carlo Optimization of Redundancy Allocation Problem

Most existing approaches to heterogeneous redundancy allocation problem (RAP) are prone to getting trapped in local optimal modes during optimization, mainly due to the rugged combinatoric landscapes. Recently, optimization-by-sampling paradigm based on the stochastic approximation Monte Carlo (SAMC) sampling has shown superior performance in solving the heterogeneous RAP for multi-state systems (MSSs). However, one drawback of this method is that the global move of a Markov chain relying only on a uniform distribution is typically hard to hit the low-energy regions due to the uninformative proposal, leading to insufficient global exploration in sampling. To address the problem of where to sample for efficient optimization of heterogeneous RAP, we introduce a rejection-free Monte Carlo method to sample from the target distribution over the combinatorial space. Specifically, a model-based proposal learning algorithm is derived to guide the global exploration towards promising regions of the descrete state space. Experimental evaluations on a set of benchmark instances show the superiority of the proposed approach compared with the several state-of-the-arts in terms of the solution quality and computational efficiency.

Convergence of Stochastic Approximation Monte Carlo and modified Wang–Landau algorithms: Tests for the Ising model

We investigate the behavior of the deviation of the estimator for the density of states (DOS) with respect to the exact solution in the course of Wang–Landau and Stochastic Approximation Monte Carlo (SAMC) simulations of the two-dimensional Ising model. We find that the deviation saturates in the Wang–Landau case. This can be cured by adjusting the refinement scheme. To this end, the 1∕t-modification of the Wang–Landau algorithm has been suggested. A similar choice of refinement scheme is employed in the SAMC algorithm. The convergence behavior of all three algorithms is examined. It turns out that the convergence of the SAMC algorithm is very sensitive to the onset of the refinement. Finally, the internal energy and specific heat of the Ising model are calculated from the SAMC DOS and compared to exact values.

Stochastic approximation Monte Carlo algorithm for calculation of diagram of states of a single flexible-semiflexible copolymer chain

We report in this paper on details and special aspects of using the stochastic approximation Monte Carlo (SAMC) algorithm for the calculation of the diagram of states of a single flexible-semiflexible copolymer chain. The SAMC algorithm is a quite recently suggested mathematical generalization of Wang-Landautype algorithms for very precise Monte Carlo estimates of the density of states function g(E) in computer simulations. It has been mathematically proven that the SAMC algorithm converges to the true g(E) function in the limit of infinite sampling, i.e. systematic errors are smaller than statistical errors. However, in practice one faces the reality that statistical errors become small enough only in the limit of the prohibitively large computation time, if one applies this algorithm in a straightforward way, as it is described. Therefore, one usually needs to apply additional technical tricks to accelerate the convergence to a reasonably good function g(E) sampling all most important conformations. In this paper we discuss all details of these inner workings to make the reader aware of real computational efforts, available accuracy, reachable limits, etc. We do this for the first realization of this algorithm for calculation of the two-dimensional density of states function g(Econtact, Estiffness) which depends on two contributions to the total energy—intermonomer contact energy and the intramolecular stiffness energy due to chain bending.

Multidimensional stochastic approximation Monte Carlo.

Stochastic Approximation Monte Carlo (SAMC) has been established as a mathematically founded powerful flat-histogram Monte Carlo method, used to determine the density of states, g(E), of a model system. We show here how it can be generalized for the determination of multidimensional probability distributions (or equivalently densities of states) of macroscopic or mesoscopic variables defined on the space of microstates of a statistical mechanical system. This establishes this method as a systematic way for coarse graining a model system, or, in other words, for performing a renormalization group step on a model. We discuss the formulation of the Kadanoff block spin transformation and the coarse-graining procedure for polymer models in this language. We also apply it to a standard case in the literature of two-dimensional densities of states, where two competing energetic effects are present g(E_{1},E_{2}). We show when and why care has to be exercised when obtaining the microcanonical density of states g(E_{1}+E_{2}) from g(E_{1},E_{2}).

Stochastic approximation Monte Carlo and Wang–Landau Monte Carlo applied to a continuum polymer model

We discuss Stochastic Approximation Monte Carlo (SAMC) simulations, and Wang–Landau Monte Carlo (WLMC) simulations as one form of SAMC simulations, in an application to determine the density of states of a class of continuum polymer models. WLMC has been established in the literature as a powerful tool to determine the density of states of polymer models, but it has also been established that not all versions of WLMC really converge to the desired density of states. Convergence of SAMC simulations has been established in the mathematical literature and discussing WLMC as a special case of SAMC brings a clearer perspective to the properties of WLMC. On the other hand, practical convergence of SAMC simulations with a fixed simulation effort needs to be established for given physical problems and, for practical applications, the relative efficiency and accuracy of the two approaches need to be compared.

An Overview of Stochastic Approximation Monte Carlo

During the past few decades, Markov chain Monte Carlo (MCMC) has been widely used in Bayesian statistical inference and scientific computing. Its successes have proven it to be a very powerful and typically unique computational tool for analyzing data of complex structures. However, conventional MCMC algorithms often suffer from the local trap problem which renders the simulation ineffective. This paper provides an overview of the theory, variants, and applications for stochastic approximation Monte Carlo (SAMC), an advanced MCMC algorithm that is essentially immune to the local trap problem. WIREs Comput Stat 2014, 6:240–254. doi: 10.1002/wics.1305This article is categorized under: Statistical and Graphical Methods of Data Analysis > Markov Chain Monte Carlo (MCMC)

Population SAMC vs SAMC: Convergence and Applications to Gene Selection Problems

The Bayesian model selection approach has been adopted by more and more people when analyzing a large data. However, it is known that the reversible jump MCMC (RJMCMC) algorithm, which is perhaps the most popular MCMC algorithm for Bayesian model selection, is prone to get trapped into local modes when the model space is complex. The stochastic approximation Monte Carlo (SAMC) algorithm essentially overcomes the local trap problem suffered by conventional MCMC algorithms by introducing a self-adjusting mechanism based on the past samples. In this paper, we propose a population SAMC (Pop-SAMC) algorithm, which works on a population of SAMC chains and can make use of crossover operators from genetic algorithms to further improve its efficiency. Under mild conditions, we show the convergence of this algorithm. Comparing to the single chain SAMC algorithm, Pop-SAMC provides a more efficient self-adjusting mechanism and thus can converge faster. The effectiveness of Pop-SAMC for Bayesian model selection problems is examined through a change-point identification problem and a gene selection problem. The numerical results indicate that Pop-SAMC significantly outperforms both the single chain SAMC and RJMCMC.

A Bayesian structural-change analysis via the stochastic approximation Monte Carlo and Gibbs sampler

In this article, we propose a Bayesian approach to estimate the multiple structural change-points in a level and the trend when the number of change-points is unknown. Our formulation of the structural-change model involves a binary discrete variable that indicates the structural change. The determination of the number and the form of structural changes are considered as a model selection issue in Bayesian structural-change analysis. We apply an advanced Monte Carlo algorithm, the stochastic approximation Monte Carlo (SAMC) algorithm, to this structural-change model selection issue. SAMC effectively functions for the complex structural-change model estimation, since it prevents entrapment in local posterior mode. The estimation of the model parameters in each regime is made using the Gibbs sampler after each change-point is detected. The performance of our proposed method has been investigated on simulated and real data sets, a long time series of US real gross domestic product, US uses of force between 1870 and 1994 and 1-year time series of temperature in Seoul, South Korea.

Abrupt Motion Tracking Via Intensively Adaptive Markov-Chain Monte Carlo Sampling

The robust tracking of abrupt motion is a challenging task in computer vision due to its large motion uncertainty. While various particle filters and conventional Markov-chain Monte Carlo (MCMC) methods have been proposed for visual tracking, these methods often suffer from the well-known local-trap problem or from poor convergence rate. In this paper, we propose a novel sampling-based tracking scheme for the abrupt motion problem in the Bayesian filtering framework. To effectively handle the local-trap problem, we first introduce the stochastic approximation Monte Carlo (SAMC) sampling method into the Bayesian filter tracking framework, in which the filtering distribution is adaptively estimated as the sampling proceeds, and thus, a good approximation to the target distribution is achieved. In addition, we propose a new MCMC sampler with intensive adaptation to further improve the sampling efficiency, which combines a density-grid-based predictive model with the SAMC sampling, to give a proposal adaptation scheme. The proposed method is effective and computationally efficient in addressing the abrupt motion problem. We compare our approach with several alternative tracking algorithms, and extensive experimental results are presented to demonstrate the effectiveness and the efficiency of the proposed method in dealing with various types of abrupt motions.

Simultaneous measurement of thermal properties by thermal probe using stochastic approximation method

A novel thermal probe method is proposed for the simultaneous measurement of the thermal properties by the Monte Carlo stochastic approximation method. In this method, thermal capacity of probe and thermal contact resistance between probe and sample are considered. An experimental system is set up with the method to validate the measurement accuracy of the method. The thermal properties of several liquid samples as well as solid samples are measured. The results show that: (1) the thermal conductivity and the volumetric heat capacity can be measured with an error of less than 1.2% and 3% respectively, therefore, the measurement accuracy by the method is much higher than the conventional method and (2) the thermal contact resistance has a great effect on thermal conductivity for solid sample, while little influence on thermal conductivity for liquid sample and volumetric heat capacity.

Longitudinal functional principal component modelling via Stochastic Approximation Monte Carlo

The authors consider the analysis of hierarchical longitudinal functional data based upon a functional principal components approach. In contrast to standard frequentist approaches to selecting the number of principal components, the authors do model averaging using a Bayesian formulation. A relatively straightforward reversible jump Markov Chain Monte Carlo formulation has poor mixing properties and in simulated data often becomes trapped at the wrong number of principal components. In order to overcome this, the authors show how to apply Stochastic Approximation Monte Carlo (SAMC) to this problem, a method that has the potential to explore the entire space and does not become trapped in local extrema. The combination of reversible jump methods and SAMC in hierarchical longitudinal functional data is simplified by a polar coordinate representation of the principal components. The approach is easy to implement and does well in simulated data in determining the distribution of the number of principal components, and in terms of its frequentist estimation properties. Empirical applications are also presented.

Improving SAMC using smoothing methods: Theory and applications to Bayesian model selection problems

Stochastic approximation Monte Carlo (SAMC) has recently been proposed by Liang, Liu and Carroll [J. Amer. Statist. Assoc. 102 (2007) 305–320] as a general simulation and optimization algorithm. In this paper, we propose to improve its convergence using smoothing methods and discuss the application of the new algorithm to Bayesian model selection problems. The new algorithm is tested through a change-point identification example. The numerical results indicate that the new algorithm can outperform SAMC and reversible jump MCMC significantly for the model selection problems. The new algorithm represents a general form of the stochastic approximation Markov chain Monte Carlo algorithm. It allows multiple samples to be generated at each iteration, and a bias term to be included in the parameter updating step. A rigorous proof for the convergence of the general algorithm is established under verifiable conditions. This paper also provides a framework on how to improve efficiency of Monte Carlo simulations by incorporating some nonparametric techniques.

Multiple change-point detection of multivariate mean vectors with the Bayesian approach

Bayesian multiple change-point models are proposed for multivariate means. The models require that the data be from a multivariate normal distribution with a truncated Poisson prior for the number of change-points and conjugate priors for the distributional parameters. We apply the stochastic approximation Monte Carlo (SAMC) algorithm to the multiple change-point detection problems. Numerical results show that SAMC makes a significant improvement over RJMCMC for complex Bayesian model selection problems in change-point estimation.

Multiple Change-Point Estimation of Air Pollution Mean Vectors

The Bayesian multiple change-point estimation has been applied to the daily means of ozone and PM10 data in Seoul for the period 1999. We focus on the detection of multiple change-points in the ozone and PM10 bivariate vectors by evaluating the posterior probabilities and Bayesian information criterion(BIC) using the stochastic approximation Monte Carlo(SAMC) algorithm. The result gives 5 change-points of mean vectors of ozone and PM10, which are related with the seasonal characteristics.

On the use of stochastic approximation Monte Carlo for Monte Carlo integration

The stochastic approximation Monte Carlo (SAMC) algorithm has recently been proposed as a dynamic optimization algorithm in the literature. In this paper, we show in theory that the samples generated by SAMC can be used for Monte Carlo integration via a dynamically weighted estimator by calling some results from the literature of nonhomogeneous Markov chains. Our numerical results indicate that SAMC can yield significant savings over conventional Monte Carlo algorithms, such as the Metropolis-Hastings algorithm, for the problems for which the energy landscape is rugged.

Learning Bayesian networks for discrete data

Bayesian networks have received much attention in the recent literature. In this article, we propose an approach to learn Bayesian networks using the stochastic approximation Monte Carlo (SAMC) algorithm. Our approach has two nice features. Firstly, it possesses the self-adjusting mechanism and thus avoids essentially the local-trap problem suffered by conventional MCMC simulation-based approaches in learning Bayesian networks. Secondly, it falls into the class of dynamic importance sampling algorithms; the network features can be inferred by dynamically weighted averaging the samples generated in the learning process, and the resulting estimates can have much lower variation than the single model-based estimates. The numerical results indicate that our approach can mix much faster over the space of Bayesian networks than the conventional MCMC simulation-based approaches.