Auxiliary Markov Research Articles

Hamiltonian Switch Metropolis Monte Carlo Simulations for Improved Conformational Sampling of Intrinsically Disordered Regions Tethered to Ordered Domains of Proteins

There is growing interest in the topic of intrinsically disordered proteins (IDPs). Atomistic Metropolis Monte Carlo (MMC) simulations based on novel implicit solvation models have yielded useful insights regarding sequence-ensemble relationships for IDPs modeled as autonomous units. However, a majority of naturally occurring IDPs are tethered to ordered domains. Tethering introduces additional energy scales and this creates the challenge of broken ergodicity for standard MMC sampling or molecular dynamics that cannot be readily alleviated by using generalized tempering methods. We have designed, deployed, and tested our adaptation of the Nested Markov Chain Monte Carlo sampling algorithm. We refer to our adaptation as Hamiltonian Switch Metropolis Monte Carlo (HS-MMC) sampling. In this method, transitions out of energetic traps are enabled by the introduction of an auxiliary Markov chain that draws conformations for the disordered region from a Boltzmann distribution that is governed by an alternative potential function that only includes short-range steric repulsions and conformational restraints on the ordered domain. We show using multiple, independent runs that the HS-MMC method yields conformational distributions that have similar and reproducible statistical properties, which is in direct contrast to standard MMC for equivalent amounts of sampling. The method is efficient and can be deployed for simulations of a range of biologically relevant disordered regions that are tethered to ordered domains.

Response of dynamic systems to renewal impulse processes: Generating equation for moments based on the integro-differential Chapman–Kolmogorov equations

In the present paper the method is developed for the derivation of differential equations for statistical moments of the state vector (response) of a non-linear dynamic system subjected to a random train of impulses. The arrival times of the impulses are assumed to be driven by a non-Poisson counting process. The state vector of the dynamic system is then a non-Markov process and no method is directly available for the derivation of the equations for response moments. The original non-Markov problem is converted into a Markov one by recasting the excitation process with the aid of an auxiliary, pure-jump stochastic process characterized by a Markov chain. Hence the conversion is carried out at the expense of augmentation of the state space of the dynamic system by auxiliary Markov states. For the augmented problem the sets of forward and backward integro-differential Chapman–Kolmogorov equations are formulated. The general, generating equation for moments is obtained with the aid of the forward and backward integro-differential Chapman–Kolmogorov operators. The developed method is illustrated by the examples of several renewal impulse processes.

A Monte Carlo Metropolis-Hastings Algorithm for Sampling from Distributions with Intractable Normalizing Constants

Simulating from distributions with intractable normalizing constants has been a long-standing problem in machine learning. In this letter, we propose a new algorithm, the Monte Carlo Metropolis-Hastings (MCMH) algorithm, for tackling this problem. The MCMH algorithm is a Monte Carlo version of the Metropolis-Hastings algorithm. It replaces the unknown normalizing constant ratio by a Monte Carlo estimate in simulations, while still converges, as shown in the letter, to the desired target distribution under mild conditions. The MCMH algorithm is illustrated with spatial autologistic models and exponential random graph models. Unlike other auxiliary variable Markov chain Monte Carlo (MCMC) algorithms, such as the Møller and exchange algorithms, the MCMH algorithm avoids the requirement for perfect sampling, and thus can be applied to many statistical models for which perfect sampling is not available or very expensive. The MCMH algorithm can also be applied to Bayesian inference for random effect models and missing data problems that involve simulations from a distribution with intractable integrals.

Almost sure stability and instability for switching-jump-diffusion systems with state-dependent switching

This work develops almost sure stability criteria for switching-jump-diffusion systems with state-dependent switching. By means of introducing certain auxiliary Markov chains and constructing certain order-preserving couplings, upper and lower “stability envelops” are constructed, which lead to systems with “upper and lower” approximating Markov chains. Using these approximations, sufficient conditions that are relatively easily verifiable for the almost sure stability and instability are obtained. When the jump process is missing, it is demonstrated that the techniques work equally well and provide a way to analyze the corresponding switching diffusion systems with x-dependent switching. In addition, stochastic stabilization and destabilization are examined. Moreover, illustrative examples are provided for demonstration.

Use of SAMC for Bayesian analysis of statistical models with intractable normalizing constants

Statistical inference for the models with intractable normalizing constants has attracted much attention. During the past two decades, various approximation- or simulation-based methods have been proposed for the problem, such as the Monte Carlo maximum likelihood method and the auxiliary variable Markov chain Monte Carlo methods. The Bayesian stochastic approximation Monte Carlo algorithm specifically addresses this problem: It works by sampling from a sequence of approximate distributions with their average converging to the target posterior distribution, where the approximate distributions can be achieved using the stochastic approximation Monte Carlo algorithm. A strong law of large numbers is established for the Bayesian stochastic approximation Monte Carlo estimator under mild conditions. Compared to the Monte Carlo maximum likelihood method, the Bayesian stochastic approximation Monte Carlo algorithm is more robust to the initial guess of model parameters. Compared to the auxiliary variable MCMC methods, the Bayesian stochastic approximation Monte Carlo algorithm avoids the requirement for perfect samples, and thus can be applied to many models for which perfect sampling is not available or very expensive. The Bayesian stochastic approximation Monte Carlo algorithm also provides a general framework for approximate Bayesian analysis.

Stability of a Markov-modulated Markov Chain, with Application to a Wireless Network Governed by two Protocols

We consider a discrete-time Markov chain (Xt,Yt), t = 0, 1, 2,…, where the X-component forms a Markov chain itself. Assume that (Xt) is Harris-ergodic and consider an auxiliary Markov chain [Formula: see text] whose transition probabilities are the averages of transition probabilities of the Y-component of the (X, Y)-chain, where the averaging is weighted by the stationary distribution of the X-component. We first provide natural conditions in terms of test functions ensuring that the [Formula: see text]-chain is positive recurrent and then prove that these conditions are also sufficient for positive recurrence of the original chain (Xt, Yt). The we prove a “multi-dimensional” extension of the result obtained. In the second part of the paper, we apply our results to two versions of a multi-access wireless model governed by two randomised protocols.

Stability of a Markov-modulated Markov chain, with application to a wireless network governed by two protocols

We consider a discrete-time Markov chain (Xt,Yt), t = 0, 1, 2,…, where the X-component forms a Markov chain itself. Assume that (Xt) is Harris-ergodic and consider an auxiliary Markov chain {Yt} whose transition probabilities are the averages of transition probabilities of the Y-component of the (X, Y)-chain, where the averaging is weighted by the stationary distribution of the X-component. We first provide natural conditions in terms of test functions ensuring that the Y-chain is positive recurrent and then prove that these conditions are also sufficient for positive recurrence of the original chain (Xt, Yt). The we prove a “multi-dimensional” extension of the result obtained. In the second part of the paper, we apply our results to two versions of a multi-access wireless model governed by two randomised protocols.

Excited Brownian motions as limits of excited random walks

We obtain the convergence in law of a sequence of excited (also called cookies) random walks toward an excited Brownian motion. This last process is a continuous semi-martingale whose drift is a function, say φ, of its local time. It was introduced by Norris, Rogers and Williams as a simplified version of Brownian polymers, and then recently further studied by the authors. To get our results we need to renormalize together the sequence of cookies, the time and the space in a convenient way. The proof follows a general approach already taken by Toth and his coauthors in multiple occasions, which goes through Ray-Knight type results. Namely we first prove, when φ is bounded and Lipschitz, that the convergence holds at the level of the local time processes. This is done via a careful study of the transition kernel of an auxiliary Markov chain which describes the local time at a given level. Then we prove a tightness result and deduce the convergence at the level of the full processes.

Asymptotic analysis for the generalized Langevin equation

Various qualitative properties of solutions to the generalized Langevin equation (GLE) in a periodic or a confining potential are studied in this paper. We consider a class of quasi-Markovian GLEs, similar to the model that was introduced in Eckmann J-P et al 1999 Commun. Math. Phys. 201 657–97. Ergodicity, exponentially fast convergence to equilibrium, short time asymptotics, a homogenization theorem (invariance principle) and the white noise limit are studied. Our proofs are based on a careful analysis of a hypoelliptic operator which is the generator of an auxiliary Markov process. Systematic use of the recently developed theory of hypocoercivity (Villani C 2009 Mem. Am. Math. Soc. 202 iv, 141) is made.

Waiting time distribution of generalized later patterns

In this paper the concept of later waiting time distributions for patterns in multi-state trials is generalized to cover a collection of compound patterns that must all be counted pattern-specific numbers of times, and a practical method is given to compute the generalized distribution. The solution given applies to overlapping counting and two types of non-overlapping counting, and the underlying sequences are assumed to be Markovian of a general order. Patterns are allowed to be weighted so that an occurrence is counted multiple times, and patterns may be completely included in longer patterns. Probabilities are computed through an auxiliary Markov chain. As the state space associated with the auxiliary chain can be quite large if its setup is handled in a naïve fashion, an algorithm is given for generating a “minimal” state space that leaves out states that can never be reached. For the case of overlapping counting, a formula that relates probabilities for intersections of events to probabilities for unions of subsets of the events is also used, so that the distribution is also computed in terms of probabilities for competing patterns. A detailed example is given to illustrate the methodology.

Application of auxiliary Markov chains to start-up demonstration tests

We use auxiliary Markov chains to derive probabilistic results for five types of start-up demonstration tests, with start-ups that are Markovian of a general order. Four of the tests are based on consecutive (or total) successful start-ups and consecutive (or total) failures; the fifth has two rejection criteria. For each test type, we obtain the probability of the test ending with acceptance of the unit, the probability distribution and moments of the number of start-ups in the test, the probability of acceptance (or rejection) of the equipment in a specified number of trials, and the conditional distribution of the number of start-ups in the test given that the unit is accepted or rejected. Numerical examples are given. Though the results are for these specific types of start-up demonstration tests, the method of derivation may be used for tests with other stopping criteria, and in other situations as well.

Distributions associated with general runs and patterns in hidden Markov models

This paper gives a method for computing distributions associated with patterns in the state sequence of a hidden Markov model, conditional on observing all or part of the observation sequence. Probabilities are computed for very general classes of patterns (competing patterns and generalized later patterns), and thus, the theory includes as special cases results for a large class of problems that have wide application. The unobserved state sequence is assumed to be Markovian with a general order of dependence. An auxiliary Markov chain is associated with the state sequence and is used to simplify the computations. Two examples are given to illustrate the use of the methodology. Whereas the first application is more to illustrate the basic steps in applying the theory, the second is a more detailed application to DNA sequences, and shows that the methods can be adapted to include restrictions related to biological knowledge.

Statistical simulation of one type of pairs of random variables with the use of fictitious jumps

Statistical simulation based on the sampling technique for a pair of random variables (T, Open image in new window ), where T ∈ [0, +∞) and Open image in new window ∈ ℜd (d ≥ 1) is considered. The simultaneous distribution of the pair is specified in the form that is common for analogous problems in various fields. It has the form Open image in new window , where f is a function and m is a measure. The first variable T is the well-known random waiting time. A simulation method for the pair (T, Open image in new window ) is constructed using a realization of an auxiliary Markov sequence of trial pairs. Applications of this method in particle transport theory and in kinetics of rarefied gases are discussed.

The mixing time for simple exclusion

We obtain a tight bound of O(L2logk) for the mixing time of the exclusion process in Zd/LZd with k≤½Ld particles. Previously the best bound, based on the log Sobolev constant determined by Yau, was not tight for small k. When dependence on the dimension d is considered, our bounds are an improvement for all k. We also get bounds for the relaxation time that are lower order in d than previous estimates: our bound of O(L2logd) improves on the earlier bound O(L2d) obtained by Quastel. Our proof is based on an auxiliary Markov chain we call the chameleon process, which may be of independent interest.

A model for supply planning under lead time uncertainty

The paper is focused on the search of the optimal values of the planned lead times for the MRP method under lead time uncertainty. The problem is to find the planned lead times, which minimize the expected backlogging and holding costs. The aim of the paper is to give a mathematical formulation of this single-level, multi-item, dynamic multi-period planning problem. For the case where lead time does not depend on the lot size and demand level is constant, an auxiliary Markov model is proposed. Some new formulations of optimal control policies are obtained using this Markov model.

OPTIMAL CONTROL OF STOCHASTIC LINEAR SYSTEM WITH DELAYED RELAY OUTPUT

We describe random disturbances as a Poisson process. After several simplifications, we construct a Markov decision process. Its solution built with the help of the dynamic programming approach gives quasi-optimal control strategy to the initial problem. The auxiliary Markov decision process is of its own interest and can be applied to e.g. some reliability problems.

Synthesis of an Optimal Control for Continuous Markov Processes by Semidefinite Quality Criteria

Optimization of control of continuous Markov processes by semidefinite quality functionals with nonadditive control cost functions is investigated. Equation for the loss function, its parameters for different integrands in the quality criterion, and its interrelation with the Fokker–Planck–Kolmogorov equation for the auxiliary noncontrollable Markov processes are determined. Examples are given.

An asymptotic formula for the Bayes risk in discriminating between two Markov chains

The problem of discriminating between two Markov chains is considered. It is assumed that the common state space of the chains is finite and all the finite dimensional distributions are mutually absolutely continuous. The Bayes risk is expressed through large deviation probabilities for sums of random variables defined on an auxiliary Markov chain. The proofs are based on a large deviation theorem recently established by Z. Szewczak.

An asymptotic formula for the Bayes risk in discriminating between two Markov chains

The problem of discriminating between two Markov chains is considered. It is assumed that the common state space of the chains is finite and all the finite dimensional distributions are mutually absolutely continuous. The Bayes risk is expressed through large deviation probabilities for sums of random variables defined on an auxiliary Markov chain. The proofs are based on a large deviation theorem recently established by Z. Szewczak.

Time-Invariance Estimating Equations

We describe a general method for deriving estimators of the parameter of a statistical model, with particular relevance to highly structured stochastic systems such as spatial random processes and 'graphical' conditional independence models. The method is based on representing the stochastic model X as the equilibrium distribution of an auxiliary Markov process Y = (Yt, t > 0) where the discrete or continuous 'time' index t is to be understood as a fictional extra dimension added to the original setting. The parameter estimate 0 is obtained by equating to zero the generator of Y applied to a suitable statistic and evaluated at the data x. This produces an unbiased estimating equation for 0. Natural special cases include maximum likelihood, the method of moments, the reduced sample estimator in survival analysis, the maximum pseudolikelihood estimator for random fields and for point processes, the Takacs-Fiksel method for point processes, 'variational' estimators for random fields and multivariate distributions, and many standard estimators in stochastic geometry. The approach has some affinity with the Stein-Chen method for distributional approximation.