Class Of Markov Chains Research Articles

Bayesian multi-way balanced nested MANOVA models with random effects and a large number of the main factor levels

This article considers the balanced nested multi-way multivariate analysis of variance (MANOVA) models with random effects and a large number of main factor levels under certain prior assumptions. Two different parametrizations for the MANOVA models with random effects and the corresponding explicit asymptotics are established. The asymptotic approximations are then compared with those obtained from the classical large-sample approximation and Markov chain Monte Carlo method via a balanced nested two-way MANOVA model with random effects. Simulation results demonstrate that our approach is superior to the classical approximation method on estimating the posterior standard deviations of variance component parameters.

Toward sampling from undirected probabilistic graphical models using a D-Wave quantum annealer

A D-Wave quantum annealer (QA) having a 2048 qubit lattice, with no missing qubits and couplings, allowed embedding of a complete graph of a restricted Boltzmann machine (RBM). A handwritten digit OptDigits dataset having 8 × 7 pixels of visible units was used to train the RBM using classical contrastive divergence. Embedding of the classically trained RBM into the D-Wave lattice was used to demonstrate that the QA offers a high-efficiency alternative to the classical Markov chain Monte Carlo (MCMC) for reconstructing missing labels of the test images as well as a generative model on a classically trained RBM. At any training iteration, the D-Wave-based classification had classification error less than half of that in MCMC. The main goal of this study was to investigate the quality of the QA sample from the RBM model probability distribution and compare it to a classical MCMC during the Gibbs sampling traditionally used in RBM training. For the OptDigits dataset, the states in the D-Wave sample belonged to about twice as many local valleys (LVs) as in the MCMC sample. All the lowest energy (the highest joint probability), local minima in the MCMC sample were also found by the D-Wave. The D-Wave missed many of the higher-energy LVs, while also finding many “new” LVs consistently missed by the MCMC. It was established that the “new” LVs that the D-Wave finds are important for the model distribution in terms of the energy of the corresponding local minima, the width of the LVs and the height of the escape barrier.

Combining discrete cosine transform and convolutional neural networks to speed up the Hamiltonian Monte Carlo inversion of pre‐stack seismic data

ABSTRACTMarkov chain Monte Carlo algorithms are commonly employed for accurate uncertainty appraisals in non‐linear inverse problems. The downside of these algorithms is the considerable number of samples needed to achieve reliable posterior estimations, especially in high‐dimensional model spaces. To overcome this issue, the Hamiltonian Monte Carlo algorithm has recently been introduced to solve geophysical inversions. Different from classical Markov chain Monte Carlo algorithms, this approach exploits the derivative information of the target posterior probability density to guide the sampling of the model space. However, its main downside is the computational cost for the derivative computation (i.e. the computation of the Jacobian matrix around each sampled model). Possible strategies to mitigate this issue are the reduction of the dimensionality of the model space and/or the use of efficient methods to compute the gradient of the target density. Here we focus the attention to the estimation of elastic properties (P‐, S‐wave velocities and density) from pre‐stack data through a non‐linear amplitude versus angle inversion in which the Hamiltonian Monte Carlo algorithm is used to sample the posterior probability. To decrease the computational cost of the inversion procedure, we employ the discrete cosine transform to reparametrize the model space, and we train a convolutional neural network to predict the Jacobian matrix around each sampled model. The training data set for the network is also parametrized in the discrete cosine transform space, thus allowing for a reduction of the number of parameters to be optimized during the learning phase. Once trained the network can be used to compute the Jacobian matrix associated with each sampled model in real time. The outcomes of the proposed approach are compared and validated with the predictions of Hamiltonian Monte Carlo inversions in which a quite computationally expensive, but accurate finite‐difference scheme is used to compute the Jacobian matrix and with those obtained by replacing the Jacobian with a matrix operator derived from a linear approximation of the Zoeppritz equations. Synthetic and field inversion experiments demonstrate that the proposed approach dramatically reduces the cost of the Hamiltonian Monte Carlo inversion while preserving an accurate and efficient sampling of the posterior probability.

Verification of the IEC 61508 PFH formula for 2oo3 configuration using Markov chains and Petri nets

PurposeThe purpose of this paper is to check the consistency of the IEC 61508 standard formula related to the average failure frequency (PFH: the probability of dangerous failure per hour) for a commonly used safety instrumented system (SIS) architecture in the process industry: 2-out-of-3 voting (2oo3), also known as Triple Modular Redundancy (TMR).Design/methodology/approachIEC 61508 standard provided PFH formulas for different SIS architectures, without explanations, assuming that the SIS puts the equipment under control into a safe state on the detection of dangerous failure. This assumption renders the use of classical reliability approaches such as fault trees and reliability block diagrams impractical for PFH calculation. That said, the consistency verification was performed thanks to a dynamic and flexible reliability approach, namely Markov chains following these steps: (1) developing the multi-phase Markov chains (MPMC) model for 2oo3 configuration, (2) deducing the related classical Markov chains (CMC) model and (3) deriving a new PFH formula for the 2oo3 architecture based on the CMC model and thoroughly comparing it to that given in the IEC 61508. Moreover, 2oo3 architecture has been modeled through Petri nets for numerical comparison purposes. That comparison has been carried out between the numerical results obtained from IEC 61508 formula, the newly derived formula, Markov chains and Petri nets models.FindingsThe newly obtained formula for 2oo3 configuration contains extra terms compared with the IEC 61508 one. Therefore, this latter formula induces an underestimated PFH results, which is dangerous from a safety point of view. This fact was corroborated by the numerical comparison.Research limitations/implicationsThis paper does not consider the different configurations given in IEC 61508.Originality/valueIn our knowledge, no verification works have been conducted before on the IEC 61508 PFH formulas with shutdown capability. Therefore, the nonaccuracy of the PFH formula related to the 2oo3 has not been stated before. This paper proposes a new and more accurate formula.

MCMC methods applied to the reconstruction of the autumn 2017 Ruthenium-106 atmospheric contamination source

Abstract In autumn 2017, small amounts of Ruthenium-106 of unknown origin were detected in Europe by several independent monitoring networks. To study the dispersion of this radionuclide, inverse modelling methods are applied to retrieve the location, time, duration and magnitude of the source. The inverse problem is solved within the Bayesian framework to yield a full reconstruction of the uncertainties. We first develop a classical Markov chain Monte Carlo (MCMC) method - the Metropolis Hastings (MH) algorithm - to reconstruct the source. However, the algorithm fails to converge in acceptable time because of local minima of the cost function, whose existence is due to a poor distribution of the observations. To overcome this obstacle, the parallel tempering algorithm, an enhanced MCMC method, is assessed and applied to the Ruthenium dispersion event. The convergence of the algorithm is studied and keys to implement and accelerate it are provided. The probability distribution functions of the variables associated with the source and the observation errors are obtained: they point to an area in south Ural and a total release of several hundreds of TBq on the 26 th of September 2017. The results are compared and proven to be consistent with other estimates. Moreover, the method converges fast enough to make it suitable for operational use.

Accelerating Metropolis-within-Gibbs sampler with localized computations of differential equations

Inverse problem is ubiquitous in science and engineering, and Bayesian methodologies are often used to infer the underlying parameters. For high dimensional temporal-spatial models, classical Markov chain Monte Carlo (MCMC) methods are often slow to converge, and it is necessary to apply Metropolis-within-Gibbs (MwG) sampling on parameter blocks. However, the computation cost of each MwG iteration is typically $O(n^2)$, where $n$ is the model dimension. This can be too expensive in practice. This paper introduces a new reduced computation method to bring down the computation cost to $O(n)$, for the inverse initial value problem of a stochastic differential equation (SDE) with local interactions. The key observation is that each MwG proposal is only different from the original iterate at one parameter block, and this difference will only propagate within a local domain in the SDE computations. Therefore we can approximate the global SDE computation with a surrogate updated only within the local domain for reduced computation cost. Both theoretically and numerically, we show that the approximation errors can be controlled by the local domain size. We discuss how to implement the local computation scheme using Euler--Maruyama and 4th order Runge--Kutta methods. We numerically demonstrate the performance of the proposed method with the Lorenz 96 model and a linear stochastic flow model.

A paradox for expected hitting times

We prove a counterintuitive result concerning the expected hitting/absorption time for a class of Markov chains. The “paradox” already shows itself in the following elementary example that is suitable for undergraduate teaching: Batman and the Joker perform independent discrete-time random walks on the vertices of a square until they meet, starting from opposite vertices. Batman always moves (and clockwise and anticlockwise steps are equally likely), while the Joker remains still on any given step with probability On average the Joker survives for twice as long by staying still with arbitrarily small but positive probability (in the limit as ) than by always moving (when q = 0).

Distributed algorithm for AP association with random arrivals and departures of users

Here, the authors study the novel problem of optimising access point (AP) association by maximising the network throughput, subject to the degree bound of AP. The formulated problem is a combinatorial optimisation. They resort to the Markov Chain approximation technique to design a distributed algorithm. They first approximate their optimal objective via Log-Sum-Exp function. Thereafter, they construct a special class of Markov Chain with steady-state distribution specify to their problem to yield a distributed solution. Furthermore, they extend the static problem setting to a dynamic environment where the users can randomly leave or join the system. Their proposed algorithm has provable performance, achieving an approximation gap of ( 1 / η ) log ⁡ | F | . It is simple and can be implemented in a distributed manner. Their extensive simulation results show that the proposed algorithm can converge very fast, and achieve a close-to-optimal performance with a guaranteed loss bound.

On obtaining sharp bounds of the rate of convergence for a class of continuous-time Markov chains

We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of convergence for such class of Markov chains.

A distribution-free EWMA control chart for monitoring time-between-events-and-amplitude data

ABSTRACT Many control charts have been developed for the simultaneous monitoring of the time interval T between successive occurrences of an event E and its magnitude X. All these TBEA (Time Between Events and Amplitude) control charts assume a known distribution for the random variables T and X. But, in practice, as it is rather difficult to know their actual distributions, proposing a distribution free approach could be a way to overcome this ‘distribution choice’ dilemma. For this reason, we propose in this paper a distribution free upper-sided EWMA (Exponentially Weighted Moving Average) type control chart, for simultaneously monitoring the time interval T and the magnitude X of an event. In order to investigate the performance of this control chart and obtain its run length properties, we also develop a specific method called ‘continuousify’ which, coupled with a classical Markov chain technique, allows to obtain reliable and replicable results. A numerical comparison shows that our distribution-free EWMA TBEA chart performs as the parametric Shewhart TBEA chart, but without the need to pre-specify any distribution. An illustrative example obtained from a French forest fire database is also provided to show the implementation of the proposed EWMA TBEA control chart.

Quantum Markov Chains: Recurrence, Schur Functions and Splitting Rules

In this work we study the recurrence problem for quantum Markov chains, which are quantum versions of classical Markov chains introduced by S. Gudder and described in terms of completely positive maps. A notion of monitored recurrence for quantum Markov chains is examined in association with Schur functions, which codify information on the first return to some given state or subspace. Such objects possess important factorization and decomposition properties which allow us to obtain probabilistic results based solely on those parts of the graph where the dynamics takes place, the so-called splitting rules. These rules also yield an alternative to the folding trick to transform a doubly infinite system into a semi-infinite one which doubles the number of internal degrees of freedom. The generalization of Schur functions --so-called FR-functions-- to the general context of closed operators in Banach spaces is the key for the present applications to open quantum systems. An important class of examples included in this setting are the open quantum random walks, as described by S. Attal et al., but we will state results in terms of general completely positive trace preserving maps. We also take the opportunity to discuss basic results on recurrence of finite dimensional iterated quantum channels and quantum versions of Kac's Lemma, in close association with recent results on the subject.

Quantum Markov Chains Associated with Open Quantum Random Walks

In this paper we construct (nonhomogeneous) quantum Markov chains associated with open quantum random walks. The quantum Markov chain, like the classical Markov chain, is a fundamental tool for the investigation of the basic properties such as reducibility/irreducibility, recurrence/transience, accessibility, ergodicity, etc, of the underlying dynamics. Here we focus on the discussion of the reducibility and irreducibility of open quantum random walks via the corresponding quantum Markov chains. Particularly we show that the concept of reducibility/irreducibility of open quantum random walks in this approach is equivalent to the one previously done by Carbone and Pautrat. We provide with some examples. We will see also that the classical Markov chains can be reconstructed as quantum Markov chains.

Quantum Markov semigroups for continuous-time open quantum random walk

Open quantum random walks in Attal et al. (Phys Lett A 376:1545–1548, 2012) are quantum generalizations of classical Markov chains. In this paper, we firstly consider a problem to construct a quantum Markov semigroup (QMS) $${\mathcal {T}}_{t}$$ directly from a continuous-time open quantum random walk (CTOQRW) developed in Pellegrini (J Stat Phys 154(3):838–865, 2014) and characterize its generator as well as its predual semigroup $${\mathcal {T}}_{*t}$$ . We show that this type of CTOQRWs brings about a large field of exploration for the behavior of open quantum system. Finally, we discuss some properties of this type of QMS constructed from CTOQRWs.

Unraveling the Large Deviation Statistics of Markovian Open Quantum Systems.

We analyze dynamical large deviations of quantum trajectories in Markovian open quantum systems in their full generality. We derive a quantum level-2.5 large deviation principle for these systems, which describes the joint fluctuations of time-averaged quantum jump rates and of the time-averaged quantum state for long times. Like its level-2.5 counterpart for classical continuous-time Markov chains (which it contains as a special case), this description is both explicit and complete, as the statistics of arbitrary time-extensive dynamical observables can be obtained by contraction from the explicit level-2.5 rate functional we derive. Our approach uses an unraveled representation of the quantum dynamics which allows these statistics to be obtained by analyzing a classical stochastic process in the space of pure states. For quantum reset processes we show that the unraveled dynamics is semi-Markovian and derive bounds on the asymptotic variance of the number of quantum jumps which generalize classical thermodynamic uncertainty relations. We finish by discussing how our level-2.5 approach can be used to study large deviations of nonlinear functions of the state, such as measures of entanglement.

Moments of the Count of a Regular Expression in a Heterogeneous Random Sequence

We focus here on the distribution of the random count N of a regular expression in a multi-state random sequence generated by a heterogenous Markov source. We first briefly recall how classical Markov chain embedding techniques allow reducing the problem to the count of specific transitions in a (heterogenous) order 1 Markov chain over a deterministic finite automaton state space. From this result we derive the expression of both the mgf/pgf of N as well as the factorial and non-factorial moments of N. We then introduce the notion of evidence-based constraints in this context. Following the classical forward/backward algorithm in hidden Markov models, we provide explicit recursions allowing to compute the mgf/pgf of N under the evidence constraint. All the results presented are illustrated with a toy example.

A unified stability theory for classical and monotone Markov chains

AbstractIn this paper we integrate two strands of the literature on stability of general state Markov chains: conventional, total-variation-based results and more recent order-theoretic results. First we introduce a complete metric over Borel probability measures based on ‘partial’ stochastic dominance. We then show that many conventional results framed in the setting of total variation distance have natural generalizations to the partially ordered setting when this metric is adopted.

Simulation of quantum walks and fast mixing with classical processes

We compare discrete-time quantum walks on graphs to their natural classical equivalents, which we argue are lifted Markov chains (LMCs), that is, classical Markov chains with added memory. We show that LMCs can simulate the mixing behavior of any quantum walk, under a commonly satisfied invariance condition. This allows us to answer an open question on how the graph topology ultimately bounds a quantum walk's mixing performance, and that of any stochastic local evolution. The results highlight that speedups in mixing and transport phenomena are not necessarily diagnostic of quantum effects, although superdiffusive spreading is more prominent with quantum walks. The general simulating LMC construction may lead to large memory, yet we show that for the main graphs under study (i.e., lattices) this memory can be brought down to the same size employed in the quantum walks proposed in the literature.

Frequentist Consistency of Variational Bayes

ABSTRACTA key challenge for modern Bayesian statistics is how to perform scalable inference of posterior distributions. To address this challenge, variational Bayes (VB) methods have emerged as a popular alternative to the classical Markov chain Monte Carlo (MCMC) methods. VB methods tend to be faster while achieving comparable predictive performance. However, there are few theoretical results around VB. In this article, we establish frequentist consistency and asymptotic normality of VB methods. Specifically, we connect VB methods to point estimates based on variational approximations, called frequentist variational approximations, and we use the connection to prove a variational Bernstein–von Mises theorem. The theorem leverages the theoretical characterizations of frequentist variational approximations to understand asymptotic properties of VB. In summary, we prove that (1) the VB posterior converges to the Kullback–Leibler (KL) minimizer of a normal distribution, centered at the truth and (2) the corresponding variational expectation of the parameter is consistent and asymptotically normal. As applications of the theorem, we derive asymptotic properties of VB posteriors in Bayesian mixture models, Bayesian generalized linear mixed models, and Bayesian stochastic block models. We conduct a simulation study to illustrate these theoretical results. Supplementary materials for this article are available online.

Optimal Detection and Error Exponents for Hidden Semi-Markov Models

We study detection of random signals corrupted by noise that over time switch their values (states) between a finite set of possible values, where the switchings occur at unknown points in time. We model such signals as hidden semi-Markov signals, which generalize classical Markov chains by introducing explicit (possibly nongeometric) distribution for the time spent in each state. Assuming two possible signal states and Gaussian noise, we derive optimal likelihood ratio test and show that it has a computationally tractable form of a matrix product, with the number of matrices involved in the product being the number of process observations. The product matrices are independent and identically distributed, constructed by a simple measurement modulation of the sparse semi-Markov model transition matrix that we define in the paper. Using this result, we show that the Neyman–Pearson error exponent is equal to the top Lyapunov exponent for the corresponding random matrices. Using theory of large deviations, we derive a lower bound on the error exponent. Finally, we show that this bound is tight by means of numerical simulations.

Regions-of-interest discovering and predicting in smartphone environments

Location-Based Services (LBSs) provide users especially in smartphones with information and services based on their location and interests. Predicting people’s next location could help in improving the quality of such services and, in turn, boosting people’s confidence on those services. Any successful LBS algorithm or model should target three major goals: Location prediction accuracy, high throughput or fast response and efficiency in terms of utilizing smartphone resources. This paper proposes a new approach to discover and predict people’s next location based on their mobility patterns, while being computationally efficient. The approach starts by discovering Regions-of-Interest (RoIs) in people’s historical trajectories (which denote the locations where users have been previously, frequently). A new model based on Markov Chain (MC) is proposed to overcome the drawback of classical MC. Our proposed model considers both space and time contexts where the space represents a specific location that has been visited by a user while the time represents the location visiting time. We show how classical MC model can be extended to include movement times and how time will improve prediction accuracy. One unique finding in our research is related to the value of integrating users’ mobility location/space with time context. In particular, time context is formulated in a way to add extra information to the space context. For better abstraction during building the model, a general transformation function is used to transform the n-order MC into first order. Results show that the proposed approach provides significant improvements in predicting people’s next location compared with the state-of-the-art models when applied on two large real-life datasets, GeoLife and Gowalla.