Transition Kernel Research Articles

Informed sub-sampling MCMC: approximate Bayesian inference for large datasets

This paper introduces a framework for speeding up Bayesian inference conducted in presence of large datasets. We design a Markov chain whose transition kernel uses an unknown fraction of fixed size of the available data that is randomly refreshed throughout the algorithm. Inspired by the Approximate Bayesian Computation literature, the subsampling process is guided by the fidelity to the observed data, as measured by summary statistics. The resulting algorithm, Informed Sub-Sampling MCMC, is a generic and flexible approach which, contrary to existing scalable methodologies, preserves the simplicity of the Metropolis–Hastings algorithm. Even though exactness is lost, i.e the chain distribution approximates the posterior, we study and quantify theoretically this bias and show on a diverse set of examples that it yields excellent performances when the computational budget is limited. If available and cheap to compute, we show that setting the summary statistics as the maximum likelihood estimator is supported by theoretical arguments.

PyABC: distributed, likelihood-free inference.

Likelihood-free methods are often required for inference in systems biology. While approximate Bayesian computation (ABC) provides a theoretical solution, its practical application has often been challenging due to its high computational demands. To scale likelihood-free inference to computationally demanding stochastic models, we developed pyABC: a distributed and scalable ABC-Sequential Monte Carlo (ABC-SMC) framework. It implements a scalable, runtime-minimizing parallelization strategy for multi-core and distributed environments scaling to thousands of cores. The framework is accessible to non-expert users and also enables advanced users to experiment with and to custom implement many options of ABC-SMC schemes, such as acceptance threshold schedules, transition kernels and distance functions without alteration of pyABC's source code. pyABC includes a web interface to visualize ongoing and finished ABC-SMC runs and exposes an API for data querying and post-processing. pyABC is written in Python 3 and is released under a 3-clause BSD license. The source code is hosted on https://github.com/icb-dcm/pyabc and the documentation on http://pyabc.readthedocs.io. It can be installed from the Python Package Index (PyPI). Supplementary data are available at Bioinformatics online.

Simulating conditionally specified models

Expert systems routinely use conditional reasoning. Conditionally specified statistical models offer several advantages over joint models; one is that Gibbs sampling can be used to generate realizations of the model. As a result, full conditional specification for multiple imputation is gaining popularity because it is flexible and computationally straightforward. However, it would be restrictive to require that every regression/classification must involve all of the variables. Feature selection often removes some variables from the set of predictors, thus making the regression local. A mixture of full and local conditionals is referred to as a partially collapsed Gibbs sampler, which often achieves faster convergence due to reduced conditioning. However, its implementation requires choosing a correct scan order. Using an invalid scan order will bring about an incorrect transition kernel, which leads to the wrong stationary distribution. We prove a necessary and sufficient condition for Gibbs sampling to correctly sample the joint distribution. We propose an algorithm that identifies all of the valid scan orders for a given conditional model. A forward search algorithm is discussed. Checking compatibility among conditionals of different localities is also discussed.

Integral transforms of the quantum mechanical path integral: Hit function and path-averaged potential.

We introduce two integral transforms of the quantum mechanical transition kernel that represent physical information about the path integral. These transforms can be interpreted as probability distributions on particle trajectories measuring respectively the relative contribution to the path integral from paths crossing a given spatial point (the hit function) and the likelihood of values of the line integral of the potential along a path in the ensemble (the path-averaged potential).

Solutions of the average cost optimality equation for Markov decision processes with weakly continuous kernel: The fixed-point approach revisited

This paper shows the existence of lower semicontinuous solutions of the average cost optimality equation for Markov decision processes with Borel spaces, possible unbounded cost function and weakly continuous transition kernel. This is done imposing a growth condition on the cost function, a Lyapunov stability condition on the transition kernel and a set of standard compactness-continuity conditions. The solution of the average cost optimality equation is obtained by means of the Banach fixed-point theorem.

The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method

ABSTRACTMany Markov chain Monte Carlo techniques currently available rely on discrete-time reversible Markov processes whose transition kernels are variations of the Metropolis–Hastings algorithm. We explore and generalize an alternative scheme recently introduced in the physics literature (Peters and de With 2012) where the target distribution is explored using a continuous-time nonreversible piecewise-deterministic Markov process. In the Metropolis–Hastings algorithm, a trial move to a region of lower target density, equivalently of higher “energy,” than the current state can be rejected with positive probability. In this alternative approach, a particle moves along straight lines around the space and, when facing a high energy barrier, it is not rejected but its path is modified by bouncing against this barrier. By reformulating this algorithm using inhomogeneous Poisson processes, we exploit standard sampling techniques to simulate exactly this Markov process in a wide range of scenarios of interest. Additionally, when the target distribution is given by a product of factors dependent only on subsets of the state variables, such as the posterior distribution associated with a probabilistic graphical model, this method can be modified to take advantage of this structure by allowing computationally cheaper “local” bounces, which only involve the state variables associated with a factor, while the other state variables keep on evolving. In this context, by leveraging techniques from chemical kinetics, we propose several computationally efficient implementations. Experimentally, this new class of Markov chain Monte Carlo schemes compares favorably to state-of-the-art methods on various Bayesian inference tasks, including for high-dimensional models and large datasets. Supplementary materials for this article are available online.

Approximation of Discounted Minimax Markov Control Problems and Zero-Sum Markov Games Using Hausdorff and Wasserstein Distances

This paper is concerned with a minimax control problem (also known as a robust Markov decision process (MDP) or a game against nature) with general state and action spaces under the discounted cost optimality criterion. We are interested in approximating numerically the value function and an optimal strategy of this general discounted minimax control problem. To this end, we derive structural Lipschitz continuity properties of the solution of this robust MDP by imposing suitable conditions on the model, including Lipschitz continuity of the elements of the model and absolute continuity of the Markov transition kernel with respect to some probability measure $$\mu $$ . Then, we are able to provide an approximating minimax control model with finite state and action spaces, and hence computationally tractable, by combining these structural properties with a suitable discretization procedure of the state space (related to a probabilistic criterion) and the action spaces (associated to a geometric criterion). Finally, it is shown that the corresponding approximation errors for the value function and the optimal strategy can be controlled in terms of the discretization parameters. These results are also extended to a two-player zero-sum Markov game.

Multiobject Tracking for Generic Observation Model Using Labeled Random Finite Sets

This paper presents an exact Bayesian filtering solution for the multi-object tracking problem with the generic observation model. The proposed solution is designed in the labeled random finite set framework, using the product styled representation of labeled multi-object densities, with the standard multi-object transition kernel and no particular simplifying assumptions on the multi-object likelihood. Computationally tractable solutions are also devised by applying a principled approximation involving the replacement of the full multi-object density with a labeled multi-Bernoulli density that minimizes the Kullback-Leibler divergence and preserves the first-order moment. To achieve the fast performance, a dynamic grouping procedure based implementation is presented with a step-by-step algorithm. The performance of the proposed filter and its tractable implementations are verified and compared with the state-of-the-art in numerical experiments.

Martingales associated to peacocks using the curtain coupling

We consider right-continuous peacocks, that is, families of real probability measures $(\mu _t)_{t\in [0,1]}$ that are increasing in convex order. Given a sequence of time partitions we associate the sequence of martingales characterised by the fact that they are Markovian, constant on the partition intervals $[t_k,t_{k+1}[$, and such that the transition kernels at times $t_{k+1}$ are the curtain couplings of marginals $\mu _{t_k}$ and $\mu _{t_{k+1}}$. We study the limit curtain processes obtained when the mesh of the partition tends to zero and study existence, uniqueness and relevancy with respect to the original data. For any right-continuous peacock we show there exist sequences of partitions such that a limit process exists (for the finite-dimensional convergence). Under certain additional regularity assumptions, we prove that there is a unique limit curtain process and that it is a Markovian martingale. We first study by elementary methods peacocks whose marginals correspond to uniform distributions in convex order. In this case, the results and techniques complete the results and techniques used in a parallel work by Henry-Labordere, Tan and Touzi [9]. We obtain the same type of results for all limit curtain processes associated to a class of analytic discrete peacocks, i.e., the measures $\mu _t$ are finitely supported and vary analytically in $t$. Finally, we give examples of peacocks and sequences of partitions such that the limit curtain process is a non-Markovian martingale.

The argmin process of random walks, Brownian motion and Lévy processes

In this paper we investigate the argmin process of Brownian motion $B$ defined by $\alpha _t:=\sup \left \{s \in [0,1]: B_{t+s}=\inf _{u \in [0,1]}B_{t+u} \right \}$ for $t \geq 0$. The argmin process $\alpha $ is stationary, with invariant measure which is arcsine distributed. We prove that $(\alpha _t; t \geq 0)$ is a Markov process with the Feller property, and provide its transition kernel $Q_t(x,\cdot )$ for $t>0$ and $x \in [0,1]$. Similar results for the argmin process of random walks and Lévy processes are derived. We also consider Brownian extrema of a given length. We prove that these extrema form a delayed renewal process with an explicit path construction. We also give a path decomposition for Brownian motion at these extrema.

Was Poland the next Spain? Parallel analysis of regional convergence patterns after accession to the European Union

Research background: Poland and Spain share many common features resulting both from similarities of historical experience, and also cultural, political, socio-demographic factors. Both countries have a similar area, population and GDP structure. They also share historical experience related to political and economic transformation after a long period of non-democratic, centralized governments. Therefore, the experience of Spanish membership in the EU is often considered as a model for Poland.
 Purpose of the article: The purpose of this research is to perform a comparative empirical analysis of income convergence processes in Poland and Spain on a regional level. We aim to verify if and how these processes are related to one an-other (show similar paths). Special attention is paid to the periods after accession of these countries to the EU. Convergence patterns in both countries are compared with several tools.
 Methods: Spatial econometric model for absolute beta convergence, sigma convergence indicators and the analysis of distribution dynamics ? transition matrices and kernel density estimation.
 Findings & Value added: The impact on EU accession on income convergence in Spain was positive both at the national and regional level. Regional convergence processes sped-up and interregional disparities decreased. The poorest subregions had relatively high probability to increase their income and catch-up with initially more developed regions. In the first decade after accession to the EU Poland has also achieved a significant improvement of income indicators at the national level. However, empirical analyses of GDP per capita distribution and its dynamics at the regional level in Poland show that the above mentioned progress does not spread out proportionally on all regions. Neither beta nor sigma convergence is observed. Instead, relatively fastest growth of initially richest regions (mostly large cities) introduces convergence of clubs leading to polarization. EU accession has accelerated divergence processes in Poland.

Perturbation Bounds for Markov Chains with General State Space

The aim of this paper is to investigate the stability of Markov chains with general state space. We present new conditions for the strong stability of Markov chains after a small perturbation of their transition kernels. Also, we obtain perturbation bounds with respect to different quantities.

LOCAL LARGE DEVIATIONS: McMILLIAN THEOREM FOR MULTITYPE GALTON-WATSON PROCESS

In this article we prove a local large deviation principle (LLDP) for the critical multitype Galton-Watson process from spectral potential point. We define the so-called a spectral potential $U_{\skrik}(\,\cdot,\,\pi)$ for the Galton-Watson process, where $\pi$ is the normalized eigen vector corresponding to the leading \emph{Perron-Frobenius eigen value } $\1$ of the transition matrix $\skria(\cdot,\,\cdot)$ defined from ${\skrik},$ the transition kernel. We show that the Kullback action or the deviation function, $J(\pi,\rho),$ with respect to an empirical offspring measure, $\rho,$ is the Legendre dual of $U_{\skrik}(\,\cdot,\,\pi).$ From the LLDP we deduce a conditional large deviation principle and a weak variant of the classical McMillian Theorem for the multitype Galton-Watson process. To be specific, given any empirical offspring measure $\varpi,$ we show that the number of critical multitype Galton-Watson processes on $n$ vertices is approximately $e^{n\langle \skrih_{\varpi},\,\pi\rangle},$ where $\skrih_{\varpi}$ is a suitably defined entropy.

Slow recurrent regimes for a class of one-dimensional stochastic growth models

We classify the possible behaviors of a class of one-dimensional stochastic recurrent growth models. In our main result, we obtain nearly optimal bounds for the tail of hitting times of some compact sets. If the process is an aperiodic irreducible Markov chain, we determine whether it is null recurrent or positive recurrent and in the latter case, we obtain a subgeometric convergence of its transition kernel to its invariant measure. We apply our results in particular to state-dependent Galton–Watson processes and we give precise estimates of the tail of the extinction time.

Smoothness of continuous state branching with immigration semigroups

In this work we develop an original and thorough analysis of the (non)-smoothness properties of the semigroups, and their heat kernels, associated to a large class of continuous state branching processes with immigration. Our approach is based on an in-depth analysis of the regularity of the absolutely continuous part of the invariant measure combined with a substantial refinement of Ogura's spectral expansion of the transition kernels. In particular, we find new representations for the eigenfunctions and eigenmeasures that allow us to derive delicate uniform bounds that are useful for establishing the uniform convergence of the spectral representation of the semigroup acting on linear spaces that we identify. We detail several examples which illustrate the variety of smoothness properties that CBI transition kernels may enjoy and also reveal that our results are sharp. Finally, our technique enables us to provide the (eventually) strong Feller property as well as the rate of convergence to equilibrium in the total variation norm.

On the effectiveness of Monte Carlo for initial uncertainty forecasting in nonlinear dynamical systems

This paper considers the problem of initial uncertainty forecasting in deterministic nonlinear continuous-time dynamical systems via particle ensembles. The popular Monte Carlo method, which while simple to implement, faces fundamental issues. In particular, it is not clear how well the propagated particles continue to represent the true state-uncertainty at future times. This paper evaluates the performance of Monte Carlo forecasting by analyzing it in the context of Markov chain Monte Carlo (MCMC) theory. The propagated ensemble is viewed as the realization of a Markov chain at each time instant, generated by an associated instantaneous transition kernel. It is shown that for a special class of nonlinear systems that have zero divergence, the propagated kernel is in detailed balance with the true state probability density function. This guarantees statistical consistency of the Monte Carlo ensemble with the truth at all times for such systems. On the other hand, no such guarantee is possible for systems with non-zero divergence.

Kolmogorov-type and general extension results for nonlinear expectations

We provide extension procedures for nonlinear expectations to the space of all bounded measurable functions. We first discuss a maximal extension for convex expectations which have a representation in terms of finitely additive measures. One of the main results of this article is an extension procedure for convex expectations which are continuous from above and therefore admit a representation in terms of countably additive measures. This can be seen as a nonlinear version of the Daniell–Stone theorem. From this, we deduce a robust Kolmogorov extension theorem which is then used to extend nonlinear kernels to an infinite-dimensional path space. We then apply this theorem to construct nonlinear Markov processes with a given family of nonlinear transition kernels.

Random walk on random walks: higher dimensions

We study the evolution of a random walker on a conservative dynamic random environment composed of independent particles performing simple symmetric random walks, generalizing results of [16] to higher dimensions and more general transition kernels without the assumption of uniform ellipticity or nearest-neighbour jumps. Specifically, we obtain a strong law of large numbers, a functional central limit theorem and large deviation estimates for the position of the random walker under the annealed law in a high density regime. The main obstacle is the intrinsic lack of monotonicity in higher-dimensional, non-nearest neighbour settings. Here we develop more general renormalization and renewal schemes that allow us to overcome this issue. As a second application of our methods, we provide an alternative proof of the ballistic behaviour of the front of (the discrete-time version of) the infection model introduced in [23].

Limit theorems for the zig-zag process

AbstractMarkov chain Monte Carlo (MCMC) methods provide an essential tool in statistics for sampling from complex probability distributions. While the standard approach to MCMC involves constructing discrete-time reversible Markov chains whose transition kernel is obtained via the Metropolis–Hastings algorithm, there has been recent interest in alternative schemes based on piecewise deterministic Markov processes (PDMPs). One such approach is based on the zig-zag process, introduced in Bierkens and Roberts (2016), which proved to provide a highly scalable sampling scheme for sampling in the big data regime; see Bierkenset al.(2016). In this paper we study the performance of the zig-zag sampler, focusing on the one-dimensional case. In particular, we identify conditions under which a central limit theorem holds and characterise the asymptotic variance. Moreover, we study the influence of the switching rate on the diffusivity of the zig-zag process by identifying a diffusion limit as the switching rate tends to ∞. Based on our results we compare the performance of the zig-zag sampler to existing Monte Carlo methods, both analytically and through simulations.

Sharp kernel estimates for elliptic operators with second-order discontinuous coefficients

We study parabolic problems associated to the second-order elliptic operator $$L = \Delta + (a-1)\sum_{i,j=1}^N\frac{x_{i}x_{j}}{|x|^2}D_{ij}+c\frac{x}{|x|^2}\cdot\nabla - b|x|^{-2}$$ with a > 0 and b, c real coefficients and prove kernel estimates for the transition kernel.