Zigzag Process Research Articles

Zigzag path connects two Monte Carlo samplers: Hamiltonian counterpart to a piecewise deterministic Markov process

Zigzag and other piecewise deterministic Markov process samplers have attracted significant interest for their non-reversibility and other appealing properties for Bayesian posterior computation. Hamiltonian Monte Carlo is another state-of-the-art sampler, exploiting fictitious momentum to guide Markov chains through complex target distributions. We establish an important connection between the zigzag sampler and a variant of Hamiltonian Monte Carlo based on Laplace-distributed momentum. The position and velocity component of the corresponding Hamiltonian dynamics travels along a zigzag path paralleling the Markovian zigzag process; however, the dynamics is non-Markovian in this position-velocity space as the momentum component encodes non-immediate pasts. This information is partially lost during a momentum refreshment step, in which we preserve its direction but re-sample magnitude. In the limit of increasingly frequent momentum refreshments, we prove that Hamiltonian zigzag converges strongly to its Markovian counterpart. This theoretical insight suggests that, when retaining full momentum information, Hamiltonian zigzag can better explore target distributions with highly correlated parameters by suppressing the diffusive behavior of Markovian zigzag. We corroborate this intuition by comparing performance of the two zigzag cousins on high-dimensional truncated multivariate Gaussians, including a 11,235-dimensional target arising from a Bayesian phylogenetic multivariate probit modeling of hiv virus data.

PDMP Monte Carlo methods for piecewise smooth densities

Abstract There has been substantial interest in developing Markov chain Monte Carlo algorithms based on piecewise deterministic Markov processes. However, existing algorithms can only be used if the target distribution of interest is differentiable everywhere. The key to adapting these algorithms so that they can sample from densities with discontinuities is to define appropriate dynamics for the process when it hits a discontinuity. We present a simple condition for the transition of the process at a discontinuity which can be used to extend any existing sampler for smooth densities, and give specific choices for this transition which work with popular algorithms such as the bouncy particle sampler, the coordinate sampler, and the zigzag process. Our theoretical results extend and make rigorous arguments that have been presented previously, for instance constructing samplers for continuous densities restricted to a bounded domain, and we present a version of the zigzag process that can work in such a scenario. Our novel approach to deriving the invariant distribution of a piecewise deterministic Markov process with boundaries may be of independent interest.

NuZZ: Numerical Zig-Zag for general models

Markov chain Monte Carlo (MCMC) is a key algorithm in computational statistics, and as datasets grow larger and models grow more complex, many popular MCMC algorithms become too computationally expensive to be practical. Recent progress has been made on this problem through development of MCMC algorithms based on Piecewise Deterministic Markov Processes (PDMPs), irreversible processes which can be engineered to converge at a rate which is independent of the size of the dataset. While there has understandably been a surge of theoretical studies following these results, PDMPs have so far only been implemented for models where certain gradients can be bounded in closed form, which is not possible in many relevant statistical problems. Furthermore, there has been substantionally less focus on practical implementation, or the efficiency of PDMP dynamics in exploring challenging densities. Focusing on the Zig-Zag process, we present the Numerical Zig-Zag (NuZZ) algorithm, which is applicable to general statistical models without the need for bounds on the gradient of the log posterior. This allows us to perform numerical experiments on: (i) how the Zig-Zag dynamics behaves on some test problems with common challenging features; and (ii) how the error between the target and sampled distributions evolves as a function of computational effort for different MCMC algorithms including NuZZ. Moreover, due to the specifics of the NuZZ algorithms, we are able to give an explicit bound on the Wasserstein distance between the exact posterior and its numerically perturbed counterpart in terms of the user-specified numerical tolerances of NuZZ.

Automation of trimming die design inspection by zigzag process between AI and CAD domains

Quality control using artificial intelligence (AI) has led to considerable improvements in the manufacturing industry. However, inspection of computer-aided design (CAD) drawing to prevent potential errors is still performed manually by engineers. The design inspection which involves sequential and repetitive processes is labor-intensive, time-consuming, and prone to human error. In this study, we developed an automatic design inspection system for CAD data by integrating AI module and CAD software. The target product of this study is automobile trimming die design. The AI modules replace the engineers’ judgment, and the CAD software performs the operations requested by the AI modules. The design inspection is completed through a zigzag interaction between the AI modules and CAD software by a one-click operation without the intervention of experts. In the zigzag interaction process, 3D CAD files are converted to 2D images for interaction from the CAD software to the AI modules, and the output of the AI modules is transferred to the CAD software through VBA macros. The AI modules are designed to be CAD-independent and data-efficient; therefore, they can be easily adapted to other CAD software and achieve a high performance, even when trained with a few trimming dies. The length measurement error is only 2.4 % on average, which is at the 14th place out of 18 grades according to ISO 286–1:1988 standard tolerance. Additionally, the inspection time is reduced to approximately one-fifth of the manual inspection time by experts.

High-dimensional scaling limits of piecewise deterministic sampling algorithms

Piecewise deterministic Markov processes are an important new tool in the design of Markov chain Monte Carlo algorithms. Two examples of fundamental importance are the bouncy particle sampler (BPS) and the zig–zag process (ZZ). In this paper scaling limits for both algorithms are determined. Here the dimensionality of the space tends towards infinity and the target distribution is the multivariate standard normal distribution. For several quantities of interest (angular momentum, first coordinate and negative log-density) the scaling limits show qualitatively very different and rich behaviour. Based on these scaling limits the performance of the two algorithms in high dimensions can be compared. Although for angular momentum both processes require only a computational effort of O(d) to obtain approximately independent samples, the computational effort for negative log-density and first coordinate differ: for these BPS requires O(d2) computational effort whereas ZZ requires O(d). Finally we provide a criterion for the choice of the refreshment rate of BPS.

Video compression based on zig-zag 3D DCT and run-length encoding for multimedia communication systems

Purpose With the advent of technology, a huge amount of data is being transmitted and received through the internet. Large bandwidth and storage are required for the exchange of data and storage, respectively. Hence, compression of the data which is to be transmitted over the channel is unavoidable. The main purpose of the proposed system is to use the bandwidth effectively. The videos are compressed at the transmitter’s end and reconstructed at the receiver’s end. Compression techniques even help for smaller storage requirements. Design/methodology/approach The paper proposes a novel compression technique for three-dimensional (3D) videos using a zig-zag 3D discrete cosine transform. The method operates a 3D discrete cosine transform on the videos, followed by a zig-zag scanning process. Finally, to convert the data into a single bit stream for transmission, a run-length encoding technique is used. The videos are reconstructed by using the inverse 3D discrete cosine transform, inverse zig-zag scanning (quantization) and inverse run length coding techniques. The proposed method is simple and reduces the complexity of the convolutional techniques. Findings Coding reduction, code word reduction, peak signal to noise ratio (PSNR), mean square error, compression percent and compression ratio values are calculated, and the dominance of the proposed method over the convolutional methods is seen. Originality/value With zig-zag quantization and run length encoding using 3D discrete cosine transform for 3D video compression, gives compression up to 90% with a PSNR of 41.98 dB. The proposed method can be used in multimedia applications where bandwidth, storage and data expenses are the major issues.

A note on the polynomial ergodicity of the one-dimensional Zig-Zag process

AbstractWe prove polynomial ergodicity for the one-dimensional Zig-Zag process on heavy-tailed targets and identify the exact order of polynomial convergence of the process when targeting Student distributions.

Differentiation, Integration and Exclusion: Dynamics and Challenges to Nation-Building

Ethnicity, nation, nationalism and the nation-state have been politically volatile concepts marked by ambiguities and predominantly Eurocentric bias. These have impacted the formation of states and the dynamics within them. Is the nation-state the exclusive domain of ethno-nations forming a territorial state by exercising the right of self-determination? The emancipation of post-colonial states added a new dimension to the on-going discourse; among these, the contribution of the Indian subcontinent is prominent. Their leaders who were in the forefront of the freedom struggle were not only familiar with this discourse, but actually played out their politics on the basis of either subscribing to, or, in outright rejection of, the Eurocentric model. The latter, argued for, and established, a civic-secular nation-state that admitted ethnic plurality and class realities of equality, within an overarching civilisational bonding. This article examines how this dynamic played itself out in the laboratory of South Asia, particularly, with reference to the formations of India and Pakistan. Their top leaders Mahatma Gandhi and Quad-e-Azam Jinnah held opposing views on the concept of the nation and nation-state. I have argued that the Indian nation-state has steadily evolved through a continuous zig-zag process of differentiation and integration. In recent times, a new contradiction has surfaced out of believers in ethno-nation, introducing a volatile churning in the political process.

Complexity of zigzag sampling algorithm for strongly log-concave distributions

We study the computational complexity of zigzag sampling algorithm for strongly log-concave distributions. The zigzag process has the advantage of not requiring time discretization for implementation, and that each proposed bouncing event requires only one evaluation of partial derivative of the potential, while its convergence rate is dimension independent. Using these properties, we prove that the zigzag sampling algorithm achieves $\varepsilon$ error in chi-square divergence with a computational cost equivalent to $O\bigl(\kappa^2 d^\frac{1}{2}(\log\frac{1}{\varepsilon})^{\frac{3}{2}}\bigr)$ gradient evaluations in the regime $\kappa \ll \frac{d}{\log d}$ under a warm start assumption, where $\kappa$ is the condition number and $d$ is the dimension.

Spectral analysis of the zigzag process

Le processus de ZigZag est une variante du processus du télégraphe avec des intensités de retournement dépendant de la position. Nous obtenons une caractérisation du spectre en norme L2 du générateur du processus de zigzag unidimensionnel, dans le cas où la distribution marginale stationnaire sur R est unimodale, avec une fréquence de rééchantillonnage nulle. Nous obtenons des conditions suffisantes pour un théorème d’isomorphisme spectral, identifiant le spectre du générateur à celui du semi-groupe de Markov correspondant. Par ailleurs, nous obtenons des résultats pour les distributions stationnaires symétriques ainsi que pour les perturbations du spectre dans le cas particulier d’une fréquence de rééchantillonnage non nulle. Enfin, nous considérons dans les exemples (avec une distribution cible gaussienne) un faible taux de rafraichissement positif par rapport aux résultats du taux nul, ce qui induit une plus grande bande dans le spectre L2, correspondant à une convergence améliorée en norme L2.

On explicit L2-convergence rate estimate for piecewise deterministic Markov processes in MCMC algorithms

We establish $L^2$-exponential convergence rate for three popular piecewise deterministic Markov processes for sampling: the randomized Hamiltonian Monte Carlo method, the zigzag process, and the bouncy particle sampler. Our analysis is based on a variational framework for hypocoercivity, which combines a Poincar\'{e}-type inequality in time-augmented state space and a standard $L^2$ energy estimate. Our analysis provides explicit convergence rate estimates, which are more quantitative than existing results.

Zig-Zag Sampling for Discrete Structures and Nonreversible Phylogenetic MCMC

We construct a zig-zag process targeting a posterior distribution defined on a hybrid state space consisting of both discrete and continuous variables. The construction does not require any assumptions on the structure among discrete variables. We demonstrate our method on two examples in genetics based on the Kingman coalescent, showing that the zig-zag process can lead to efficiency gains of up to several orders of magnitude over classical Metropolis–Hastings algorithms, and that it is well suited to parallel computation. Our construction resembles existing techniques for Hamiltonian Monte Carlo on a hybrid state space, which suffers from implementationally and analytically complex boundary crossings when applied to the coalescent. We demonstrate that the continuous-time zig-zag process avoids these complications. Supplementary materials for this article are available online.

A Simple Image Encryption Based on Binary Image Affine Transformation and Zigzag Process

In this paper, we propose a new and simple method for image encryption. It uses an external secret key of 128 bits long and an internal secret key. The novelties of the proposed encryption process are the methods used to extract an internal key to apply the zigzag process, affine transformation, and substitution-diffusion process. Initially, an original gray-scale image is converted into binary images. An internal secret key is extracted from binary images. The two keys are combined to compute the substitution-diffusion keys. The zigzag process is firstly applied on each binary image. Using an external key, every zigzag binary image is reflected or rotated and a new gray-scale image is reconstructed. The new image is divided into many nonoverlapping subblocks, and each subblock uses its own key to take out a substitution-diffusion process. We tested our algorithms on many biomedical and nonmedical images. It is seen from evaluation metrics that the proposed image encryption scheme provides good statistical and diffusion properties and can resist many kinds of attacks. It is an efficient and secure scheme for real-time encryption and transmission of biomedical images in telemedicine.

Large deviations for the empirical measure of the zig-zag process

The zig-zag process is a piecewise deterministic Markov process in position and velocity space. The process can be designed to have an arbitrary Gibbs type marginal probability density for its position coordinate, which makes it suitable for Monte Carlo simulation of continuous probability distributions. An important question in assessing the efficiency of this method is how fast the empirical measure converges to the stationary distribution of the process. In this paper we provide a partial answer to this question by characterizing the large deviations of the empirical measure from the stationary distribution. Based on the Feng–Kurtz approach, we develop an abstract framework aimed at encompassing piecewise deterministic Markov processes in position-velocity space. We derive explicit conditions for the zig-zag process to allow the Donsker–Varadhan variational formulation of the rate function, both for a compact setting (the torus) and one-dimensional Euclidean space. Finally we derive an explicit expression for the Donsker–Varadhan functional for the case of a compact state space and use this form of the rate function to address a key question concerning the optimal choice of the switching rate of the zig-zag process.

Integration of axiomatic design and design structure matrix for the modular design of automobile parts

As the demands of customers in the modern industry increased, the number of products, and the variety of components has increased. These issues have led to difficulties in product development and production. Modularization of products has advantages such as cost reduction, product development time reduction, and production time reduction. Modular design of products has been studied in the design activities of the modern industry. In this study, a modular design method is proposed to design a modular product based on axiomatic design (AD) and design structure matrix (DSM). AD and DSM are efficiently integrated into the proposed method. Functional requirements and design parameters are defined based on the Independence Axiom of AD, and the zigzagging process of AD is employed for the decomposition of the functional requirements (FRs) and design parameters (DPs). The design sequence is established based on the design matrix. Coupled or functionally close DPs are grouped into a module (Module 1). These modules are efficiently used in the design sequence. DSM is used to modularize the design parameters of the lowest level of axiomatic design. DSM is constructed based on physical interfaces and numerical clustering algorithms are used to identify strongly related components. They are grouped into a module (Module 2). Module 2 is exploited for production and management. Therefore, these two modules for different purposes can be used to increase efficiency in the design and production process. The proposed method is applied to two automobile parts such as the suspension system and cooling system. The results are discussed from the viewpoint of usefulness.

Image encryption algorithm based on dynamic DNA coding operations and 3D chaotic systems

This paper presents a new image encryption algorithm based on 3D chaotic system and deoxyribonucleic acid (DNA) coding. It uses two keys, an external one of 128 bits long and an internal one of 64 gray values coming from the plain image. The initial conditions come from the two keys and vary from one line of the image to the other and from one image to the other and consequently the sequences of substitutions too. In general, in image encryption based on 3D chaotic systems, each chaotic variable is used independently of the others in one phase of the encryption. Here we use the zigzag process to combine the sequences from all the variables before using them simultaneously in the encryption process. DNA coding generally uses one of the 24 DNA rules and one of the 16 join operations to perform the encryption. Here we use all the 24 rules dynamically as well as the 16 join operations in the encryption. Also we apply the chaotic permutation on DNA chain. The logical DNA operations are used according to an algorithm similar to that of the Fast Walsh Transform (FWT) in those 24 DNA rules. The algorithm has been evaluated. It has good statistical properties of cipher images. The proposed system presents high sensitivity on the encryption-decryption keys and on the plain images. These latter features make the proposed algorithm very efficient, robust and resistant against brute force attacks and good for the future secure image communication.

Cores for piecewise-deterministic Markov processes used in Markov chain Monte Carlo

We show fundamental properties of the Markov semigroup of recently proposed MCMC algorithms based on Piecewise-deterministic Markov processes (PDMPs) such as the Bouncy Particle Sampler, the Zig-Zag process or the Randomized Hamiltonian Monte Carlo method. Under assumptions typically satisfied in MCMC settings, we prove that PDMPs are Feller and that their generator admits the space of infinitely differentiable functions with compact support as a core. As we illustrate via martingale problems and a simplified proof of the invariance of target distributions, these results provide a fundamental tool for the rigorous analysis of these algorithms and corresponding stochastic processes.

Low-Lying Eigenvalues and Convergence to the Equilibrium of Some Piecewise Deterministic Markov Processes Generators in the Small Temperature Regime

In this work we study the number of small eigenvalues and the convergence to the equilibrium of the Bouncy Particle Sampler process and the Zig-Zag process generators in the small temperature regime. Such processes, which fall in the class of Piecewise Deterministic Markov Processes, are non diffusive and non reversible.

A Novel Approach for Copy-move Forgery Detection using Bilateral Filtering

Digital image processing methods have a wide area of usage and their complexity is increasing, as well as the tampering methods. A widely used tampering method is copy-move forgery. In this study, a hybrid method combining the DCT and Bilateral filtering is developed. In this method, first overlapping blocks are obtained from the input image. Then, bilateral filtering and DCT of these blocks are multiplied to obtain the refined block features. The block features are scanned by a zig-zag process followed by a lexicographic sorting. Finally, a similarity detection by a predetermined threshold parameter is applied to detect the forgery. Both visual and quantitative results demonstrated that the proposed method can determine the copy-move forgery regions.

Kinetic walks for sampling

The persistent walk is a classical model in kinetic theory, which has also been studied as a toy model for MCMC questions. Its continuous limit, the telegraph process, has recently been extended to various velocity jump processes (Bouncy Particle Sampler, Zig-Zag process, etc.) in order to sample general target distributions on $\mathbb R^d$. This paper studies, from a sampling point of view, general kinetic walks that are natural discrete-time (and possibly discrete-space) counterparts of these continuous-space processes. The main contributions of the paper are the definition and study of a discrete-space Zig-Zag sampler and the definition and time-discretisation of hybrid jump/diffusion kinetic samplers for multi-scale potentials on $\mathbb R^d$.