Class Of Markov Chains Research Articles

A class of Markov chains with no spectral gap

In this paper we extend the results of the research started by the first author in which Karlin-McGregor diagonalization of certain reversible Markov chains over countably infinite general state spaces by orthogonal polynomials was used to estimate the rate of convergence to a stationary distribution. We use a method of Koornwinder to generate a large and interesting family of random walks which exhibits a lack of spectral gap, and a polynomial rate of convergence to the stationary distribution. For the Chebyshev type subfamily of Markov chains, we use asymptotic techniques to obtain an upper bound of order O ( log ⁡ t t ) O\left ({\log {t} \over \sqrt {t}}\right ) and a lower bound of order O ( 1 t ) O\left ({1 \over \sqrt {t}}\right ) on the distance to the stationary distribution regardless of the initial state. Due to the lack of a spectral gap, these results lie outside the scope of geometric ergodicity theory.

Pure Probability Global Convergence of Improved Clonal Selection Algorithm

Clonal Selection Algorithm (CSA) has been applied widely in intelligent computation field, but the global convergence theoretical analysis and research works about CSA were rare relatively. The global convergence of an improved clonal selection algorithm is investigated in this paper. Through the definition of global convergence in probability, the classical homogeneous Markov chain analyses is replaced by a new pure probability and iterative formula method, and the convergence results of the improved clonal selection algorithm are obtained. This method makes the global convergence research greatly simplified and enriches new contents for the theoretical foundation of the clonal selection algorithm.

The Stationary Distributions of a Class of Markov Chains

The objective of this paper is to find the stationary distribution of a certain class of Markov chains arising in a biological population involved in a specific type of evolutionary conflict, known as Parker’s model. In a population of such players, the result of repeated, infrequent, attempted invasions using strategies from{0,1,2,…,m-1}, is a Markov chain. The stationary distributions of this class of chains, for m e {3,4,…,∞} are derived in terms of previously known integer sequences. The asymptotic distribution (for m →∞) is derived.

On the Markov chain tree theorem in the Max algebra

A max-algebraic analogue of the Markov Chain Tree Theorem is presented, and its connections with the classical Markov Chain Tree Theorem and the max-algebraic spectral theory are investigated.

On first returning time and last exit time of a class of Markov chain

Let {X n } be a Markov chain with transition probability $$p_{ij} = a_{j - (i - 1)^ + } ,\forall i,j \geqslant 0$$ , where a j = 0 provided j < 0, a 0 > 0, a 0 + a 1 < 1 and Σ =0 ∞ a n = 1. Let µ = Σ =1 ∞ na n . It is known that {X n } is positive recurrent when µ 1. In this paper, the integrability of the first returning time and the last exit time are discussed.

A Classical MCMC Approach to the Estimation of Limited Dependent Variable Models of Time Series

Estimating limited dependent variable time series models through standard extremum methods can be a daunting computational task because of the need for integration of high order multiple integrals and/or numerical optimization of difficult objective functions. This paper proposes a classical Markov Chain Monte Carlo (MCMC) estimation technique with data augmentation that overcomes both of these problems. The asymptotic properties of the proposed estimator are discussed. Furthermore, a practical and flexible algorithmic framework for this class of models is proposed and is illustrated using simulated data, thus also offering some insight into the small-sample biases of such estimators. Finally, the proposed framework is used to estimate a dynamic, discrete-choice monetary policy reaction function for the United States during the Greenspan years.

A SUCCESSIVE LUMPING PROCEDURE FOR A CLASS OF MARKOV CHAINS

A class of Markov chains we call successively lumbaple is specified for which it is shown that the stationary probabilities can be obtained by successively computing the stationary probabilities of a propitiously constructed sequence of Markov chains. Each of the latter chains has a(typically much) smaller state space and this yields significant computational improvements. We discuss how the results for discrete time Markov chains extend to semi-Markov processes and continuous time Markov processes. Finally, we will study applications of successively lumbaple Markov chains to classical reliability and queueing models.

A support vector density-based importance sampling for reliability assessment

An importance sampling method based on the adaptive Markov chain simulation and support vector density estimation is developed in this paper for efficient structural reliability assessment. The methodology involves the generation of samples that can adaptively populate the important region by the adaptive Metropolis algorithm, and the construction of importance sampling density by support vector density. The use of the adaptive Metropolis algorithm may effectively improve the convergence and stability of the classical Markov chain simulation. The support vector density can approximate the sampling density with fewer samples in comparison to the conventional kernel density estimation. The proposed importance sampling method can effectively reduce the number of structural analysis required for achieving a given accuracy. Examples involving both numerical and practical structural problems are given to illustrate the application and efficiency of the proposed methodology.

Open Quantum Random Walks

A new model of quantum random walks is introduced, on lattices as well as on finite graphs. These quantum random walks take into account the behavior of open quantum systems. They are the exact quantum analogues of classical Markov chains. We explore the “quantum trajectory” point of view on these quantum random walks, that is, we show that measuring the position of the particle after each time-step gives rise to a classical Markov chain, on the lattice times the state space of the particle. This quantum trajectory is a simulation of the master equation of the quantum random walk. The physical pertinence of such quantum random walks and the way they can be concretely realized is discussed. Differences and connections with the already well-known quantum random walks, such as the Hadamard random walk, are established.

A Classical MCMC Approach to the Estimation of Limited Dependent Variable Models of Time Series

Estimating Limited Dependent Variable Time Series models through standard extremum methods can be a daunting computational task because of the need for integration of high order multiple integrals and/or numerical optimization of difficult objective functions. This paper proposes a classical Markov Chain Monte Carlo (MCMC) estimation technique with data augmentation that overcomes both of these problems. The asymptotic properties of the proposed estimator are discussed. Furthermore, a practical and flexible algorithmic framework for this class of models is proposed and is illustrated using simulated data, thus also offering some insight into the small-sample biases of such estimators. Finally, the proposed framework is used to estimate a dynamic, discrete-choice monetary policy reaction function for the United States during the Greenspan years.

Improved synthetic wind speed generation using modified Mycielski approach

SUMMARY In this paper, novel approaches for wind speed data generation using Mycielski algorithm are developed and presented. To show the accuracy of developed approaches, we used three-year collected wind speed data belonging to deliberately selected two different regions of Turkey (Izmir and Kayseri) to generate artificial wind speed data. The data belonging to the first two years are used for training, whereas the remaining one-year data are used for testing and accuracy comparison purposes. The concept of distinct synthetic data production with correlation-wise and distribution-wise similar statistical properties constitutes the main idea of the proposed methods for a successful artificial wind speed generation. Generated data are compared with test data for both regions in the sense of basic statistics, Weibull distribution parameters, transition probabilities, spectral densities, and autocorrelation functions; and are also compared with the data generated by the classical first-order Markov chains method. Results indicate that the accuracy and realistic behavior of the proposed method is superior to the classical method in the literature. Comparisons and results are discussed in detail. Copyright © 2011 John Wiley & Sons, Ltd.

Search via Quantum Walk

We propose a new method for designing quantum search algorithms for finding a “marked” element in the state space of a classical Markov chain. The algorithm is based on a quantum walk à la Szegedy [Quantum speed-up of Markov chain based algorithms, in Proceedings of the 45th IEEE Symposium on Foundations of Computer Science, IEEE Computer Society Press, 2004, pp. 32–41] that is defined in terms of the Markov chain. The main new idea is to apply quantum phase estimation to the quantum walk in order to implement an approximate reflection operator. This operator is then used in an amplitude amplification scheme. As a result we considerably expand the scope of the previous approaches of Ambainis [Quantum walk algorithm for Element Distinctness, in Proceedings of the 45th IEEE Symposium on Foundations of Computer Science, IEEE Computer Society Press, 2004, pp. 22–31] and Szegedy (2004). Our algorithm combines the benefits of these approaches in terms of being able to find marked elements, incurring the smaller cost of the two, and being applicable to a larger class of Markov chains. In addition, it is conceptually simple and avoids some technical difficulties in the previous analyses of several algorithms based on quantum walk.

The $\chi ^2$χ2-divergence and mixing times of quantum Markov processes

We introduce quantum versions of the \documentclass[12pt]{minimal}\begin{document}$\chi ^2$\end{document}χ2-divergence, provide a detailed analysis of their properties, and apply them in the investigation of mixing times of quantum Markov processes. An approach similar to the one presented in the literature for classical Markov chains is taken to bound the trace-distance from the steady state of a quantum processes. A strict spectral bound to the convergence rate can be given for time-discrete as well as for time-continuous quantum Markov processes. Furthermore, the contractive behavior of the \documentclass[12pt]{minimal}\begin{document}$\chi ^2$\end{document}χ2-divergence under the action of a completely positive map is investigated and contrasted to the contraction of the trace norm. In this context we analyze different versions of quantum detailed balance and, finally, give a geometric conductance bound to the convergence rate for unital quantum Markov processes.

Generalized crested products of Markov chains

We define a finite Markov chain, called generalized crested product, which naturally appears as a generalization of the first crested product of Markov chains. A complete spectral analysis is developed and the k -step transition probability is given. It is important to remark that this Markov chain describes a more general version of the classical Ehrenfest diffusion model. As a particular case, one gets a generalization of the classical Insect Markov chain defined on the ultrametric space. Finally, an interpretation in terms of representation group theory is given, by showing the correspondence between the spectral decomposition of the generalized crested product and the Gelfand pairs associated with the generalized wreath product of permutation groups.

Multicomponent reaction models in ozonation and reduction in the number of model parameters

The paper discusses multireaction models in ozonation, in particular residual COD models and radical chain models. As a result of the large number of estimable parameters, multicomponent reaction models in ozonation may lead to overparametrization in nonlinear parameter estimation. The use of theoretical COD values of known components in the residual COD model and knowledge obtained from the stoichiometry of reaction equations enables a remarkable reduction in the number of parameters to be estimated. The reliability and identifiability of the estimated parameters is demonstrated using both classical statistical analysis and Markov chain Monte Carlo (MCMC) analysis.

Efficient circuits for quantum walks

We present an efficient general method for realizing a quantum walk operator corresponding to an arbitrary sparse classical random walk. Our approach is based on Grover and Rudolph's method for preparing coherent versions of efficiently integrable probability distributions \cite{GroverRudolph}. This method is intended for use in quantum walk algorithms with polynomial speedups, whose complexity is usually measured in terms of how many times we have to apply a step of a quantum walk \cite{Szegedy}, compared to the number of necessary classical Markov chain steps. We consider a finer notion of complexity including the number of elementary gates it takes to implement each step of the quantum walk with some desired accuracy. The difference in complexity for various implementation approaches is that our method scales linearly in the sparsity parameter and poly-logarithmically with the inverse of the desired precision. The best previously known general methods either scale quadratically in the sparsity parameter, or polynomially in the inverse precision. Our approach is especially relevant for implementing quantum walks corresponding to classical random walks like those used in the classical algorithms for approximating permanents \cite{Vigoda, Vazirani} and sampling from binary contingency tables \cite{Stefankovi}. In those algorithms, the sparsity parameter grows with the problem size, while maintaining high precision is required.

Discrete Hit-and-Run for Sampling Points from Arbitrary Distributions Over Subsets of Integer Hyperrectangles

We consider the problem of sampling a point from an arbitrary distribution π over an arbitrary subset S of an integer hyperrectangle. Neither the distribution π nor the support set S are assumed to be available as explicit mathematical equations, but may only be defined through oracles and, in particular, computer programs. This problem commonly occurs in black-box discrete optimization as well as counting and estimation problems. The generality of this setting and high dimensionality of S precludes the application of conventional random variable generation methods. As a result, we turn to Markov chain Monte Carlo (MCMC) sampling, where we execute an ergodic Markov chain that converges to π so that the distribution of the point delivered after sufficiently many steps can be made arbitrarily close to π. Unfortunately, classical Markov chains, such as the nearest-neighbor random walk or the coordinate direction random walk, fail to converge to π because they can get trapped in isolated regions of the support set. To surmount this difficulty, we propose discrete hit-and-run (DHR), a Markov chain motivated by the hit-and-run algorithm known to be the most efficient method for sampling from log-concave distributions over convex bodies in Rn. We prove that the limiting distribution of DHR is π as desired, thus enabling us to sample approximately from π by delivering the last iterate of a sufficiently large number of iterations of DHR. In addition to this asymptotic analysis, we investigate finite-time behavior of DHR and present a variety of examples where DHR exhibits polynomial performance.

Crested products of Markov chains

In this work we define two kinds of crested product for reversible Markov chains, which naturally appear as a generalization of the case of crossed and nested product, as in association schemes theory, even if we do a construction that seems to be more general and simple. Although the crossed and nested product are inspired by the study of Gelfand pairs associated with the direct and the wreath product of two groups, the crested products are a more general construction, independent from the Gelfand pairs theory, for which a complete spectral theory is developed. Moreover, the $k$-step transition probability is given. It is remarkable that these Markov chains describe some classical models (Ehrenfest diffusion model, Bernoulli--Laplace diffusion model, exclusion model) and give some generalization of them. As a particular case of nested product, one gets the classical Insect Markov chain on the ultrametric space. Finally, in the context of the second crested product, we present a generalization of this Markov chain to the case of many insects and give the corresponding spectral decomposition.

Asymptotically homogeneous iterated random functions with applications to the HARCH process

We show that a certain class of Markov chains satisfies the general Foster–Lyapunov condition under nonrestrictive assumptions. In particular, if a chain belongs to this class, the existence of an invariant measure can be established by means of the theorem provided. As an application of the theorem, the HARCH process is considered.

Re'class'ification of 'quant'ified classical simulated annealing

We discuss a classical reinterpretation of quantum-mechanics-based analysis of classical Markov chains with detailed balance, that is based on the quantum-classical correspondence. The classical reinterpretation is then used to demonstrate that it successfully reproduces a sufficient condition for cooling schedule in classical simulated annealing, which has the inverse-logarithmic scaling.