Transition Matrices Of Markov Chains Research Articles

Markov modeling of phase singularity interaction effects in human atrial and ventricular fibrillation.

Atrial and ventricular fibrillation (AF/VF) are characterized by the repetitive regeneration of topological defects known as phase singularities (PSs). The effect of PS interactions has not been previously studied in human AF and VF. We hypothesized that PS population size would influence the rate of PS formation and destruction in human AF and VF, due to increased inter-defect interaction. PS population statistics were studied in computational simulations (Aliev-Panfilov), human AF and human VF. The influence of inter-PS interactions was evaluated by comparison between directly modeled discrete-time Markov chain (DTMC) transition matrices of the PS population changes, and M/M/∞ birth-death transition matrices of PS dynamics, which assumes that PS formations and destructions are effectively statistically independent events. Across all systems examined, PS population changes differed from those expected with M/M/∞. In human AF and VF, the formation rates decreased slightly with PS population when modeled with the DTMC, compared with the static formation rate expected through M/M/∞, suggesting new formations were being inhibited. In human AF and VF, the destruction rates increased with PS population for both models, with the DTMC rate increase exceeding the M/M/∞ estimates, indicating that PS were being destroyed faster as the PS population grew. In human AF and VF, the change in PS formation and destruction rates as the population increased differed between the two models. This indicates that the presence of additional PS influenced the likelihood of new PS formation and destruction, consistent with the notion of self-inhibitory inter-PS interactions.

PO-01-156 MARKOV MODELLING OF PHASE SINGULARITY INTERACTION EFFECTS IN HUMAN ATRIAL AND VENTRICULAR FIBRILLATION

Atrial and ventricular fibrillation (AF/VF) are characterized by the repetitive regeneration of topological defects known as phase singularities (PS). The interaction between phase singularities has not been studied in human AF and VF, but is potentially of profound significance in understanding how cardiac fibrillation self-sustains. In this study we hypothesized that PS population size would influence the rate of PS formation and destruction in human AF and VF, due to an increase in inter-defect interaction. PS population statistics were first studied in (1) computational simulations of spiral defect chaos (Aliev-Panfilov, APV), followed by analysis in (2) human datasets of AF mapped using basket catheters prior to AF ablation & (3) VF mapped using epicardial electrode socks prior to cardiac surgery. The influence of inter-PS interactions was evaluated by comparison between directly modelled discrete time Markov chain (DTMC) transition matrices of the PS population changes, and M/M/∞ birth-death transition matrices of PS dynamics, which assume PS formation and destruction occur effectively independent of other PS. In all systems examined, PS population changes modelled by the DTMC differed from the M/M/∞. In APV simulations the DTMC formation rates decreased with PS population size (β estimate = -1.08 %/ms (95%CI: -1.20,-0.970), p<0.001), the DTMC destruction rates increased (β estimate = 1.78 %/ms (95% CI: 1.66,1.90), p<0.001), and M/M/∞ destruction rates increased at a slightly greater rate (β estimate = 2.00 %/ms (95% CI: 1.83,2.18), p<0.001). In AF (n=15) the DTMC formation rates decreased with PS population (β estimate = -0.179 %/ms (95% CI: -0.253,-0.105), p<0.001), M/M/∞ destruction rates increased (β estimate = 1.57 %/ms (95% CI: 1.43,1.70), p<0.001), and the DTMC destruction rates increased at a greater rate (β estimate = 2.65 %/ms (95% CI: 2.42,2.88), p<0.001). All stages of VF (n=8) demonstrated similar trends as AF, with the DTMC formation rates decreasing (βPerfusion estimate = -1.02 %/ms (95% CI: -1.37,-0.667), p=0.034), the M/M/∞ destruction rates increasing (βPerfusion estimate = 0.970 %/ms (95% CI: 0.780,1.15), p<0.001), and the DTMC destruction rates increasing at a greater rate (βPerfusion estimate = 3.42 %/ms (95% CI: 2.63,4.22), p<0.001). We present evidence of PS interaction in human AF and VF on rotor formation and destruction rates, with evidence for a potential self-inhibitory effect due to increased PS population size.

Dynamic Transitions of Pediatric Sepsis: A Markov Chain Analysis.

Pediatric sepsis is a heterogeneous disease with varying physiological dynamics associated with recovery, disability, and mortality. Using risk scores generated from a sepsis prediction model to define illness states, we used Markov chain modeling to describe disease dynamics over time by describing how children transition among illness states. We analyzed 18,666 illness state transitions over 157 pediatric intensive care unit admissions in the 3 days following blood cultures for suspected sepsis. We used Shannon entropy to quantify the differences in transition matrices stratified by clinical characteristics. The population-based transition matrix based on the sepsis illness severity scores in the days following a sepsis diagnosis can describe a sepsis illness trajectory. Using the entropy based on Markov chain transition matrices, we found a different structure of dynamic transitions based on ventilator use but not age group. Stochastic modeling of transitions in sepsis illness severity scores can be useful in describing the variation in transitions made by patient and clinical characteristics.

On monotonically proceeding structures and stepwise increasing transition matrices of Markov chains

In general, the transition matrix of a Markov chain is a stochastic matrix. For a system that is modeled by a Markov chain, the transition matrix must reflect the characteristics of that system. The present paper introduces a particular class of transition matrices in order to model Markov systems for which, as the length of the time interval becomes greater, a transition from one state to another is more likely. We call these transition matrices stepwise increasing. Moreover in some contexts it is less desirable that the stocks fluctuate over time. In those situations, one is interested in monotonically proceeding stock vectors. This paper examines monotonically proceeding stock vectors and stepwise increasing transition matrices. We present conditions on the transition matrix such that all stock vectors are monotonically proceeding. In particular, the set of monotonically proceeding vectors is characterized for the two-state and three-state cases.

Interpretation and approximation tools for big, dense Markov chain transition matrices in population genetics.

BackgroundMarkov chains are a common framework for individual-based state and time discrete models in evolution. Though they played an important role in the development of basic population genetic theory, the analysis of more complex evolutionary scenarios typically involves approximation with other types of models. As the number of states increases, the big, dense transition matrices involved become increasingly unwieldy. However, advances in computational technology continue to reduce the challenges of “big data”, thus giving new potential to state-rich Markov chains in theoretical population genetics.ResultsUsing a population genetic model based on genotype frequencies as an example, we propose a set of methods to assist in the computation and interpretation of big, dense Markov chain transition matrices. With the help of network analysis, we demonstrate how they can be transformed into clear and easily interpretable graphs, providing a new perspective even on the classic case of a randomly mating, finite population with mutation. Moreover, we describe an algorithm to save computer memory by substituting the original matrix with a sparse approximate while preserving its mathematically important properties, including a closely corresponding dominant (normalized) eigenvector. A global sensitivity analysis of the approximation results in our example shows that size reduction of more than 90 % is possible without significantly affecting the basic model results. Sample implementations of our methods are collected in the Python module mamoth.ConclusionOur methods help to make stochastic population genetic models involving big, dense transition matrices computationally feasible. Our visualization techniques provide new ways to explore such models and concisely present the results. Thus, our methods will contribute to establish state-rich Markov chains as a valuable supplement to the diversity of population genetic models currently employed, providing interesting new details about evolution e.g. under non-standard reproductive systems such as partial clonality.Electronic supplementary materialThe online version of this article (doi:10.1186/s13015-015-0061-5) contains supplementary material, which is available to authorized users.

Interconnections between Eurozone and US Booms and Busts Using a Bayesian Panel Markov-Switching VAR Mode

Interconnections between Eurozone and United States booms and busts and among major Eurozone economies are analyzed using a Panel Markov-Switching VAR model. The model accommodates changes in low and high data frequencies and incorporates endogenous time-varying transition matrices of country-specific Markov chains. These country-specific Markov chains depend on their own past history and the history of other chains, thus allowing for interconnections between cycles, and an endogenous common Eurozone cycle is derived by aggregating the country-specific cycles. The model is estimated using a simulation based Bayesian approach in which an efficient multi-move sampling algorithm is defined to draw time-varying Markov-switching chains. Using industrial production growth and credit spread data for all countries, several empirical results have emerged. Recession, slow gro wth and expansion are empirically identified as three regimes with slow growth becoming persistent in the Eurozone in recent years different from the US. The Eurozone and the US regimes appear not fully synchronized, with evidence of more recessions in the Eurozone. Second, turning point analysis indicates larger synchronization at the beginning of the Great Financial Crisis: this shock affects the US first, leading the Eurozone cycle, and spreads then rapidly among these economies. Third, amplification effects influence recession probabilities for Eurozone countries when shocks occur. The evidence is different for the US where this reinforcement does not exist. In recent years there are more imbalances among regimes in Eurozone countries. Fourth, a credit shock results in substantial negative industrial production growth for several months in Germany, Spain and the US.

On transition matrices of Markov chains corresponding to Hamiltonian cycles

In this paper, we present some algebraic properties of a particular class of probability transition matrices, namely, Hamiltonian transition matrices. Each matrix P in this class corresponds to a Hamiltonian cycle in a given graph G on n nodes and to an irreducible, periodic, Markov chain. We show that a number of important matrices traditionally associated with Markov chains, namely, the stationary, fundamental, deviation and the hitting time matrix all have elegant expansions in the first n−1 powers of P , whose coefficients can be explicitly derived. We also consider the resolvent-like matrices associated with any given Hamiltonian cycle and its reverse cycle and prove an identity about the product of these matrices.

Interactions between Eurozone and US Booms and Busts: A Bayesian Panel Markov-Switching VAR Model

Interactions between the eurozone and US booms and busts and among major eurozone economies are analyzed by introducing a panel Markov-switching VAR model well suitable for a multi-country cyclical analysis. The model accommodates changes in low and high data frequencies and endogenous time-varying transition matrices of the country-specific Markov chains. The transition matrix of each Markov chain depends on its own past history and on the history of the other chains, thus allowing for modeling of the interactions between cycles. An endogenous common eurozone cycle is derived by aggregating country-specific cycles. The model is estimated using a simulation based Bayesian approach in which an efficient multi-move strategy algorithm is defined to draw common time-varying Markov-switching chains. Our results show that the US and eurozone cycles are not fully synchronized over the 1991-2013 sample period, with evidence of more recessions in the Eurozone. Shocks affect the US 1-quarter in advance of the eurozone, but these spread very rapidly among economies. An increase in the number of eurozone countries in recession increases the probability of the US to stay within recession, while the US recession indicator has a negative impact on the probability to stay in recession for eurozone countries. Turning point analysis shows that the cycles of Germany, France and Italy are closer to the US cycle than other countries. Belgium, Spain, and Germany, provide more timely information on the aggregate recession than Netherlands and France.

Interactions between Eurozone and US Booms and Busts: A Bayesian Panel Markov-Switching VAR Model

markdownabstract__Abstract__ Interactions between the eurozone and US booms and busts and among major eurozone economies are analyzed by introducing a panel Markov-switching VAR model well suitable for a multi-country cyclical analysis. The model accommodates changes in low and high data frequencies and endogenous time-varying transition matrices of the country-specific Markov chains. The transition matrix of each Markov chain depends on its own past history and on the history of the other chains, thus allowing for modeling of the interactions between cycles. An endogenous common eurozone cycle is derived by aggregating country-specific cycles. The model is estimated using a simulation based Bayesian approach in which an efficient multi-move strategy algorithm is defined to draw common time-varying Markov-switching chains. Our results show that the US and eurozone cycles are not fully synchronized over the 1991-2013 sample period, with evidence of more recessions in the Eurozone. Shocks affect the US 1-quarter in advance of the eurozone, but these spread very rapidly among economies. An increase in the number of eurozone countries in recession increases the probability of the US to stay within recession, while the US recession indicator has a negative impact on the probability to stay in recession for eurozone countries. Turning point analysis shows that the cycles of Germany, France and Italy are closer to the US cycle than other countries. Belgium, Spain, and Germany, provide more timely information on the aggregate recession than Netherlands and France.

Interactions between Eurozone and US Booms and Busts: A Bayesian Panel Markov-Switching VAR Model

Interactions between eurozone and United States booms and busts and among major eurozone economies are analyzed by introducing a panel Markov-switching VAR model. The model is well suitable for a multi-country cyclical analysis and accommodates changes in low and high data frequencies and endogenous time-varying transition matrices of the country-specific Markov chains. The transition matrix of each Markov chain depends on its own past history and on the history of other chains, thus allowing for modelling the interactions between cycles. An endogenous common eurozone cycle is derived by aggregating country-specific cycles. The model is estimated using a simulation based Bayesian approach in which an efficient multi-move algorithm is defined to draw time-varying Markov-switching chains. Using real and financial data on industrial production growth and credit spread for all countries, our main empirical results are as follows. Recession, slow recovery and expansion are empirically identified as three regimes with slow recovery becoming persistent in the eurozone in recent years differing from the US. US and eurozone cycles are not fully synchronized over the 1991-2013 period, with evidence of more recessions in the eurozone, in particular during the 90’s. Larger synchronization across regions occurs at beginning of the financial crisis but recently more heterogeneity takes place. Cluster analysis yields a group of core countries: Germany, France and Netherlands and a group of peripheral countries Spain and Italy. Reinforcement effects in the recession probabilities and in the probabilities of exiting recessions occur for both eurozone and US with substantial differences in phase transitions within the eurozone. Finally, credit spreads provide accurate predictive content for business cycle fluctuations. A credit shock results in statistically significant negative industrial production growth for several months in Germany, Spain and US. Our empirical result may serve as important information for the specification of a coordinated policy between the eurozone and the US and within the eurozone.

Comparative Monte Carlo efficiency by Monte Carlo analysis

We propose a modified power method for computing the subdominant eigenvalue λ{2} of a matrix or continuous operator. While useful both deterministically and stochastically, we focus on defining simple Monte Carlo methods for its application. The methods presented use random walkers of mixed signs to represent the subdominant eigenfunction. Accordingly, the methods must cancel these signs properly in order to sample this eigenfunction faithfully. We present a simple procedure to solve this sign problem and then test our Monte Carlo methods by computing λ{2} of various Markov chain transition matrices. As |λ{2}| of this matrix controls the rate at which Monte Carlo sampling relaxes to a stationary condition, its computation also enabled us to compare efficiencies of several Monte Carlo algorithms as applied to two quite different types of problems. We first computed λ{2} for several one- and two-dimensional Ising models, which have a discrete phase space, and compared the relative efficiencies of the Metropolis and heat-bath algorithms as functions of temperature and applied magnetic field. Next, we computed λ{2} for a model of an interacting gas trapped by a harmonic potential, which has a mutidimensional continuous phase space, and studied the efficiency of the Metropolis algorithm as a function of temperature and the maximum allowable step size Δ. Based on the λ{2} criterion, we found for the Ising models that small lattices appear to give an adequate picture of comparative efficiency and that the heat-bath algorithm is more efficient than the Metropolis algorithm only at low temperatures where both algorithms are inefficient. For the harmonic trap problem, we found that the traditional rule of thumb of adjusting Δ so that the Metropolis acceptance rate is around 50% is often suboptimal. In general, as a function of temperature or Δ , λ{2} for this model displayed trends defining optimal efficiency that the acceptance ratio does not. The cases studied also suggested that Monte Carlo simulations for a continuum model are likely more efficient than those for a discretized version of the model.

Maximum a posteriori estimation for Markov chains based on Gaussian Markov random fields

In this paper, we present a Gaussian Markov random field (GMRF) model for the transition matrices (TMs) of Markov chains (MCs) by assuming the existence of a neighborhood relationship between states, and develop the maximum a posteriori (MAP) estimators under different observation conditions. Unlike earlier work on TM estimation, our method can make full use of the similarity between different states to improve the estimated accuracy, and the estimator can be performed very efficiently by solving a convex programming problem. In addition, we discuss the parameter choice of the proposed model, and introduce a Monte Carlo cross validation (MCCV) method. The numerical simulations of a diffusion process are employed to show the effectiveness of the proposed models and algorithms.

Inter-regional output distribution: a comparison of Russian and Chinese experience

Several studies report increasing inter-regional inequality in transition countries over the course of economic reforms, but most of them fail to look at the underlying dynamics. Using the cases of Russia and China, this article analyses the evolution of inter-regional output distribution during economic transition. One non-parametric method, kernel density estimation, and one parametric method, a Markov chain transition matrix, are used to evaluate the shape of the inter-regional output distribution and to evaluate regions' mobility within this distribution. Estimated distributions for both countries are skewed with long right tails. Whereas the distribution for Russian regions shows multiple modes, the hypothesis of unimodality could not be rejected for Chinese regions over the last two decades. Stationary distributions of the Markov chain transition matrices support this finding. It turns out that increasing inequality and multimodality in both countries are driven by a few outliers with very distinct characteristics.

Determinants and Longest Cycles of Graphs

We consider the Hamiltonian cycle problem on a given graph $G$. With such a graph we can associate a family $\mathcal{F}$ of probability transition matrices of Markov chains whose entries represent the probabilities of traversing corresponding arcs of the graph. When the underlying graph is Hamiltonian, we show the transition probability matrix induced by a Hamiltonian cycle maximizes—over $\mathcal{F}$—the determinant of a matrix that is a rank-one correction of the generator matrix of a Markov chain. In the case when the graph does not possess a Hamiltonian cycle, the above maximization yields a transition matrix of a chain with a longest simple cycle (in $G$) comprising that chain's unique ergodic class. These problems also have analogous eigenvalue interpretations.

Analysis of protein sequences and their secondary structures based on transition matrices

Protein database is growing rapidly, but it is difficult to obtain information from protein sequences directly. Therefore, many kinds of methods have been proposed to analyze the protein sequences, the existing methods have their limitations to numerically characterize the protein sequences exactly. Here, we regard a protein sequence as a discrete-time Markov chain and construct transition matrices to numerically characterize it. Based on the properties of Markov chains, we predict the yesterday’s, today’s, and tomorrow’s distributions of every amino acid, from which we can analyze the similarities of different species in the past or in the future. Meanwhile, we give a simple way to evaluate the methods for protein secondary structure prediction.

Aggregate economic fluctuations and regional convergence: the Finnish case 1988-95

This paper analyses the connection between aggregate economic fluctuations and regional productivity convergence in Finland during the period 1988–95. Markov chain transition matrices and mobility indices were used to examine the regional distribution of productivity. The results indicated that high intra-distribution mobility occurred during booms, when regional convergence potential was also at its highest. Conversely, recession years were characterized by much lower mobility and a more divergent regional pattern. These findings bear important implications in terms of regional policy planning, as it seems that poor regions do not manage to keep up with the rich ones during slumps, whereas regional disparities diminish naturally during boom years.

The Maclaurin series for performance functions of Markov chains

We derive formulas for the first- and higher-order derivatives of the steady state performance measures for changes in transition matrices of irreducible and aperiodic Markov chains. Using these formulas, we obtain a Maclaurin series for the performance measures of such Markov chains. The convergence range of the Maclaurin series can be determined. We show that the derivatives and the coefficients of the Maclaurin series can be easily estimated by analysing a single sample path of the Markov chain. Algorithms for estimating these quantities are provided. Markov chains consisting of transient states and multiple chains are also studied. The results can be easily extended to Markov processes. The derivation of the results is closely related to some fundamental concepts, such as group inverse, potentials, and realization factors in perturbation analysis. Simulation results are provided to illustrate the accuracy of the single sample path based estimation. Possible applications to engineering problems are discussed.

The Maclaurin series for performance functions of Markov chains

We derive formulas for the first- and higher-order derivatives of the steady state performance measures for changes in transition matrices of irreducible and aperiodic Markov chains. Using these formulas, we obtain a Maclaurin series for the performance measures of such Markov chains. The convergence range of the Maclaurin series can be determined. We show that the derivatives and the coefficients of the Maclaurin series can be easily estimated by analysing a single sample path of the Markov chain. Algorithms for estimating these quantities are provided. Markov chains consisting of transient states and multiple chains are also studied. The results can be easily extended to Markov processes. The derivation of the results is closely related to some fundamental concepts, such as group inverse, potentials, and realization factors in perturbation analysis. Simulation results are provided to illustrate the accuracy of the single sample path based estimation. Possible applications to engineering problems are discussed.

Statistical Significance Test for Transition Matrices of Atmospheric Markov Chains

Low-frequency variability of large-scale atmospheric dynamics can be represented schematically by a Markov chain of multiple flow regimes. This Markov chain contains useful information for the long-range forecaster, provided that the statistical significance of the associated transition matrix can be reliably tested. Monte Carlo simulation yields a very reliable significance test for the elements of this matrix. The results of this test agree with previously used empirical formulae when each cluster of maps identified as a distinct flow regime is sufficiently large and when they all contain a comparable number of maps. Monte Carlo simulation provides a more reliable way to test the statistical significance of transitions to and from small clusters. It can determine the most likely transitions, as well as the most unlikely ones, with a prescribed level of statistical significance.