Simple Markov Research Articles

Bayesian Multi-level Mixed-effects Model for Influenza Dynamics

Abstract Influenza A viruses (IAV) are the only influenza viruses known to cause flu pandemics. Understanding the evolution of different sub-types of IAV on their natural hosts is important for preventing and controlling the virus. We propose a mechanism-based Bayesian multi-level mixed-effects model for characterising influenza viral dynamics, described by a set of ordinary differential equations (ODE). Both strain-specific and subject-specific random effects are included for the ODE parameters. Our models can characterise the common features in the population while taking into account the variations among individuals. The random effects selection is conducted at strain level through re-parameterising the covariance parameters of the corresponding random effect distribution. Our method does not need to solve ODE directly. We demonstrate that the posterior computation can proceed via a simple and efficient Markov chain Monte Carlo algorithm. The methods are illustrated using simulated data and a real data from a study relating virus load estimates from influenza infections in ducks.

The promises (and perils) of control-contingent forward guidance

We develop a model with control-contingent forward guidance: the central bank explicitly anchors future policy announcements to short run inflation. Even though the model features past promises, we compute a closed form solution using a simple Markov chain representation. This allows us to show analytically that control-contingent forward guidance can rid the model of sunspot liquidity traps. The same holds for a policy of price level targeting, which emerges as a special case. Finally, we leverage this new framework to formally show that announced interest rates are only a means to an end: what truly matters is expected inflation.

Graph-combinatorial approach for large deviations of Markov chains

We consider discrete-time Markov chains and study large deviations of the pair empirical occupation measure, which is useful to compute fluctuations of pure-additive and jump-type observables. We provide an exact expression for the finite-time moment generating function, which is split in cycles and paths contributions, and scaled cumulant generating function of the pair empirical occupation measure via a graph-combinatorial approach. The expression obtained allows us to give a physical interpretation of interaction and entropic terms, and of the Lagrange multipliers, and may serve as a starting point for sub-leading asymptotics. We illustrate the use of the method for a simple two-state Markov chain.

Chaotic diffusion of dissipative solitons: From anti-persistent random walks to Hidden Markov Models

In previous publications, we showed that the incremental process of the chaotic diffusion of dissipative solitons in a prototypical complex Ginzburg-Landau equation, known, e.g., from nonlinear optics, is governed by a simple Markov process leading to an Anti-Persistent Random Walk of motion or by a more complex Hidden Markov Model with continuous output densities. In this article, we reveal the transition between these two models by examining the dependence of the soliton dynamics on the main bifurcation parameter of the cubic-quintic Ginzburg-Landau equation, and by identifying the underlying hidden Markov processes. These models capture the non-trivial decay of correlations in jump widths and sequences of symbols representing the symbolic dynamics of short and long jumps, the statistics of anti-persistent walk episodes, and the multimodal density of the jump widths. We demonstrate that there exists a physically meaningful reduction of the dynamics of an infinite-dimensional deterministic system to one of a probabilistic finite state machine and provide a deeper understanding of the soliton dynamics under parameter variation of the underlying nonlinear dynamics.

Conditional multivariate distributions of phase-type for a finite mixture of Markov jump processes given observations of sample path

This paper presents conditional joint probability distributions of first-exit times to absorbing sets of a finite mixture of right-continuous Markov jump processes given observations of its sample path. Each underlying Markov jump process comprising the mixture has its own intensity matrix and is defined on the same finite state space with overlapping absorbing sets. The main distinctive aspect of the mixture is that despite the underlying processes have the Markov property, the mixture itself in general does not. Conditional transition matrix and the joint probability distribution of the first-exit times form non-stationary functions of time with ability to capture path dependence when conditioning on available information (either full or partial) of the sample path. In particular, initial profile of the joint distribution forms a generalized mixture of the multivariate phase-type distributions of Assaf et al. (1984). When the underlying Markov processes have the same intensity matrix, in which case the mixture becomes a simple Markov jump process, the path dependence and non-stationary properties are removed from the transition matrix and the joint distribution, while the initial profile distribution reduces to Assaf et al. (1984). Some explicit and numerical examples are discussed to illustrate the results.

Brownian regularity for the Airy line ensemble, and multi-polymer watermelons in Brownian last passage percolation

The Airy line ensemble is a positive-integer indexed system of random continuous curves whose finite dimensional distributions are given by the multi-line Airy process. It is a natural object in the KPZ universality class: for example, its highest curve, the Airy 2 _2 process, describes after the subtraction of a parabola the limiting law of the scaled energy of a geodesic running from the origin to a variable point on an anti-diagonal line in such problems as Poissonian last passage percolation. The ensemble of curves resulting from the Airy line ensemble after the subtraction of the same parabola enjoys a simple and explicit spatial Markov property, the Brownian Gibbs property. In this paper, we employ the Brownian Gibbs property to make a close comparison between the Airy line ensemble’s curves after affine shift and Brownian bridge, proving the finiteness of a superpolynomially growing moment bound on Radon-Nikodym derivatives. We also determine the value of a natural exponent describing in Brownian last passage percolation the decay in probability for the existence of several near geodesics that are disjoint except for their common endpoints, where the notion of ‘near’ refers to a small deficit in scaled geodesic energy, with the parameter specifying this nearness tending to zero. To prove both results, we introduce a technique that may be useful elsewhere for finding upper bounds on probabilities of events concerning random systems of curves enjoying the Brownian Gibbs property. Several results in this article play a fundamental role in a further study of Brownian last passage percolation in three companion papers (Hammond 2017a,b,c), in which geodesic coalescence and geodesic energy profiles are investigated in scaled coordinates.

Probabilistic Inference with Polymerizing Biochemical Circuits.

Probabilistic inference—the process of estimating the values of unobserved variables in probabilistic models—has been used to describe various cognitive phenomena related to learning and memory. While the study of biological realizations of inference has focused on animal nervous systems, single-celled organisms also show complex and potentially “predictive” behaviors in changing environments. Yet, it is unclear how the biochemical machinery found in cells might perform inference. Here, we show how inference in a simple Markov model can be approximately realized, in real-time, using polymerizing biochemical circuits. Our approach relies on assembling linear polymers that record the history of environmental changes, where the polymerization process produces molecular complexes that reflect posterior probabilities. We discuss the implications of realizing inference using biochemistry, and the potential of polymerization as a form of biological information-processing.

Sampling from the low temperature Potts model through a Markov chain on flows

AbstractIn this article, we consider the algorithmic problem of sampling from the Potts model and computing its partition function at low temperatures. Instead of directly working with spin configurations, we consider the equivalent problem of sampling flows. We show, using path coupling, that a simple and natural Markov chain on the set of flows is rapidly mixing. As a result, we find a ‐approximate sampling algorithm for the Potts model at low enough temperatures, whose running time is bounded by for graphs with edges.

Application of projection operator method to coarse-grained dynamics with transient potential.

We show that the coarse-grained dynamics model with the time-dependent and fluctuating potential (transient potential) can be derived from the microscopic Hamiltonian dynamics. The concept of the transient potential was first introduced rather phenomenologically, and its relation to the underlying microscopic dynamics has not been clarified yet. This is in contrast to the generalized Langevin equation, the relation of which to the microscopic dynamics is well-established. In this work, we show that the dynamic equationswith the transient potential can be derived for the coupled oscillator model, without any approximations. It is known that the dynamics of the coupled oscillator model can be exactly described by the generalized Langevin-type equations. This fact implies that the dynamic equationswith the transient potential can be utilized as a coarse-grained dynamics model in a similar way to the generalized Langevin equation. Then we show that the dynamic equationsfor the transient potential can also be formally derived for the microscopic Hamiltonian dynamics, without any approximations. We use the projection operator method for the coarse-grained variables and transient potential. The dynamic equationsfor the coarse-grained positions and momenta are similar to those in the Hamiltonian dynamics, but the interaction potential is replaced by the transient potential. The dynamic equationfor the transient potential is the generalized Langevin equationwith the memory effect. Our result justifies the use of the transient potential to describe the coarse-grained dynamics. We propose several approximations to obtain the simplified dynamics model. We show that, under several approximations, the dynamic equationfor the transient potential reduces to the relatively simple Markovian dynamic equationfor the potential parameters. We also show that with several additional approximations, the approximate dynamics model further reduces to the Markovian Langevin-type equationswith the transient potential.

Ordinal synchronization and typical states in high-frequency digital markets

In this paper we study Algorithmic High-Frequency Financial Markets as dynamical networks. After an individual analysis of 24 stocks of the US market during a trading year of fully automated transactions by means of ordinal pattern series, we define an information-theoretic measure of pairwise synchronization for time series which allows us to study this subset of the US market as a dynamical network. We apply to the resulting network a couple of clustering algorithms in order to detect collective market states, characterized by their degree of centralized or decentralized synchronicity. This collective analysis has shown to reproduce, classify and explain the anomalous behavior previously observed at the individual level. We also find two whole coherent seasons of highly centralized and decentralized synchronicity, respectively. Finally, we model these states dynamics through a simple Markov model.

The Application of Homogeneous Markov Chains in the Analysis of the Technical State of Tractors

Planning maintenance activities on tractors can be improved and become more effective when using a mathematical apparatus for modelling the processes of the operation and maintenance of machines and tractors using homogeneous Markov chains. For this purpose, it is necessary to switch from the traditional methods for determining the indicators of their technical state to dynamic methods based on the theory of random processes. The most promising method is the use of homogeneous Markov chains. Based on the accumulated statistical data, it is possible to determine the probabilities of transition from one technical state to another. The values of the transition probabilities are used to construct a probability matrix describing a simple Markov chain. For tractors, taking into account the approaching of their limit state, which is characterized by the full depletion of resources, an absorbing Markov chain is best suited. The algorithms for the decomposition and study of the properties of reducible absorbing Markov chains, which allow obtaining useful information for an efficient operation, maintenance and repair of tractors and complex agricultural machinery, are presented.

Generating Facial Character: A Systematic Method Accumulating Expressive Histories

Abstract The author presents a method to simulate facial character development by accumulating an expressive history onto a face. The model analytically combines facial features from Paul Ekman's seven universal facial expressions using a simple Markov chain algorithm. The output is a series of 3D digital faces created in Blender with Python. The results show that systematically imprinting features from emotional expressions onto a neutral face transforms it into one with distinct character. This method could be applied to creative works that depend on character creation, ranging from figurative sculpture to game design, and allows the creator to incorporate chance into the creative process. The author demonstrates the method's application to sculpture with ceramic casts of generated faces.

Probabilistic representation of integration by parts formulae for some stochastic volatility models with unbounded drift

In this paper, we establish a probabilistic representation as well as some integration by parts formulae for the marginal law at a given time maturity of some stochastic volatility model with unbounded drift. Relying on a perturbation technique for Markov semigroups, our formulae are based on a simple Markov chain evolving on a random time grid for which we develop a tailor-made Malliavin calculus. Among other applications, an unbiased Monte Carlo path simulation method stems from our formulas so that it can be used in order to numerically compute with optimal complexity option prices as well as their sensitivities with respect to the initial values or Greeks in finance, namely the Delta and Vega, for a large class of non-smooth European payoff. Numerical results are proposed to illustrate the efficiency of the method.

Multiple-Timescale Neural Networks: Generation of History-Dependent Sequences and Inference Through Autonomous Bifurcations.

Sequential transitions between metastable states are ubiquitously observed in the neural system and underlying various cognitive functions such as perception and decision making. Although a number of studies with asymmetric Hebbian connectivity have investigated how such sequences are generated, the focused sequences are simple Markov ones. On the other hand, fine recurrent neural networks trained with supervised machine learning methods can generate complex non-Markov sequences, but these sequences are vulnerable against perturbations and such learning methods are biologically implausible. How stable and complex sequences are generated in the neural system still remains unclear. We have developed a neural network with fast and slow dynamics, which are inspired by the hierarchy of timescales on neural activities in the cortex. The slow dynamics store the history of inputs and outputs and affect the fast dynamics depending on the stored history. We show that the learning rule that requires only local information can form the network generating the complex and robust sequences in the fast dynamics. The slow dynamics work as bifurcation parameters for the fast one, wherein they stabilize the next pattern of the sequence before the current pattern is destabilized depending on the previous patterns. This co-existence period leads to the stable transition between the current and the next pattern in the non-Markov sequence. We further find that timescale balance is critical to the co-existence period. Our study provides a novel mechanism generating robust complex sequences with multiple timescales. Considering the multiple timescales are widely observed, the mechanism advances our understanding of temporal processing in the neural system.

Processor Sharing Queues With Impatient Customers and State-Dependent Rates

We study queues with impatient customers and Processor Sharing (PS) discipline as well as other variants of PS discipline, namely, Discriminatory Processor Sharing (DPS) and Generalized Processor Sharing (GPS) disciplines, where customers have deadlines until the end of service (DES). Customers arrive according to a state-dependent Poisson process and have general impatience. Customers have exponential service times with state-dependent service rates. Analytical methods based on simple Markov chains are given for the performance analysis of such queues. The principal measures of performance are the steady-state probability of missing deadline and the steady-state probability of blocking. Similar results are obtained for related queues with Random Order Service (ROS) discipline where customers have deadlines until the beginning of service (DBS). In view of a lack of exact analytical results for First Come First Served (FCFS) queues with state-dependent rates, a highly accurate approximation method is also given for these latter queues. The efficacy and accuracy of the approach are illustrated by some numerical examples and simulation experiments.

Markov modelling of treatment response in a 30-year cohort study of newly diagnosed epilepsy.

People with epilepsy have variable and dynamic trajectories in response to antiseizure medications. Accurately modelling long-term treatment response will aid prognostication at the individual level and health resource planning at the societal level. Unfortunately, a robust model is lacking. We aimed to develop a Markov model to predict the probability of future seizure-freedom based on current seizure state and number of antiseizure medication regimens trialled. We included 1795 people with newly diagnosed epilepsy who attended a specialist clinic in Glasgow, Scotland, between July 1982 and October 2012. They were followed up until October 2014 or death. We developed a simple Markov model, based on current seizure state only, and a more detailed model, based on both current seizure state and number of antiseizure medication regimens trialled. Sensitivity analyses were performed for the regimen-based states model to examine the effect of regimen changes due to adverse effects. The model was externally validated in a separate cohort of 455 newly diagnosis epilepsy patients seen in Perth, Australia, between May 1999 and May 2016. Our models suggested that once seizure-freedom was achieved, it was likely to persist, regardless of the number of antiseizure medications trialled to reach that point. The likelihood of achieving long-term seizure-freedom was highest with the first antiseizure medication regimen, at approximately 50%. The chance of achieving seizure-freedom fell with subsequent regimens. Fluctuations between seizure-free and not seizure-free states were highest earlier on but decreased with chronicity of epilepsy. Seizure-freedom/recurrence risk tables were constructed with these probability data, similar to cardiovascular risk tables. Sensitivity analyses showed that the general trends and conclusions from the base model were maintained despite perturbing the model and input data with regimen changes due to adverse effects. Quantitative comparison with the external validation cohort showed excellent consistency at Year 1, good at Year 3 and moderate at Year 5. Quantitative models, as used in this study, can provide pertinent clinical insights that are not apparent from simple statistical analysis alone. Attaining seizure freedom at any time in a patient's epilepsy journey will confer durable benefit. Seizure-freedom risk tables may be used to individualize the prediction of future seizure control trajectory.

An Absorbing Markov Chain Model to Predict Dairy Cow Calving Time.

Abnormal behavioral changes in the regular daily mobility routine of a pregnant dairy cow can be an indicator or early sign to recognize when a calving event is imminent. Image processing technology and statistical approaches can be effectively used to achieve a more accurate result in predicting the time of calving. We hypothesize that data collected using a 360-degree camera to monitor cows before and during calving can be used to establish the daily activities of individual pregnant cows and to detect changes in their routine. In this study, we develop an augmented Markov chain model to predict calving time and better understand associated behavior. The objective of this study is to determine the feasibility of this calving time prediction system by adapting a simple Markov model for use on a typical dairy cow dataset. This augmented absorbing Markov chain model is based on a behavior embedded transient Markov chain model for characterizing cow behavior patterns during the 48 h before calving and to predict the expected time of calving. In developing the model, we started with an embedded four-state Markov chain model, and then augmented that model by adding calving as both a transient state, and an absorbing state. Then, using this model, we derive (1) the probability of calving at 2 h intervals after a reference point, and (2) the expected time of calving, using their motions between the different transient states. Finally, we present some experimental results for the performance of this model on the dairy farm compared with other machine learning techniques, showing that the proposed method is promising.

An alternative interpretation of residual feed intake by phenotypic recursive relationships in dairy cattle

There has been increasing interest in residual feed intake (RFI) as a measure of net feed efficiency in dairy cattle. Residual feed intake phenotypes are obtained as residuals from linear regression encompassing relevant factors (i.e., energy sinks) to account for body tissue mobilization. By rearranging the single-trait linear regression, we showed a causal RFI interpretation underlying the linear regression for RFI. It postulates recursive effects in energy allocation from energy sinks on dry matter intake, but the feedback or simultaneous effects are nonexistent. A Bayesian recursive structural equation model was proposed for directly predicting RFI and energy sinks and estimating relevant genetic parameters simultaneously. A simplified Markov chain Monte Carlo algorithm was described. The recursive model is asymptotically equivalent to one-step linear regression for RFI, yet extends the analytical capacity to multiple-trait analysis.

The Newcomb–Benford law: Scale invariance and a simple Markov process based on it

The Newcomb–Benford law, also known as the first-digit law, gives the probability distribution associated with the first digit of a dataset so that, for example, the first significant digit has a probability of 30.1% of being 1 and 4.58% of being 9. This law can be extended to the second and next significant digits. This article presents an introduction to the discovery of the law and its derivation from the scale invariance property as well as some applications and examples. Additionally, a simple model of a Markov process inspired by scale invariance is proposed. Within this model, it is proved that the probability distribution irreversibly converges to the Newcomb–Benford law, in analogy to the irreversible evolution toward equilibrium of physical systems in thermodynamics and statistical mechanics.

Analysis Estimating the Potential Cost Impact of Utilizing Flash Glucose Monitoring with Optional Alarms in Swedish Adults with Diabetes with Impaired Awareness of Hypoglycaemia, Using Intensive Insulin

IntroductionThe objective of this analysis was to estimate the costs associated with using flash glucose monitoring with optional alarms as a replacement for either traditional continuous glucose monitoring (CGM) or routine self-monitoring of blood glucose (SMBG) in adults with diabetes and impaired awareness of hypoglycaemia (IAH) who use intensified insulin therapy, from a Swedish payer perspective, applying assumptions to simulate hypothetical scenarios.MethodsA simple two-state cohort Markov model was used to calculate the cost per patient treated over a 3-year period, capturing the risk of severe hypoglycaemic events requiring medical assistance and non-adherence using quarterly Markov cycles. The costs considered were those for glucose monitoring and resource use to treat severe hypoglycaemic events. Cost inputs were sourced from Swedish price lists, manufacturer data and resource use reported in the control arm of the HypoDE study. Targeted literature searches were run in PubMed to source the clinical inputs. Uncertainty in the model was considered through one-way sensitivity analysis and scenario analysis.ResultsOver 3 years, flash monitoring with optional alarms resulted in potential cost-savings of Swedish krona (SEK) 7708 and SEK 69,908 per patient when compared to routine SMBG or CGM respectively. Sensitivity and scenario analyses were largely supportive of this conclusion with respect to SMBG, and large cost-savings were consistent across all sensitivity and scenario analyses with respect to CGM.ConclusionUtilizing flash monitoring with optional alarms is potentially a cost-saving treatment strategy compared to routine SMBG or traditional CGM in adults with diabetes using intensive insulin and IAH from a Swedish payer perspective. Future studies in the IAH population will help to assess more precisely the relative cost impact of flash glucose monitoring with optional alarms compared with SMBG and traditional CGM.Supplementary InformationThe online version contains supplementary material available at 10.1007/s13300-021-01099-7.