Markov Chain Representation Research Articles

Seasonal variability and stochastic branching process in malaria outbreak probability

Background: Malaria is the world’s most fatal and challenging parasitic disease, caused by the Plasmodium parasite, which is transmitted to humans by the bites of infected female mosquitoes. Bangladesh is the most vulnerable region to spread malaria because of its geographic position. In this paper, we have considered the dynamics of vector-host models and observed the stochastic behavior. This study elaborates on the seasonal variability and calculates the probability of disease outbreaks. Methods: We present a model for malaria disease transmission and develop its corresponding continuous-time Markov chain (CTMC) representation. The proposed vector-host models illustrate the malaria transmission model along with sensitivity analysis. The deterministic model with CTMC curves is depicted to show the randomness in real scenarios. Sequentially, we expand these studies to a time-varying stochastic vector-host model that incorporates seasonal variability. Phase plane analysis is conducted to explore the characteristics of the disease, examine interactions among various compartments, and evaluate the impact of key parameters. The branching process approximation is developed for the corresponding vector-host model to calculate the probability outbreak. Numerous numerical results are accomplished to observe the analytical investigation. Results: Seasonality and contact patterns affect the dynamics of disease outbreaks. The numerical illustration provides that the probability of a disease outbreak depends on the infected host or vector. Additionally, periodic transmission rates have a great influence on the probability outbreak. The basic reproduction number (R0) is derived, which is the main justification for studying the dynamical behavior of epidemic models. Conclusions: Seasonal variability significantly impacts malaria transmission, and the probability of disease outbreaks is influenced by time and the initial number of infected individuals. Moreover, the branching process approximation is applicable when the population size is large enough and the basic reproduction number is less than 1. In the future, such analysis can help decision-makers understand the impact of various parameters and their stochastic behavior in the vector-host model to prevent such types of disease outbreaks.

Network representations of attractors for change point detection

A common approach to monitoring the status of physical and biological systems is through the regular measurement of various system parameters. Changes in a system’s underlying dynamics manifest as changes in the behaviour of the observed time series. For example, the transition from healthy cardiac activity to ventricular fibrillation results in erratic dynamics in measured electrocardiogram (ECG) signals. Identifying these transitions—change point detection—can be valuable in preparing responses to mitigate the effects of undesirable system changes. Here, we present a data-driven method of detecting change points using a phase space approach. Delay embedded trajectories are used to construct an ‘attractor network’, a discrete Markov-chain representation of the system’s attractor. Once constructed, the attractor network is used to assess the level of surprise of future observations where unusual movements in phase space are assigned high surprise scores. Persistent high surprise scores indicate deviations from the attractor and are used to infer change points. Using our approach, we find that the attractor network is effective in automatically detecting the onset of ventricular fibrillation (VF) from observed ECG data. We also test the flexibility of our method on artificial data sets and demonstrate its ability to distinguish between normal and surrogate time series.

Multi-opinion Markovian agent networks: Parametrization, second order moment and social power

This paper extends the theory of Markovian multi-agent opinion networks, previously studied in the binary opinion case, to the situation of multiple opinions. The first step is the definition of a suitable canonical representation of a multi-state Markov chain, to describe the behavior of any non-interacting agent in terms of its prejudice. Based on this parametrization, the time evolution of both first- and second-order moments of the opinion shares when the agents are connected in a social network is completely characterized, both in transient and at steady-state. The steady-state analysis allows one to introduce an appropriate notion of marginal social power, measuring the sensitivity of the average network opinion to the agents’ prejudices.

The promises (and perils) of control-contingent forward guidance

We develop a model with control-contingent forward guidance: the central bank explicitly anchors future policy announcements to short run inflation. Even though the model features past promises, we compute a closed form solution using a simple Markov chain representation. This allows us to show analytically that control-contingent forward guidance can rid the model of sunspot liquidity traps. The same holds for a policy of price level targeting, which emerges as a special case. Finally, we leverage this new framework to formally show that announced interest rates are only a means to an end: what truly matters is expected inflation.

Phase-type models for competing risks, with emphasis on identifiability issues

We first review some main results for phase-type distributions, including a discussion of Coxian distributions and their canonical representations. We then consider the extension of phase-type modeling to cover competing risks. This extension involves the consideration of finite state Markov chains with more than one absorbing state, letting each absorbing state correspond to a particular risk. The non-uniqueness of Markov chain representations of phase-type distributions is well known. In the paper we study corresponding issues for the competing risks case with the aim of obtaining identifiable parameterizations. Statistical inference for the Coxian competing risks model is briefly discussed and some real data are analyzed for illustration.

Re-exploring the Penney-Ante Game

We propose a single loop diagram and use it to devise a single loop matrix method to computationally solve the Penney-Ante game. This method avoids the nuances of repeated use of conditional probability and Markov chain representations. We remove the limitations of Conway’s trick as applied to a fair coin and generalize the method where the coin is allowed to be biased. A uniform random number generator is used to simulate the game and formulate implicit mathematical relations to explore nontransitivity.

Unilateral 2D Markov-switching autoregressive model

For nonlinear two-dimensional spatial data, a 2D Markov-switching unilateral autoregressive model is defined. Strict stationarity is studied and parameter estimation via the variational expectation maximisation (VEM) algorithm is performed. First, a 2D Markov random field (MRF) is constructed for which an imposed causality allows to establish an analogy between this 2D MRF and the Markov chain representation. Based on the proposed 2D MRF, the 2D MS-AR model is defined under some necessary assumptions and useful notations. Finally, parameter estimation of the model is discussed paving the way to broad exploitation perspectives of the proposed 2D MS-AR processes to efficiently model several phenomena exhibiting a structural break in spatial data.

Government Spending Multipliers with the Real Cost Channel

I introduce the real cost channel in the benchmark New Keynesian model to study government spending multipliers. Even though this new model shows previous evidence that the output gap multiplier in normal times can be overestimated by ignoring the cost channel, I provide a simple Markov chain representation closed-form solution. Nevertheless, in liquidity traps, it fundamentally departs from previous implications with the nominal cost channel that the deflation pressure can be exacerbated in the presence of real cost channel and then the output gap multiplier can be lower. In particular, after the zero lower bound has subsided, the long-run government spending policy is favored since it can enlarge the short term output gap multiplier. Finally, I leverage this new framework to confirm the robust role of the real cost channel on policy multipliers.

Reduced-Order Estimation of the Uniform Completely Connected Homogeneous Influence Model (UCC-HIM)

The influence model (IM) is a discrete-time stochastic automaton that captures spatiotemporal network dynamics. It constitutes a reduced-order representation of networked Markov chains and has found broad stochastic network applications. Parameter estimation from observation data is critical for utilizing IM in real applications. The master Markov chain approach used in the literature incurs significant computational cost. In this letter, we develop an efficient estimation algorithm for a special class of IM, named the uniform completely connected homogeneous influence model (UCC-HIM), through exploiting its special network topology. Specially, we introduce a reduced-order Markov chain representation for the UCC-HIM, analyze its relationship with the master Markov chain, based on which an efficient estimation algorithm is developed. Two simulation studies verify the accuracy and computation reduction of the proposed estimation approach.

Asymptotically Optimal Change Point Detection for Composite Hypothesis in State Space Models

This paper investigates change point detection in state space models, in which the pre-change distribution f <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">θ0</sup> is given, while the post-distribution f <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">θ</sup> after change is unknown. The problem is to raise an alarm as soon as possible after the distribution changes from f <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">θ0</sup> to f <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">θ</sup> , under a restriction on the false alarms. We investigate theoretical properties of a weighted Shiryayev-Roberts-Pollak (SRP) change point detection rule in state space models. By making use of a Markov chain representation for the likelihood function, exponential embedding of the induced Markovian transition operator, nonlinear Markov renewal theory, and sequential hypothesis testing theory for Markov random walks, we show that the weighted SRP procedure is second-order asymptotically optimal. To this end, we derive an asymptotic approximation for the expected stopping time of such a stopping scheme when the change time w = 1. To illustrate our method we apply the results to two types of state space models: general state Markov chains and linear state space models. Simulation study is given to illustrate the method.

A Markov Chain Representation of Human Reasoning and Scientific Thinking

Reasoning is the most important human brain operation that leads to creative methodologies, algorithms and deductions giving way to sustainable research and development. Since the basis of reasoning in the Artificial Intelligence’s expert systems is human notions and concepts, the success of those systems depends upon the correspondence between human reasoning and their formalization. The paper at hands studies human reasoning in general and scientific thinking in particular with the help of the theory of finite Markov chains. Examples are also presented illustrating the applicability of the developed Markov chain models to real life situations.

On unilateral 2D Markov-Switching Autoregressive (MS-AR) model

For nonlinear two-dimensional spatial data, a 2D Markov-switching unilateral autoregressive model is defined. Strict stationarity is studied and parameter estimation via the variational expectation maximisation (VEM) algorithm is performed. First, a 2D Markov random field (MRF) is constructed for which an imposed causality allows to establish an analogy between this 2D MRF and the Markov chain representation. Based on the proposed 2D MRF, the 2D MS-AR model is defined under some necessary assumptions and useful notations. Finally, parameter estimation of the model is discussed paving the way to broad exploitation perspectives of the proposed 2D MS-AR processes to efficiently model several phenomena exhibiting a structural break in spatial data.

Minimizing the influence of rumors during breaking news events in online social networks

The malicious rumors have tremendously attracted a more substantial number of researchers to join the fight against the propagation of these types of information in online social networks (OSNs). The spread of rumors has a severe impact on society, which can creates political conflicts, shape public opinion and weakens their trust in governments; therefore, it must be stopped as soon as it is detected. This paper investigates the problem of minimizing the influence of malicious rumors that emerge during breaking news, which are characterized by the dissemination of a large number of malicious information over a short period. Therefore, we introduce the problem of multi-rumors influence minimization (MRIM) in OSNs and propose a solution to it. To this end, we design a multi-rumor propagation model named the HISBMmodel that captures the propagation process of multi-rumors in OSNs. Moreover, we present a new formulation of an individual’s opinion toward a rumor based on a Markov chain representation, which adds a layer of realism to the proposed model. Subsequently, we propose a dynamic blocking period (DBP) approach as a solution for the MRIM problem. The main objective is to minimize both the spread and the influence of these rumors in OSNs. The proposed method selects and blocks nodes that most likely to spread a large number of rumors and support them. Different from existing methods, the proposed solution does not block nodes for an unlimited period, but this period is estimated according to the high activity of a node in an OSN. The survival theory has been exploited in this work to provide a solution formulated from the perspective of probabilistic inference of networks. Consequently, an algorithm has been proposed based on a likelihood principle to select the target nodes, which guarantees a (1−1∕e)-approximation of the optimal solution. The experimental results show that the HISBMmodel could capture the propagation of multi-rumor propagation more accurately than classical models and provides metrics to assess the impact of rumors efficiently. Moreover, the results show the outstanding performance of the proposed approach compared to the other solution in the literature. The experimental results show that in the worst-case, the DBP achieves on an average 37.66% reduction on the impact of rumors, compared to 18.46% obtained by the second-best method. However, in the best-case the performance of the proposed method reached 93.38% where second-best method achieved only 57.65% on an average. Besides, even though when the number of rumors is high, the DBP could achieve on an average 68.01% reduction on the impact of rumors.

A Markov chain representation of the “5 E’s” instructional treatment

A Markov chain representation of the “5 E’s” instructional treatment

Semi-supervised machine-learning classification of materials synthesis procedures

Digitizing large collections of scientific literature can enable new informatics approaches for scientific analysis and meta-analysis. However, most content in the scientific literature is locked-up in written natural language, which is difficult to parse into databases using explicitly hard-coded classification rules. In this work, we demonstrate a semi-supervised machine-learning method to classify inorganic materials synthesis procedures from written natural language. Without any human input, latent Dirichlet allocation can cluster keywords into topics corresponding to specific experimental materials synthesis steps, such as “grinding” and “heating”, “dissolving” and “centrifuging”, etc. Guided by a modest amount of annotation, a random forest classifier can then associate these steps with different categories of materials synthesis, such as solid-state or hydrothermal synthesis. Finally, we show that a Markov chain representation of the order of experimental steps accurately reconstructs a flowchart of possible synthesis procedures. Our machine-learning approach enables a scalable approach to unlock the large amount of inorganic materials synthesis information from the literature and to process it into a standardized, machine-readable database.

Asymptotic Bayesian Theory of Quickest Change Detection for Hidden Markov Models

In the 1960s, Shiryaev developed a Bayesian theory of change-point detection in the i.i.d. case, which was generalized in the early 2000s by Tartakovsky and Veeravalli and recently by Tartakovsky (2017) for general stochastic models assuming a certain stability of the log-likelihood ratio process. Hidden Markov models represent a wide class of stochastic processes in a variety of applications. In this paper, we investigate the performance of the Bayesian Shiryaev change-point detection rule for hidden Markov models. We propose a set of regularity conditions under which the Shiryaev procedure is first-order asymptotically optimal in a Bayesian context, minimizing moments of the detection delay up to certain order asymptotically as the probability of false alarm goes to zero. The developed theory for hidden Markov models is based on Markov chain representation for the likelihood ratio and r -quick convergence for Markov random walks. In addition, applying Markov nonlinear renewal theory, we present a high-order asymptotic approximation for the expected delay to detection and a first-order asymptotic approximation for the probability of false alarm of the Shiryaev detection rule. We also study asymptotic properties of another popular change detection rule, the Shiryaev–Roberts rule, and provide some interesting examples.

Analysis of Cough Time Stamps from COPD Patients Using Markov Chain Analysis

The objective of this study was to identify patterns of cough events for COPD patients. Simultaneously, the study was used to develop a Matlab based graphical user interface (GUI) that enables the user to analyze time-stamps of cough data. The time stamp data was received from Philips Research. They cover 17 data sets of 16 COPD patients and were determined using a semi-automated cough detection algorithm. Cough detection ran for multiple days in the living and bed rooms of the patients. The time stamp marks the event that a cough is assumed to occur. A descriptive statistics and a Markov Chain Model was used for analysis. A pattern of cough events was described by the probability that a COPD patient is in one of three possible states at a specific hour and in another state at the next hour. To define the states, the following three characteristics were used: 1) relative frequency, 2) average value-three times standard deviation band, 3) average value-three times inter-quartile range band. Relaxation time was determined to describe the dynamics of the cough event patterns. To be precise, pattern changes were characterized by considering the time it takes for the probabilities to reach stationarity. To reduce noise, the daily dynamics of the cough events over five day periods with a four day overlap were considered. From the results, we concluded that the distribution of cough events for all data sets was skewed to the right. The developed Matlab based graphical user interface allows the user to analyze the cough events of COPD patients together with their medical history. We conclude that the relaxation time and the stationary distribution of the Markov chain representation were typical characteristics of the patterns of cough events and the cough behavior of COPD patients was patient specific and varies over time.

Lagrangian Geography of the Deep Gulf of Mexico

AbstractUsing trajectories from acoustically tracked (RAFOS) floats in the Gulf of Mexico, we construct a geography of its Lagrangian circulation within the 1500–2500-m layer. This is done by building a Markov-chain representation of the Lagrangian dynamics. The geography is composed of weakly interacting provinces that constrain the connectivity at depth. The main geography includes two provinces of near-equal areas separated by a roughly meridional boundary. The residence time is about 4.5 (3.5) years in the western (eastern) province. The exchange between these provinces is effected through a slow cyclonic circulation, which is well constrained in the western basin by preservation of f/H, where f is the Coriolis parameter and H is depth. Secondary provinces of varied shapes covering smaller areas are identified with residence times ranging from about 0.4 to 1.2 years or so. Except for the main provinces, the deep Lagrangian geography does not resemble the surface Lagrangian geography recently inferred from satellite-tracked drifter trajectories. This implies disparate connectivity characteristics with potential implications for pollutant (e.g., oil) dispersal at the surface and at depth. Support for our results is provided by a Markov-chain analysis of satellite-tracked profiling (Argo) floats, which, while forming a smaller dataset and having seemingly different water-following characteristics than the RAFOS floats, replicate the main aspects of the Lagrangian geography. Our results find further validation in independent results from a chemical tracer release experiment.

A mathematical formalism for natural selection with arbitrary spatial and genetic structure

We define a general class of models representing natural selection between two alleles. The population size and spatial structure are arbitrary, but fixed. Genetics can be haploid, diploid, or otherwise; reproduction can be asexual or sexual. Biological events (e.g.births, deaths, mating, dispersal) depend in arbitrary fashion on the current population state. Our formalism is based on the idea of genetic sites. Each genetic site resides at a particular locus and houses a single allele. Each individual contains a number of sites equal to its ploidy (one for haploids, two for diploids, etc.). Selection occurs via replacement events, in which alleles in some sites are replaced by copies of others. Replacement events depend stochastically on the population state, leading to a Markov chain representation of natural selection. Within this formalism, we define reproductive value, fitness, neutral drift, and fixation probability, and prove relationships among them. We identify four criteria for evaluating which allele is selected and show that these become equivalent in the limit of low mutation. We then formalize the method of weak selection. The power of our formalism is illustrated with applications to evolutionary games on graphs and to selection in a haplodiploid population.

Capturing the influence of geopolitical ties from Wikipedia with reduced Google matrix.

Interactions between countries originate from diverse aspects such as geographic proximity, trade, socio-cultural habits, language, religions, etc. Geopolitics studies the influence of a country’s geographic space on its political power and its relationships with other countries. This work reveals the potential of Wikipedia mining for geopolitical study. Actually, Wikipedia offers solid knowledge and strong correlations among countries by linking web pages together for different types of information (e.g. economical, historical, political, and many others). The major finding of this paper is to show that meaningful results on the influence of country ties can be extracted from the hyperlinked structure of Wikipedia. We leverage a novel stochastic matrix representation of Markov chains of complex directed networks called the reduced Google matrix theory. For a selected small size set of nodes, the reduced Google matrix concentrates direct and indirect links of the million-node sized Wikipedia network into a small Perron-Frobenius matrix keeping the PageRank probabilities of the global Wikipedia network. We perform a novel sensitivity analysis that leverages this reduced Google matrix to characterize the influence of relationships between countries from the global network. We apply this analysis to two chosen sets of countries (i.e. the set of 27 European Union countries and a set of 40 top worldwide countries). We show that with our sensitivity analysis we can exhibit easily very meaningful information on geopolitics from five different Wikipedia editions (English, Arabic, Russian, French and German).