Simple Markov Model Research Articles

A two-level Markov model for packet loss in UDP/IP-based real-time video applications targeting residential users

The packet loss characteristics of Internet paths that include residential broadband links are not well understood, and there are no good models for their behaviour. This complicates the design of real-time video applications targeting home users, since it is difficult to choose appropriate error correction and concealment algorithms without a good model for the types of loss observed. Using measurements of residential broadband networks in the UK and Finland, we show that existing models for packet loss, such as the Gilbert model and simple hidden Markov models, do not effectively model the loss patterns seen in this environment. We present a new two-level Markov model for packet loss that can more accurately describe the characteristics of these links, and quantify the effectiveness of this model. We demonstrate that our new packet loss model allows for improved application design, by using it to model the performance of forward error correction on such links.

Using Time-of-Flight Measurements for Privacy-Preserving Tracking in a Smart Room

We present a method for real-time person tracking and coarse pose recognition in a smart room using time-of-flight measurements. The time-of-flight images are severely downsampled to preserve the privacy of the occupants and simulate future applications that use single-pixel sensors in “smart” ceiling panels. The tracking algorithms use grayscale morphological image reconstruction to avoid false detections and are designed to not mistakenly detect pieces of furniture as people. A maximum-likelihood estimation method using a simple Markov model was implemented for robust pose classification. We show that the algorithms work effectively even when the sensors are spaced apart by 25 cm, using both real-world experiments and environmental simulation.

A simple Markov model of sodium channels with a dynamic threshold

Characteristics of action potential generation are important to understanding brain functioning and, thus, must be understood and modeled. It is still an open question what model can describe concurrently the phenomena of sharp spike shape, the spike threshold variability, and the divisive effect of shunting on the gain of frequency-current dependence. We reproduced these three effects experimentally by patch-clamp recordings in cortical slices, but we failed to simulate them by any of 11 known neuron models, including one- and multi-compartment, with Hodgkin-Huxley and Markov equation-based sodium channel approximations, and those taking into account sodium channel subtype heterogeneity. Basing on our voltage-clamp data characterizing the dependence of sodium channel activation threshold on history of depolarization, we propose a 3-state Markov model with a closed-to-open state transition threshold dependent on slow inactivation. This model reproduces the all three phenomena. As a reduction of this model, a leaky integrate-and-fire model with a dynamic threshold also shows the effect of gain reduction by shunt. These results argue for the mechanism of gain reduction through threshold dynamics determined by the slow inactivation of sodium channels.

Probabilistic approaches to alignment with tandem repeats

BackgroundShort tandem repeats are ubiquitous in genomic sequences and due to their complex evolutionary history pose a challenge for sequence alignment tools.ResultsTo better account for the presence of tandem repeats in pairwise sequence alignments, we propose a simple tractable pair hidden Markov model that explicitly models their presence. Using the framework of gain functions, we design several optimization criteria for decoding this model and describe resulting decoding algorithms, ranging from the traditional Viterbi and posterior decoding to block-based decoding algorithms tailored to our model. We compare the accuracy of individual decoding algorithms on simulated and real data and find that our approach is superior to the classical three-state pair HMM.ConclusionsOur study illustrates versatility of pair hidden Markov models coupled with appropriate decoding criteria as a modeling tool for capturing complex sequence features.

Transformation of Markov Models for Cost-Effectiveness Analysis into the System Dynamics Methodology

Markov cohort models are one of the standard methodologies for conducting cost-effectiveness analyses of health care interventions of chronic diseases. The system dynamics (SD) approach, where a model consists of stocks and their interconnecting flows, is rarely used, but in principle it can be used to conduct the same type of analysis. The authors show that a simple transformation from the transition probabilities of a Markov model to relative rates always leads to an equivalent system dynamics model. Here the stocks match the Markov states, because they store the same number of patients as in the corresponding state. The authors demonstrate an approach on an exemplary cost-effectiveness analysis for a smoking cessation programme for chronic obstructive pulmonary disease (COPD) with a simplified Markov model based on Menn (2009). Both the Markov model and the system dynamics model lead to nearly identical results. However, the latter offers more flexibility as it can incorporate interactions between different patient groups.

Bayesian inference for day-to-day dynamic traffic models

There is significant current interest in the development of models to describe the day-to-day evolution of traffic flows over a network. We consider the problem of statistical inference for such models based on daily observations of traffic counts on a subset of network links. Like other inference problems for network-based models, the critical difficulty lies in the underdetermined nature of the linear system of equations that relates link flows to the latent path flows. In particular, Bayesian inference implemented using Markov chain Monte Carlo methods requires that we sample from the set of route flows consistent with the observed link flows, but enumeration of this set is usually computationally infeasible.We show how two existing conditional route flow samplers can be adapted and extended for use with day-to-day dynamic traffic. The first sampler employs an iterative route-by-route acceptance–rejection algorithm for path flows, while the second employs a simple Markov model for traveller behaviour to generate candidate entire route flow patterns when the network has a tree structure. We illustrate the application of these methods for estimation of parameters that describe traveller behaviour based on daily link count data alone.

Disorder in Crystals of Protein Binding Protein 3K9I Revealed by Diffuse X-Ray Scattering Data and a Simple Markov Model for Molecular Displacements

Traditional X-ray crystallography measures the intensities of photons scattered into sharp Bragg peaks, yielding the average structure for a protein in the crystalline state. Diffuse X-ray scattering measures the non-Bragg signal which is spread throughout the (reciprocal) space between Bragg reflections. This diffuse scattering signal encodes the fluctuations in the protein crystal structure due to atomic and molecular disorder. In this study we analyze the X-ray diffuse scattering from a protein binding protein (from Bacillus halodurans, 3K9I). The data can be accounted for using a simple Markov model for the crystal disorder. This model allows for determination of the correlation coefficients for displacements between nearest neighbor protein molecules in the crystal lattice.

Variable Scheduling to Mitigate Channel Losses in Energy-Efficient Body Area Networks

We consider a typical body area network (BAN) setting in which sensor nodes send data to a common hub regularly on a TDMA basis, as defined by the emerging IEEE 802.15.6 BAN standard. To reduce transmission losses caused by the highly dynamic nature of the wireless channel around the human body, we explore variable TDMA scheduling techniques that allow the order of transmissions within each TDMA round to be decided on the fly, rather than being fixed in advance. Using a simple Markov model of the wireless links, we devise a number of scheduling algorithms that can be performed by the hub, which aim to maximize the expected number of successful transmissions in a TDMA round, and thereby significantly reduce transmission losses as compared with a static TDMA schedule. Importantly, these algorithms do not require a priori knowledge of the statistical properties of the wireless channels, and the reliability improvement is achieved entirely via shuffling the order of transmissions among devices, and does not involve any additional energy consumption (e.g., retransmissions). We evaluate these algorithms directly on an experimental set of traces obtained from devices strapped to human subjects performing regular daily activities, and confirm that the benefits of the proposed variable scheduling algorithms extend to this practical setup as well.

Tradeoffs in cardiovascular disease prevention, treatment, and research.

It is widely believed that the US health care system needs to transition from a culture of reactive treatment of disease to one of proactive prevention. As a tool for understanding the appropriate allocation of spending to prevention versus treatment (including research into improved prevention and treatment), a simple Markov model is used to represent the flow of individuals among states of health, where the transition rates are governed by the magnitude of appropriately-lagged expenditures in each of these categories. The model estimates the discounted cost and discounted effectiveness (measured in quality adjusted life years or QALYs) associated with a given spending mix, and it allows computing the marginal cost-effectiveness associated with additional spending in a category. We apply the model to explore interactions of alternative investments in cardiovascular disease (CVD) and to identify an optimal spending mix. Under the assumptions of our model structure, we find that the marginal cost-effectiveness of prevention of CVD varies with changes in spending on treatment (and vice versa), and that the optimal mix of CVD spending (i.e., the spending mix that maximizes the overall QALYs achieved) would, indeed, shift spending from treatment to prevention.

Corporate Governance and the Risk-Regulation Game: A Game-Theoretic Analysis of Risk Regulation

The question posed in the title of this paper is perhaps the single-most important question in law. But what drives the decision to comply with or evade the law? For example, why do some people pay their taxes, while others evade them? Although there are a few notable exceptions, many scholars have neglected the question of compliance versus evasion, simply assuming that laws will be complied with. In this paper, however, we shall not take compliance with law for granted. Instead, following in the footsteps of Gary S. Becker and others, we shall explore this critical question -- evade or comply? -- through a variety of simple, formal models. Our paper is thus organized as follows. First, in part one, we present a simple two-state Markov model of law abiding and law evading behavior. Next, in part two, we present a simple contagion model of evasion and compliance. Part three then presents an interactive or game-theoretic model of the decision to evade or comply. Part four explores some possible practical applications of our models, while part five concludes.

Reconstructing pedigrees: some identifiability questions for a recombination-mutation model

Pedigrees are directed acyclic graphs that represent ancestral relationships between individuals in a population. Based on a schematic recombination process, we describe two simple Markov models for sequences evolving on pedigrees--Model R (recombinations without mutations) and Model RM (recombinations with mutations). For these models, we ask an identifiability question: is it possible to construct a pedigree from the joint probability distribution of extant sequences? We present partial identifiability results for general pedigrees: we show that when the crossover probabilities are sufficiently small, certain spanning subgraph sequences can be counted from the joint distribution of extant sequences. We demonstrate how pedigrees that earlier seemed difficult to distinguish are distinguished by counting their spanning subgraph sequences.

Design and Routine Test Optimization of Modern Protection Systems With Reliability and Economic Constraints

This paper approaches the topic of protection system reliability from an economic point of view by 1) designing an optimal modern protective relay based on a prescribed level of reliability subject to economic constraints and 2) determining optimal routine test intervals by balancing the cost of routine tests and losses due to relay failure. This paper proposes a method that enables the designer to optimally select hardware and software components of a digital protective relay to gain the highest possible overall reliability with a restricted budget. This design can be viewed as a reliability and redundancy allocation problem for which a new easy-to-implement algorithm is proposed. In contrast to the existing literature that views the problem of routine test interval optimization solely from a reliability viewpoint, this paper proposes an optimization procedure for determining optimal test intervals. The optimization objective function consists of three terms: cost of routine tests, losses due to relay unresponsiveness, and losses due to relay maloperation. The value of the objective function is calculated based on a new, simple, and comprehensive Markov model. Illustrative numerical examples clarify the application of the proposed methods and demonstrate their effectiveness in achieving an optimum for both cases.

Extracting Kinetic Models from Single Molecule Experiments by Direct Inference

The enterprise of kinetic model-building has been a key route to insight in biophysics. An explosive growth in single molecule experiments is yielding a wealth of information on time-ordered sequences of inter-conversion between conformational states. Current strategies for model building- Markov models - often start by picking a model topology, de-noise the data according to this topology and use the data to fit the rates of the model by maximum likelihood methods. This can bias the analysis and waste data by forcing it onto a particular model. Other methods, based on maximum entropy, do not waste data or bias the analysis though they only extract rate distributions from data not the entire kinetic model. We will discuss a method we are developing which, while based on maximum entropy, extracts the full model from single molecule time traces in an unbiased fashion. We do this by numerically extracting a quantity we call a memory kernel from data. The structure of the kernel tells us whether the data warrants a simple Markov model and, if so, towards which Markov model the data naturally tends towards within uncertainty bars and without wasting data. We apply this method to real single molecule data as well as simulated test cases.

Experimental Evaluation of Events Distribution in Markov Based Computer Performance Models

The continuous-time Markov process is a widely used abstract tool for constructing high-level models of complex computer systems in order to evaluate either the performance or reliability parameters of a system. Utilization of the continuous-time Markov process is based on the assumption of exponential distribution of the time between random events influencing the behaviour of the modelled system. A different kind of probability distribution of this time requires adaptation (extension) of the original model. This article uses a representative example to evaluate the precision of the modelled system parameters using a simple Markov model based on the exponential distribution assumption instead of a more complex model which respects another (i.e. more realistic) probability distribution.

Modeling aeolian transport in response to succession, disturbance and future climate: Dynamic long-term risk assessment for contaminant redistribution

Abstract Aeolian sediment transport is a fundamental process redistributing sediment, nutrients, and contaminants in dryland ecosystems. Over time frames of centuries or longer, horizontal sediment fluxes and associated rates of contaminant transport are likely to be influenced by succession, disturbances, and changes in climate, yet models of horizontal sediment transport that account for these fundamental factors are lacking, precluding in large part accurate assessment of human health risks associated with persistent soil-bound contaminants. We present a simple model based on empirical measurements of horizontal sediment transport (predominantly saltation) to predict potential contaminant transport rates for recently disturbed sites such as a landfill cover. Omnidirectional transport is estimated within vegetation that changes using a simple Markov model that simulates successional trajectory and considers three types of short-term disturbances (surface fire, crown fire, and drought-induced plant mortality) under current and projected climates. The model results highlight that movement of contaminated soil is sensitive to vegetation dynamics and increases substantially (e.g., > fivefold) when disturbance and/or future climate are considered. The time-dependent responses in horizontal sediment fluxes and associated contaminant fluxes were sensitive to variability in the timing of disturbance, with longer intervals between disturbance allowing woody plants to become dominant and crown fire and drought abruptly reducing woody plant cover. Our results, which have direct implications for contaminant transport and landfill management in the specific context of our assessment, also have general relevance because they highlight the need to more fully account for vegetation dynamics, disturbance, and changing climate in aeolian process studies.

Detection of viral sequence fragments of HIV-1 subfamilies yet unknown

BackgroundMethods of determining whether or not any particular HIV-1 sequence stems - completely or in part - from some unknown HIV-1 subtype are important for the design of vaccines and molecular detection systems, as well as for epidemiological monitoring. Nevertheless, a single algorithm only, the Branching Index (BI), has been developed for this task so far. Moving along the genome of a query sequence in a sliding window, the BI computes a ratio quantifying how closely the query sequence clusters with a subtype clade. In its current version, however, the BI does not provide predicted boundaries of unknown fragments.ResultsWe have developed Unknown Subtype Finder (USF), an algorithm based on a probabilistic model, which automatically determines which parts of an input sequence originate from a subtype yet unknown. The underlying model is based on a simple profile hidden Markov model (pHMM) for each known subtype and an additional pHMM for an unknown subtype. The emission probabilities of the latter are estimated using the emission frequencies of the known subtypes by means of a (position-wise) probabilistic model for the emergence of new subtypes. We have applied USF to SIV and HIV-1 sequences formerly classified as having emerged from an unknown subtype. Moreover, we have evaluated its performance on artificial HIV-1 recombinants and non-recombinant HIV-1 sequences. The results have been compared with the corresponding results of the BI.ConclusionsOur results demonstrate that USF is suitable for detecting segments in HIV-1 sequences stemming from yet unknown subtypes. Comparing USF with the BI shows that our algorithm performs as good as the BI or better.

Modelling Inflation Using Markov Switching Models: Case of Poland, 1992-2005

We investigate inflation in Poland in the period of economic transition by examining the potential application of Markov Switching Models to model the inflation generating process in Poland. The time horizon of analysis was limited to the period between March 1992 and October 2005 defined as the process of disinflation, i.e. the process of continued decrease in inflation rates following the economic transition period in early 1990s which was accompanied by a high level of inflation. According to the Ball-Friedman hypothesis, variation of inflation during periods of high inflation can be unstable. Indeed, the results show that non-linear models significantly improve the description of inflation generating process in Poland. Apart from univariate Markov Models, we also use a model that incorporates inflation expectations measured by Future Inflation Indicator (FII). We find that the model, where lagged values of FII are included as exogenous variables is significantly better in modelling inflation than simple univariate Markov Model.

A hidden Markov model to predict early mastitis from test-day somatic cell scores

In many countries, high somatic cell scores (SCS) in milk are used as an indicator for mastitis because they are collected on a routine basis. However, individual test-day SCS are not very accurate in identifying infected cows. Mathematical models may improve the accuracy of the biological marker by making better use of the information contained in the available data. Here, a simple hidden Markov model (HMM) is described mathematically and applied to SCS recorded monthly on cows with or without clinical mastitis to evaluate its accuracy in estimating parameters (mean, variance and transition probabilities) under healthy or diseased states. The SCS means were estimated at 1.96 (s.d. = 0.16) and 4.73 (s.d. = 0.71) for the hidden healthy and infected states, and the common variance at 0.83 (s.d. = 0.11). The probability of remaining uninfected, recovering from infection, getting newly infected and remaining infected between consecutive test days was estimated at 78.84%, 60.49%, 11.70% and 15%, respectively. Three different health-related states were compared: clinical stages observed by farmers, subclinical cases defined for somatic cell counts below or above 250 000 cells/ml and infected stages obtained from the HMM. The results showed that HMM identifies infected cows before the appearance of clinical and subclinical signs, which may critically improve the power of the studies on the genetic determinants of SCS and reduce biases in predicting breeding values for SCS.

Weakly-Constrained Codes for Suppression of Patterning Effects in Digital Communications

We propose weakly-constrained stream and block codes with tunable pattern-dependent statistics and demonstrate that the block code capacity at large block sizes is close to the the prediction obtained from a simple Markov model published earlier. We demonstrate the feasibility of the code by presenting original encoding and decoding algorithms with a complexity log-linear in the block size and with modest table memory requirements. We also show that when such codes are used for mitigation of patterning effects in optical fibre communications, a gain of about 0.5 dB is possible under realistic conditions, at the expense of small redundancy (≈10%).

Learning Tractable Word Alignment Models with Complex Constraints

Word-level alignment of bilingual text is a critical resource for a growing variety of tasks. Probabilistic models for word alignment present a fundamental trade-off between richness of captured constraints and correlations versus efficiency and tractability of inference. In this article, we use the Posterior Regularization framework (Graça, Ganchev, and Taskar 2007) to incorporate complex constraints into probabilistic models during learning without changing the efficiency of the underlying model. We focus on the simple and tractable hidden Markov model, and present an efficient learning algorithm for incorporating approximate bijectivity and symmetry constraints. Models estimated with these constraints produce a significant boost in performance as measured by both precision and recall of manually annotated alignments for six language pairs. We also report experiments on two different tasks where word alignments are required: phrase-based machine translation and syntax transfer, and show promising improvements over standard methods.