Two-state Continuous-time Markov Chain Research Articles

Characterizing persistence and pause dynamical behaviors in biological systems

This paper analyzed motion that randomly switches between the persistent motion runs and pause periods. A two-state continuous-time Markov chain is used to model the motion, which led to a system with coupled differential equations. Using a combined Fourier–Laplace transform, an analytical expression for calculating the mean-squared displacement is derived. The overall motion is investigated and identified from the obtained mean-squared displacement. The mean-squared displacement is a nonlinear function in time that is dependent on the phase transition rate, the direction switching rate, the average speed, and the initial state. It decays and grows with increasing the direction switching and average speed, respectively. The effective diffusivity descents exponentially in short times and remains constant in long times. The waiting time in each phase decayed exponentially. The probability density function for the position of a particle at a given time tends to be Gaussian in long times. The motion can be interpreted as a super-diffusion in short times and a standard diffusion in long times with a diffusion coefficient dependent on the phase transition rates, the direction switching rate and the average speed. Persistence influences the dynamical behavior for short times while for long times diffusive behavior is exhibited.

Optimal adaptive sampling for a symmetric two-state continuous time Markov chain

We consider the optimal sampling times for a symmetric two-state continuous time Markov chain. We first consider sampling times of the form and find the optimal τ to minimize the asymptotic variance of our estimated parameter. This optimal τ depends upon the true unknown parameters and so it is infeasible in practice. To address this, we consider propose an adaptive scheme which we requires no knowledge of the true underlying parameter, we show that this method is asymptotically equivalent to the optimal fixed time design.

A 3‐D Multiuser HAP‐MIMO Channel Model Based on Dynamic Evolution of LOS Components

A theoretical Three dimensional (3D) multiuser High altitude platform (HAP) Multipleinput multiple-output (MIMO) channel model based on dynamic evolution of the Line-of-sight (LOS) components is proposed. In this paper, we consider that the LOS components could re-appear after they have vanished, and a two-state Continuous-time Markov chain (CTMC) is used to model the dynamic evolution of the LOS components. The survival probabilities of the LOS components are derived by using Chapman-Kolmogorov (C-K) equations, and the closed-form expressions are derived in this paper. Using the survival probabilities, the spatial correlation functions are derived. Measurements show that a two-state CTMC is indispensable to investigate the dynamic properties of the LOS components.

A Mathematical Analysis of Technical Analysis

ABSTRACTIn this paper, we investigate trading strategies based on exponential moving averages (ExpMAs) of an underlying risky asset. We study both logarithmic utility maximization and long-term growth rate maximization problems and find closed-form solutions when the drift of the underlying is modelled by either an Ornstein-Uhlenbeck process or a two-state continuous-time Markov chain. For the case of an Ornstein-Uhlenbeck drift, we carry out several Monte Carlo experiments in order to investigate how the performance of optimal ExpMA strategies is affected by variations in model parameters and by transaction costs.

A joint logistic regression and covariate‐adjusted continuous‐time Markov chain model

The use of longitudinal measurements to predict a categorical outcome is an increasingly common goal in research studies. Joint models are commonly used to describe two or more models simultaneously by considering the correlated nature of their outcomes and the random error present in the longitudinal measurements. However, there is limited research on joint models with longitudinal predictors and categorical cross-sectional outcomes. Perhaps the most challenging task is how to model the longitudinal predictor process such that it represents the true biological mechanism that dictates the association with the categorical response. We propose a joint logistic regression and Markov chain model to describe a binary cross-sectional response, where the unobserved transition rates of a two-state continuous-time Markov chain are included as covariates. We use the method of maximum likelihood to estimate the parameters of our model. In a simulation study, coverage probabilities of about 95%, standard deviations close to standard errors, and low biases for the parameter values show that our estimation method is adequate. We apply the proposed joint model to a dataset of patients with traumatic brain injury to describe and predict a 6-month outcome based on physiological data collected post-injury and admission characteristics. Our analysis indicates that the information provided by physiological changes over time may help improve prediction of long-term functional status of these severely ill subjects. Copyright © 2017 John Wiley & Sons, Ltd.

Le modèle stochastique SIS pour une épidémie dans un environnement aléatoire.

The stochastic SIS epidemic model in a random environment. In a random environment that is a two-state continuous-time Markov chain, the mean time to extinction of the stochastic SIS epidemic model grows in the supercritical case exponentially with respect to the population size if the two states are favorable, and like a power law if one state is favorable while the other is unfavorable.

A Mathematical Model for Randomly-Occurring Low-Rate Denial of Service Attacks

Low rate attacks, or Denial-of-Service (DoS) attacks of the occasional misbehaviour, can throttle the throughput of robust timed-protocols, like the Transmission Control Protocol(TCP), by creating either periodic or exponentially distributed outages, or transmission disruptions. Such attacks are as effective as full-fledged DoS with high undetectability of the misbehaving network entity. In this paper, we present a mathematical model of Low-Rate. randomly occurring, Denial-of-Service attacks. By viewing the process as a twostate Continuous-Time Markov Chain(CTMC), we have successfully computed the transition and state probabilities of a compromised network entity that can behave normally, while in the normal state. and abnormally, when in the abnormal state.

Stohastic Modeling and Control of Biological Systems: The Lactose Reuglation System of Escherichia coli

In this paper, we present a comprehensive framework for stochastic modeling, model abstraction, and controller design for a biological system. The first half of the paper concerns modeling and model abstraction of the system. Most models in systems biology are deterministic models with ordinary differential equations in the concentration variables. We present a stochastic hybrid model of the lactose regulation system of E. coli bacteria that capture important phenomena which cannot be described by continuous deterministic models. We then show that the resulting stochastic hybrid model can be abstracted into a much simpler model, a two-state continuous-time Markov chain. The second half of the paper discusses controller design for a specific architecture. The architecture consists of measurement of a global quantity in a colony of bacteria as an output feedback and manipulation of global environmental variables as control actuation. We show that controller design can be performed on the abstracted (Markov chain) model and implementation on the real model yields the desired result.

Stochastic Modeling and Control of Biological Systems: The Lactose Regulation System ofEscherichia Coli

In this paper, we present a comprehensive framework for stochastic modeling, model abstraction, and controller design for a biological system. The first half of the paper concerns modeling and model abstraction of the system. Most models in systems biology are deterministic models with ordinary differential equations in the concentration variables. We present a stochastic hybrid model of the lactose regulation system of E. coli bacteria that capture important phenomena which cannot be described by continuous deterministic models. We then show that the resulting stochastic hybrid model can be abstracted into a much simpler model, a two-state continuous-time Markov chain. The second half of the paper discusses controller design for a specific architecture. The architecture consists of measurement of a global quantity in a colony of bacteria as an output feedback and manipulation of global environmental variables as control actuation. We show that controller design can be performed on the abstracted (Markov chain) model and implementation on the real model yields the desired result.

Optimal production run length for deteriorating production system with a two-state continuous-time Markovian processes under allowable shortages

The paper studies optimal production run length for a deteriorating production system in which the shortages are allowed and the deterioration processes are characterized by a two-state continuous-time Markov chain. We show that there exists a unique optimal production run length to minimize the expected total relevant cost. In addition, bounds for the optimal production run length are provided to develop the solution procedure. Finally, a numerical example is given to illustrate the results and sensitivity analysis is also performed

Optimal production run length for products sold with warranty

This article studies the optimal production run length for a deteriorating production system in which the products are sold with free minimal repair warranty. The deterioration process of the system is characterized by a two-state continuous-time Markov chain. For products sold with free minimal repair warranty, we show that there exists a unique optimal production run length such that the expected total cost per item is minimized. Since there is no closed form expression for the optimal production run length, an approximate solution is derived. In addition, three special cases which provide bounds for searching the optimal production run length are investigated and some sensitivity analysis is carried out to study the effects of the model parameters on the optimal production run length. Finally, a numerical example is given to evaluate the performance of the optimal production run length.

A Highly Efficient Monte Carlo Method for Assessment of System Reliability Based on a Markov Model

SYNOPTIC ABSTRACTThis paper proposes a Monte Carlo procedure for evaluating the reliability of a system consisting of independent components with two states, failed and working. This procedure replaces the failure distribution of each individual binary component with a two-state continuous time Markov chain. The transition rates for the process are selected such that the steady-state unavailability index equals the component unreliability. The Markov chain for each component is simulated over a chosen time interval and a simulated value of the system uptime is obtained therefrom using the system logic. The system reliability is estimated as the ratio of the system uptime to the length of the simulation time horizon. An example is given on the application of this procedure to the evaluation of a reliability index extensively used in the electric power industry. This example as well as others on several simple systems show that the proposed procedure is more efficient than crude Monte Carlo in that it requires fewer random numbers on the average than the latter to obtain the same level of precision. The relative efficiency of this procedure increases with increasing value of system reliability.

Analysis of a ( Q, r, T) inventory policy with deterministic and random yields when future supply is uncertain

This paper considers a stochastic inventory model where the quantity ordered sometimes may not be available due to strikes, etc. We represent the supplier's availability process as a two-state continuous time Markov chain where one state corresponds to availability and the other state corresponds to unavailability of the supplier. The problem is to determine the reorder point, the order quantity when the system is found in ON state, and how long to wait before the next order if system is in OFF state. It is assumed that the state of the system is identified at a cost. Using the renewal reward theorem we construct the objective function as the long-run average cost. Numerical . sensitivity analysis results are provided. We also analyze the problem when the yield (i.e., amount received, if available) is random, and discuss an example with Beta distributed yield.

A Single Server Queueing Loss Model with Heterogeneous Arrival and Service

A heterogeneous arrival and service single server queueing loss model is analyzed. The arrival process of customers is assumed to be a nonstationary Poisson process with an intensity function whose evolution is governed by a two-state continuous time Markov chain. Different service distributions for different types of customers are allowed. The explicit loss formula for the model considered is obtained. In a special case, it is shown that as the arrival process becomes more regular the loss decreases. For single server loss systems with renewal arrivals, counterexamples are given to show that regularity of arrival and service distributions do not work to good effect in general. Two sufficient conditions for it to be true are given.