Markov Dependence Research Articles

The distribution of the usual provider continuity index under Markov dependence

In the health care sector, a sequence identifying which doctor/provider was seen by a particular patient over a number of visits can be viewed as a series of outcomes from multi-state trials. The usual provider continuity (UPC) index, defined as the fraction of patient visits to some specified “usual” provider, is one of the most commonly used measures for quantifying continuity of care. In this note, we derive the exact distribution of UPC under the assumption of Markov dependence between visits, using the finite Markov chain imbedding technique. Numerical examples are given to illustrate our main result.

Diagnostic methods for statistical models of place cell spiking activity

We analyze the stochastic structure of CA1 hippocampal place cell spiking activity with stimulus–response models based on inhomogeneous Poisson (IP), inhomogeneous gamma (IG) and inhomogeneous inverse Gaussian (IIG) interspike interval probability densities that have Markov dependence. We present a technique based on quantile–quantile (Q–Q) plots derived from the intensity-rescaling transformation, and use it along with Akaike's (AIC) and Bayesian (BIC) information critieria to assess model goodness of fit. The Q–Q plots give readily interpretable, graphical diagnostic methods of the model fits, and show that the IG and IIG models give more accurate descriptions of place cell spiking activity than the IP model.

A general model for consecutive- k-out-of- n: F repairable system with exponential distribution and ( k−1)-step Markov dependence

In this paper, a general model for consecutive- k-out-of- n: F repairable system with exponential distribution and ( k−1)-step Markov dependence is introduced. The lifetime of a component is an exponential random variable, its parameter depends on the number of consecutive failed components that precede the component. The repair time is also an exponential random variable. A priority repair rule on the basis of the system failure risk is adopted. Then the transition density matrix of the system is determined. Some reliability indices, including the system availability, rate of occurrence of failures and reliability are evaluated accordingly. For the demonstration of the model and methodology, a linear system example and a circular system example are investigated.

Construction and analysis of non-Poisson stimulus-response models of neural spiking activity

A paradigm for constructing and analyzing non-Poisson stimulus-response models of neural spike train activity is presented. Inhomogeneous gamma (IG) and inverse Gaussian (IIG) probability models are constructed by generalizing the derivation of the inhomogeneous Poisson (IP) model from the exponential probability density. The resultant spike train models have Markov dependence. Quantile–quantile (Q–Q) plots and Kolmogorov–Smirnov (K–S) plots are developed based on the rate-rescaling theorem to assess model goodness-of-fit. The analysis also expresses the spike rate function of the neuron directly in terms of its interspike interval (ISI) distribution. The methods are illustrated with an analysis of 34 spike trains from rat CA1 hippocampal pyramidal neurons recorded while the animal executed a behavioral task. The stimulus in these experiments is the animal's position in its environment and the response is the neural spiking activity. For all 34 pyramidal cells, the IG and IIG models gave better fits to the spike trains than the IP. The IG model more accurately described the frequency of longer ISIs, whereas the IIG model gave the best description of the burst frequency, i.e. ISIs≤20 ms. The findings suggest that bursts are a significant component of place cell spiking activity even when position and the background variable, theta phase, are taken into account. Unlike the Poisson model, the spatial and temporal rate maps of the IG and IIG models depend directly on the spiking history of the neurons. These rate maps are more physiologically plausible since the interaction between space and time determines local spiking propensity. While this statistical paradigm is being developed to study information encoding by rat hippocampal neurons, the framework should be applicable to stimulus-response experiments performed in other neural systems.

Large Deviations for Partial Sums U-Processes in Dependent Cases

The large deviation principle (LDP) is known to hold for partial sums U-processes of real-valued kernel functions of independent identically distributed random variables $X_i$. We prove an LDP when the $X_i$ are independent but not identically distributed or fulfill some Markov dependence or mixing conditions. Moreover, we give a general condition which suffices for the LDP to carry over from the partial sums empirical processes LDP to the partial sums U-processes LDP for kernel functions satisfying an appropriate exponential tail condition.

The exact distribution of the continuity of care measure NOP

The number of different providers (NOP) visited by one individual during a follow-up period can serve as a measure of concentration, and has formed the basis for a number of indices of continuity of care in the health care sector. In this article, the exact distribution of NOP, under the assumption of i.i.d. as well as under Markov dependence, is derived using the finite Markov chain imbedding technique. Numerical examples are given to illustrate the main results.

Drought parameters in relation to truncation levels

Drought parameters, namely the longest duration, largest severity and the intensity have been calculated in relation to the truncation levels ranging from 0 to −0\5 in the standardized domain for the normal, gamma and log-normal probability distribution functions (PDFs) of the drought variables. The drought variables taken in the investigation are the annual rainfall and runoff time-series evolving randomly and obeying the Markovian dependence. The analysis showed that the assumption of independence of the drought duration and intensity works well in deriving the expression for drought severity at various truncation levels. An estimate of drought intensity has been realized from the concept of the truncated normal distribution of the standardized form of the time-series of drought variables in the normalized domain. Furthermore the non-normality and the dependence in the time-series have a significant effect on the drought parameters at the truncation levels under consideration. Copyright © 2000 John Wiley & Sons, Ltd.

Repairable consecutive-k-out-of-n:F system with Markov dependence

Repairable consecutive-k-out-of-n:F system with Markov dependence

Repairable consecutive-k-out-of-n: F system with Markov dependence

Repairable consecutive-k-out-of-n: F system with Markov dependence

Numerical iterative methods for Markovian dependability and performability models: new results and a comparison

In this paper we deal with iterative numerical methods to solve linear systems arising in continuous-time Markov chain (CTMC) models. We develop an algorithm to dynamically tune the relaxation parameter of the successive over-relaxation method. We give a sufficient condition for the Gauss–Seidel method to converge when computing the steady-state probability vector of a finite irreducible CTMC, and a sufficient condition for the generalized minimal residual projection method not to converge to the trivial solution 0 when computing that vector. Finally, we compare several splitting-based iterative methods and a variant of the generalized minimal residual projection method.

On the distribution of the number of successes in fourth- or lower-order Markovian trials

This paper presents an algorithm which may be used to compute the distribution of the number of successes in a sequence of binary trials which displays dependency. The case of independent trials is treated even in elementary texts, and these results are frequently extended in the literature to the case of first-order Markovian dependence. However, results are scarce for higher-order Markovian dependence. In this article, an algorithm is developed for computing the distribution of the number of successes in binary sequences, under the assumption that the dependence structure is fourth-order Markovian. The importance of using the best model order for a particular data set is discussed. Examination of the computed distribution estimates for various orders may assist in model determination. Scope and purpose Many statistical applications may be modelled as a sequence of n dependent trials, with outcomes which may be classified as either a “success” or a “failure”. This paper presents an algorithm which may be used to compute the distribution of the number of successes in such sequences. It is assumed that the probability of success on the nth trial depends on the outcome of trials n−v, n−v+1, …, n−1 , for some finite value of v, but is independent of trials before n− v. The algorithm extends algorithms given by Kedem [1] for v=1 or 2 to the case where v=4. The importance of selecting an appropriate value for v when modelling dependent trials is discussed.

A nonparametric approach for daily rainfall simulation

A nonparametric model for daily rainfall simulation is presented. Nearest neighbour methods are used to conditionally simulate rainfall spells and amounts. A “local” subset of the observed record is used to formulate the conditional densities needed for simulation. This provides an effective yet simple way to model local and seasonal features in the observed rainfall time series. The model is applied in two stages. First dry and wet spell lengths are conditionally simulated. Next rainfall amounts for each day of the wet spell are simulated assuming an order one Markov dependence structure. Higher order dependence is modelled by considering the number of days from start of the spell as an additional variable. Rainfall distributional characteristics are observed to have distinctly different characteristics depending on the length of the wet spell. A procedure is developed to reproduce such differences in the simulations. The model is applied to 123 years of daily rainfall from Sydney, Australia.

Analysis of repairable consecutive-2-out-of-n: F systems with Markov dependence

In this paper, we study a repairable consecutive-2-out-of-n: F system with Markov dependence. We suppose that the failure rate of a component depends on the state of the preceding component. The components are classified as key components and ordinary components, where a key component will have higher priority for repair. For both linear and circular systems, the generalized transition density matrices are then determined. Some reliability indices are also studied.

Analysis of repairable consecutive-2-out-of-n : F systems with Markov dependence

In this paper, we study a repairable consecutive-2-out-of-n: F system with Markov dependence. Suppose that the failure rate of a component depends on the state of the preceding component. The components are classified as key components and ordinary components, and a key component will have higher priority for repair. For both linear and circular systems, the generalized transition density matrices are then determined. Some reliability indices are also studied.

A generalized sequential sign detector for binary hypothesis testing

It is known that for fixed error probabilities sequential signal detection based on the sequential probability ratio test (SPRT) is optimum in terms of the average number of signal samples for detection. But, often suboptimal detectors like the sequential sign detector are preferred over the optimal SPRT. When the additive noise statistic is independent and identically distributed (i.i.d.), the sign detector is preferred for its simplicity and nonparametric properties. However, in many practical applications such as the usage of high speed sampling devices the noise is correlated. A generalized sequential sign detector for detecting binary signals in stationary, first-order Markov dependent noise is studied. Under the i.i.d. assumptions, this reduces to the usual sequential sign detector. The optimal decision thresholds and the average sample number for the test to terminate are derived. Numerical results are given to show that the proposed detector exploits the correlation in the noise and hence results in quicker detection. The method can also be extended to Mth order Markov dependence by converting it to a first-order dependence in an extended state space.

Analysis of Longitudinal Data with Non-Ignorable Non-Monotone Missing Values

SUMMARY A full likelihood method is proposed to analyse continuous longitudinal data with non-ignorable (informative) missing values and non-monotone patterns. The problem arose in a breast cancer clinical trial where repeated assessments of quality of life were collected: patients rated their coping ability during and after treatment. We allow the missingness probabilities to depend on unobserved responses, and we use a multivariate normal model for the outcomes. A first-order Markov dependence structure for the responses is a natural choice and facilitates the construction of the likelihood; estimates are obtained via the Nelder–Mead simplex algorithm. Computations are difficult and become intractable with more than three or four assessments. Applying the method to the quality-of-life data results in easily interpretable estimates, confirms the suspicion that the data are non-ignorably missing and highlights the likely bias of standard methods. Although treatment comparisons are not affected here, the methods are useful for obtaining unbiased means and estimating trends over time.

Exact failure probability formula for strict consecutive-k-out-of-n:F systems with homogeneous Markov dependence

An exact formula is presented for calculating the failure probability of strict consecutive-k-out-of-n:F systems with homogeneous Markov dependence

Start-Up Demonstration Tests Under Markov Dependence Model with Corrective Actions

A general probability model for a start-up demonstration test is studied. The joint probability generating function of some random variables appearing in the Markov dependence model of the start-up demonstration test with corrective actions is derived by the method of probability generating function. By using the probability generating function, several characteristics relating to the distribution are obtained.

On Runs and Longest Run Tests: A Method of Finite Markov Chain Imbedding

Abstract In this article, under very general assumptions on Bernoulli trials, a simple numerical method based on the finite Markov chain imbedding technique is systematically developed to determine (a) the joint distribution of the number of success runs and the number of successes; (b) the conditional distribution of the number of success runs given the number of successes, that is, the exact distribution of the success runs test statistic; (c) the joint distribution of the length of the longest success run and the number of successes; and (d) the conditional distribution of the length of the longest success run given the number of successes, that is, the exact distribution of the longest success run test statistic. The critical regions and powers of these two tests (success runs and longest success run) are derived under the null hypothesis of independence and identical distribution (iid) as well as under the alternative hypothesis of Markov dependence. The application of the success runs test is illust...

Accelerating mean time to failure computations

In this paper we consider the problem of numerical computation of the mean time to failure (MTTF) in Markovian dependability and/or performance models. The problem can be cast as a system of linear equations which is solved using an iterative method preserving sparsity of the Markov chain matrix. For highly dependable systems, system failure is a rare event and the above system solution can take an extremely large number of iterations. We propose to solve the problem by dividing the computation in two parts. First, by making some of the high probability states absorbing, we compute the MTTF of the modified Markov chain. In a subsequent step, by solving another system of linear equations, we are able to compute the MTTF of the original model. We prove that for a class of highly dependable systems, the resulting method can speed up computation of the MTTF by orders of magnitude. Experimental results supporting this claim are presented. We also obtain bounds on the convergence rate for computing the mean entrance time of a rare set of states in a class of queueing models.