Non-homogeneous Markov Process Research Articles

Neyman, Markov processes and survival analysis

J. Neyman used stochastic processes extensively in his applied work. One example is the Fix and Neyman (F-N) competing risks model (1951) that uses finite homogeneous Markov processes to analyse clinical trials with breast cancer patients. We revisit the F-N model, and compare it with the Kaplan-Meier (K-M) formulation for right censored data. The comparison offers a way to generalize the K-M formulation to include risks of recovery and relapses in the calculation of a patient's survival probability. The generalization is to extend the F-N model to a nonhomogeneous Markov process. Closed-form solutions of the survival probability are available in special cases of the nonhomogeneous processes, like the popular multiple decrement model (including the K-M model) and Chiang's staging model, but these models do not consider recovery and relapses while the F-N model does. An analysis of sero-epidemiology current status data with recurrent events is illustrated. Fix and Neyman used Neyman's RBAN (regular best asymptotic normal) estimates for the risks, and provided a numerical example showing the importance of considering both the survival probability and the length of time of a patient living a normal life in the evaluation of clinical trials. The said extension would result in a complicated model and it is unlikely to find analytical closed-form solutions for survival analysis. With ever increasing computing power, numerical methods offer a viable way of investigating the problem.

A correlated random effects model for non-homogeneous Markov processes with nonignorable missingness

Life history data arising in clusters with pre-specified assessment time points for patients often feature incomplete data since patients may choose to visit the clinic based on their needs. Markov process models provide a useful tool describing disease progression for life history data. The literature mainly focuses on time homogeneous process. In this paper we develop methods to deal with non-homogeneous Markov process with incomplete clustered life history data. A correlated random effects model is developed to deal with the nonignorable missingness, and a time transformation is employed to address the non-homogeneity in the transition model. Maximum likelihood estimate based on the Monte-Carlo EM algorithm is advocated for parameter estimation. Simulation studies demonstrate that the proposed method works well in many situations. We also apply this method to an Alzheimer’s disease study.

Markov Chain Models for the Stochastic Modeling of Pitting Corrosion

The stochastic nature of pitting corrosion of metallic structures has been widely recognized. It is assumed that this kind of deterioration retains no memory of the past, so only the current state of the damage influences its future development. This characteristic allows pitting corrosion to be categorized as a Markov process. In this paper, two different models of pitting corrosion, developed using Markov chains, are presented. Firstly, a continuous-time, nonhomogeneous linear growth (pure birth) Markov process is used to model external pitting corrosion in underground pipelines. A closed-form solution of the system of Kolmogorov's forward equations is used to describe the transition probability function in a discrete pit depth space. The transition probability function is identified by correlating the stochastic pit depth mean with the empirical deterministic mean. In the second model, the distribution of maximum pit depths in a pitting experiment is successfully modeled after the combination of two stochastic processes: pit initiation and pit growth. Pit generation is modeled as a nonhomogeneous Poisson process, in which induction time is simulated as the realization of a Weibull process. Pit growth is simulated using a nonhomogeneous Markov process. An analytical solution of Kolmogorov's system of equations is also found for the transition probabilities from the first Markov state. Extreme value statistics is employed to find the distribution of maximum pit depths.

Construction and regularity of transition functions on polish spaces under measurability conditions

This paper concerns the construction and regularity of a transition (probability) function of a non-homogeneous continuous-time Markov process with given transition rates and a general state space. Motivating from a lot of restriction in applications of a transition function with continuous (in t ≥ 0) and conservative transition rates q(t, x, Λ), we consider the case that q(t, x, Λ) are only required to satisfy a mild measurability (in t ≥ 0) condition, which is a generalization of the continuity condition. Under the measurability condition we construct a transition function with the given transition rates, provide a necessary and sufficient condition for it to be regular, and further obtain some interesting additional results.

Randomized nomination sampling for finite populations

We propose a randomized minima–maxima nomination (RMMN) sampling design for use in finite populations. We derive the first- and second-order inclusion probabilities for both with and without replacement variations of the design. The inclusion probabilities for the without replacement variation are derived using a non-homogeneous Markov process. The design is simple to implement and results in simple and easy to calculate estimators and variances. It generalizes maxima nomination sampling for use in finite populations and includes some other sampling designs as special cases. We provide some optimality results and show that, in the context of finite population sampling, maxima nomination sampling is not generally the optimum design to follow. We also show, through numerical examples and a case study, that the proposed design can result in significant improvements in efficiency compared to simple random sampling without replacement designs for a wide choice of population types. Finally, we describe a bootstrap method for choosing values of the design parameters.

Non-homogeneous Markov process models with informative observation times, with an application to Alzheimer's disease

Non-homogeneous Markov process models with informative observation times, with an application to Alzheimer's disease

Non-homogeneous Markov process models with informative observations with an application to Alzheimer's disease

Identifying risk factors for transition rates among normal cognition, mildly cognitive impairment, dementia and death in an Alzheimer's disease study is very important. It is known that transition rates among these states are strongly time dependent. While Markov process models are often used to describe these disease progressions, the literature mainly focuses on time homogeneous processes, and limited tools are available for dealing with non-homogeneity. Further, patients may choose when they want to visit the clinics, which creates informative observations. In this paper, we develop methods to deal with non-homogeneous Markov processes through time scale transformation when observation times are pre-planned with some observations missing. Maximum likelihood estimation via the EM algorithm is derived for parameter estimation. Simulation studies demonstrate that the proposed method works well under a variety of situations. An application to the Alzheimer's disease study identifies that there is a significant increase in transition rates as a function of time. Furthermore, our models reveal that the non-ignorable missing mechanism is perhaps reasonable.

Dynamic performance and cost measures with setup determination under state-age-dependent deterioration

Many manufacturing processes and production systems suffer from progressive degradation with usage and age and are subject to random failures resulting from such deterioration. Traditional models for evaluating the reliability and performance of a process/system use binary-state models, with working success or failure, to classify the states of the process/system that are unrealistic. This article proposes the classification of discrete multiple states of the deterioration process. A nonhomogeneous continuous-time Markov process is employed for modelling the process deterioration because we assume that the time for which a process stays in certain state depends not only on the current state but also on the time for which the process has been in the current state. For major and minor deteriorations, we present symbolic solutions of several differential equations by using MATLAB to estimate the probability of the process being in each state at time t. We contribute dynamic performance and cost measures for the state-age-dependent process deterioration to assess the severity of the deterioration at some point in time as well as the total severity that it causes over the entire process. The optimal setup time is determined in order to estimate the minimum total expected cost during the production period. A practical application of the proposed methodology is illustrated throughout this article.

Joint Modeling of Self-Rated Health and Changes in Physical Functioning

Self-rated health is an important indicator of future morbidity and mortality. Research has indicated that self-rated health is related to both levels of and changes in physical functioning. But no previous study has jointly modeled longitudinal functional status and self-rated health trajectories. We propose a joint model for self-rated health and physical functioning that describes the relationship between perceptions of health and the rate of change of physical functioning or disability. Our joint model uses a nonhomogeneous Markov process for discrete physical functioning states and connects this to a logistic regression model for “healthy” versus “unhealthy” self-rated health through parameters of the physical functioning model. We use simulation studies to establish finite-sample properties of our estimators and show that this model is robust to misspecification of the functional form of the relationship between self-rated health and rate of change of physical functioning. We also show that our joint model performs better than an empirical model based on observed changes in functional status. We apply our joint model to data from the Cardiovascular Health Study (CHS), a large multicenter longitudinal study of older adults. Our analysis indicates that self-rated health is associated both with level of functioning, as indicated by difficulty with activities of daily living (ADL) and instrumental activities of daily living (IADL), and with the risk of increasing difficulty with ADLs and IADLs.

Reliability assessment and replacement for machine tools under wear deterioration

This paper considers a problem of reliability assessment and determination of optimal replacement time for a machine tool under wear deterioration. Traditional models classifying a tool’s condition use binary states, with working (success) or failure (1 or 0), to evaluate its reliability and decide its optimal replacement time; however, as most machine tools deteriorate over time, a multi-state discrete model is reported as a more realistic classification for quantifying the tool wear condition. In this paper, we propose a nonhomogeneous continuous-time Markov process (NHCTMP) for modeling the tool wear process, representing the length of time the tool stays in a certain state, which depends not only on its current state but also on how long it has remained in the current state. According to this model, we first develop reliability assessment for the tool with the multi-state deterioration. Then, we derive an average expected cost function of the tool operating over the manufacturing period and optimize it to find the optimal replacement time of the tool. The optimization of the average value of the expected cost over tool replacement time emerges as a classical tradeoff idea, balancing between increasing the average cost of tool replacement and decreasing the average cost of tool deterioration. A real application is illustrated throughout the paper.

Dynamic performance measures for tools with multi-state wear processes and their applications for tool design and selection

Traditional models to evaluate the reliability and performance of tools are binary models, working (success) or failure, to classify the state of the tool. Most machine tools degrade with time and thus a multi-state discrete classification is more realistic for the continuous degradation of the tool. We propose a non-homogeneous continuous-time Markov process model for tool degradation, because the length of time the machine tool stays in a certain state depends not only on the current state, but also on how long the tool has been in the current state. The traditional reliability and life performance measures focus on the mean time between failure or the failure rate. The performance measures must capture the total experience of the manufacturer over the target life of the tool and the impact of its degradation on the quality of the products to the downstream customers. We propose several new measures for tool performance. These measures can be used to evaluate different tool designs or we can use them to select the best tool for a certain application based on the economic/disutility functions.

Electrochemical noise analysis on the pit corrosion susceptibility of Mg–10Gd–2Y–0.5Zr, AZ91D alloy and pure magnesium using stochastic model

Pit corrosion susceptibilities of Mg–10Gd–2Y–0.5Zr alloy, AZ91D alloy and pure magnesium were investigated by means of electrochemical noise (EN). The EN data have been analyzed based upon the combined stochastic theory and shot-noise theory. Pit initiation process has been modeled as a nonhomogeneous Poisson process and analyzed using a Weibull distribution function. Pit growth process has been simulated by a nonhomogeneous Markov process and analyzed using Gumbel distribution function. The EN results implied that Mg–10Gd–2Y–0.5Zr alloy had the highest pit initiation rate and pit growth probability, which revealed that Mg–10Gd–2Y–0.5Zr alloy had the highest pit susceptibility.

New patient-centered models of quality-of-life measures for evaluation of interventions for multi-stage diseases

This paper focuses on the development of quantitative dynamic models for measuring the total quality of life of patients with multi-stage diseases using disutility functions. The physician and the patient can jointly use these models to evaluate different medical interventions and make a decision on choosing the appropriate intervention for the patient. The non-homogeneous continuous-time Markov process model is first generalized for the process of the disease (e.g., change of stages in a cancer). The probability that a patient is in a particular stage at some point in time, and the expected time that the patient stays in that stage, are formulated using this stochastic model. A disutility function which is commonly used in economics is incorporated into the stochastic model to allow measurement of the patient's total experience with his/her illness and medical treatments as a function of time.

Existence and Regularity of a Nonhomogeneous Transition Matrix under Measurability Conditions

This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is to begin with a continuous (in t≥0) and conservative matrix Q(t)=[q ij (t)] of nonhomogeneous transition rates q ij (t) and use it to construct the transition probability matrix. Here we obtain the same result except that the q ij (t) are only required to satisfy a mild measurability condition, and Q(t) may not be conservative. Moreover, the resulting transition matrix is shown to be the minimum transition matrix, and, in addition, a necessary and sufficient condition for it to be regular is obtained. These results are crucial in some applications of nonhomogeneous continuous-time Markov processes, such as stochastic optimal control problems and stochastic games, and this was the main motivation for this work.

Modeling Nonhomogeneous Markov Processes via Time Transformation

Longitudinal studies are a powerful tool for characterizing the course of chronic disease. These studies are usually carried out with subjects observed at periodic visits giving rise to panel data. Under this observation scheme the exact times of disease state transitions and sequence of disease states visited are unknown and Markov process models are often used to describe disease progression. Most applications of Markov process models rely on the assumption of time homogeneity, that is, that the transition rates are constant over time. This assumption is not satisfied when transition rates depend on time from the process origin. However, limited statistical tools are available for dealing with nonhomogeneity. We propose models in which the time scale of a nonhomogeneous Markov process is transformed to an operational time scale on which the process is homogeneous. We develop a method for jointly estimating the time transformation and the transition intensity matrix for the time transformed homogeneous process. We assess maximum likelihood estimation using the Fisher scoring algorithm via simulation studies and compare performance of our method to homogeneous and piecewise homogeneous models. We apply our methodology to a study of delirium progression in a cohort of stem cell transplantation recipients and show that our method identifies temporal trends in delirium incidence and recovery.

Customer’s Cumulative Experience Measures for Reliability of Non-Repairable Aging Multi-State Systems

In this paper, the concept of customer-centered reliability measure is discussed and a new customer-centered reliability measure is developed based on the customer’s cumulative experience with the non-repairable multi-state system. The deterioration of the system is assumed to follow the nonhomogeneous continuous time Markov process (NHCTMP) which is a continuous time and discrete state process. The customer’s disutility also changes with the state of the system. An exponential disutility function, which is commonly given in economics, is incorporated with the stochastic model for estimating the total customer’s experience with the system over time. Examples for computations of reliability measures from the viewpoint of the customer are demonstrated using Mathematica 5.0.

Stochastic modeling of pitting corrosion: A new model for initiation and growth of multiple corrosion pits

In this work, a new stochastic model capable of simulating pitting corrosion is developed and validated. Pitting corrosion is modeled as the combination of two stochastic processes: pit initiation and pit growth. Pit generation is modeled as a nonhomogeneous Poisson process, in which induction time for pit initiation is simulated as the realization of a Weibull process. In this way, the exponential and Weibull distributions can be considered as the possible distributions for pit initiation time. Pit growth is simulated using a nonhomogeneous Markov process. Extreme value statistics is used to find the distribution of maximum pit depths resulting from the combination of the initiation and growth processes for multiple pits. The proposed model is validated using several published experiments on pitting corrosion. It is capable of reproducing the experimental observations with higher quality than the stochastic models available in the literature for pitting corrosion.

Continuous Time Markov Models for Binary Longitudinal Data

Longitudinal data usually consist of a number of short time series. A group of subjects or groups of subjects are followed over time and observations are often taken at unequally spaced time points, and may be at different times for different subjects. When the errors and random effects are Gaussian, the likelihood of these unbalanced linear mixed models can be directly calculated, and nonlinear optimization used to obtain maximum likelihood estimates of the fixed regression coefficients and parameters in the variance components. For binary longitudinal data, a two state, non-homogeneous continuous time Markov process approach is used to model serial correlation within subjects. Formulating the model as a continuous time Markov process allows the observations to be equally or unequally spaced. Fixed and time varying covariates can be included in the model, and the continuous time model allows the estimation of the odds ratio for an exposure variable based on the steady state distribution. Exact likelihoods can be calculated. The initial probability distribution on the first observation on each subject is estimated using logistic regression that can involve covariates, and this estimation is embedded in the overall estimation. These models are applied to an intervention study designed to reduce children's sun exposure.

A stochastic framework for estimation of summary measures in cost–effectiveness analyses

Stochastic modeling of patient health and cost outcomes is a useful tool in identifying and assessing the important individual characteristics that influence these outcomes. It is a critical step in cost–effectiveness analysis that aims to structure comparisons between competing healthcare interventions on both health outcomes and costs. The authors use a continuous-time finite-state nonhomogeneous Markov process to describe the occurrence of patient events as they unfold over time. States of the process represent health conditions or health states (e.g., well, ill or dead). Commonly used measures of health outcome, such as life expectancy and quality-adjusted survival, are defined in terms of expected values of functions of the process. Costs are incurred through medical resource use while sojourning in health states, and in transitions between health states. By combining these expenditure streams, the authors define net present values for expected total cost over a specified time period. Metrics used in cost–effectiveness analyses, such as net health benefit, net health cost and the cost–effectiveness ratio are defined as functions of expected values. In this longitudinal framework the authors describe regression models for estimation of these summary statistics and quantifying the impact thereupon of explanatory variables. With increasing attention being given to longitudinal studies for gathering economic data alongside clinical investigations, these models can be applied to analyze jointly both costs and patient outcomes and more fully exploit the dynamic mechanisms that engender cost and health outcome data.

Modelling the deterioration of buried infrastructure as a fuzzy Markov process

Research Article| March 01 2006 Modelling the deterioration of buried infrastructure as a fuzzy Markov process Yehuda Kleiner; Yehuda Kleiner 1Senior Research Officer and Group Leader, Institute for Research in Construction, National Research Council Canada (NRC), 1200 Montreal Road, Building M-20Ottawa, ON Canada K1A 0R6 Phone: 1-613-993-3805 Fax: 1-613-954-5984 yehuda.kleiner@nrc-cnrc.gc.ca E-mail: yehuda.kleiner@nrc-cnrc.gc.ca Search for other works by this author on: This Site PubMed Google Scholar Rehan Sadiq; Rehan Sadiq 2Research Officer, Institute for Research in Construction, National Research Council Canada (NRC), 1200 Montreal Road, Building M-20Ottawa, ON, Canada K1A 0R6 Phone: 1-613-993-6282 Fax: 1-613-954-5984 rehan.sadiq@nrc-cnrc.gc.ca Search for other works by this author on: This Site PubMed Google Scholar Balvant Rajani Balvant Rajani 3Principal Research Officer, Institute for Research in Construction, National Research Council Canada (NRC), 1200 Montreal Road, Building M-20Ottawa, ON Canada K1A 0R6 Phone: 1-613-993-3810 Fax: 1-613-954-5984 balvant.rajani@nrc-cnrc.gc.ca Search for other works by this author on: This Site PubMed Google Scholar Journal of Water Supply: Research and Technology-Aqua (2006) 55 (2): 67–80. https://doi.org/10.2166/aqua.2006.074 Article history Received: August 23 2005 Accepted: December 08 2005 Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Share Icon Share Twitter LinkedIn Tools Icon Tools Cite Icon Cite Permissions Search Site Search nav search search input Search input auto suggest search filter All ContentAll JournalsThis Journal Search Advanced Search Citation Yehuda Kleiner, Rehan Sadiq, Balvant Rajani; Modelling the deterioration of buried infrastructure as a fuzzy Markov process. Journal of Water Supply: Research and Technology-Aqua 1 March 2006; 55 (2): 67–80. doi: https://doi.org/10.2166/aqua.2006.074 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex Dearth of data is the greatest acknowledged obstacle to the deterioration modeling of the buried infrastructure assets. In the last two decades numerous models have been proposed with a greater emphasis on Markovian Deterioration Processes (MDP). The MDP requires that the condition of the deteriorating system be encoded as an ordinal condition rating, based on numerous distress indicators obtained possibly from direct and indirect observations, as well as from non-destructive tests. The encoding of distress indicators into condition rating is inherently imprecise and involves subjective judgment. This imprecision is not considered, let alone propagated in the traditional application of the MDP.In this paper a new approach is presented to model the deterioration of buried infrastructure assets using a fuzzy rule-based, non-homogeneous Markov process. This deterioration model yields the ‘possibility’ of failure at every time step along the life of the asset. The use of fuzzy sets and fuzzy techniques help to incorporate the inherent imprecision and subjectivity of the data, as well as to propagate these attributes throughout the model, yielding more realistic results.This paper is the first of two companion papers that describe a complete method of managing failure risk of large buried infrastructure assets. The second companion paper describes how the condition ratings along the life of the asset are converted to risk values and how effective decisions can be made about the renewal and/or scheduling the next inspection of the asset. condition rating, deterioration, failure risk, fuzzy Markov, infrastructure, large water mains This content is only available as a PDF. © NRC 2006 You do not currently have access to this content.