Nonhomogeneous Markov Processes Research Articles

A Reliability Assessment Method for Complex Systems Based on Non-Homogeneous Markov Processes.

The Markov method is a common reliability assessment method. It is often used to describe the dynamic characteristics of a system, such as its repairability, fault sequence and multiple degradation states. However, the "curse of dimensionality", which refers to the exponential growth of the system state space with the increase in system complexity, presents a challenge to reliability assessments for complex systems based on the Markov method. In response to this challenge, a novel reliability assessment method for complex systems based on non-homogeneous Markov processes is proposed. This method entails the decomposition of a complex system into multilevel subsystems, each with a relatively small state space, in accordance with the system function. The homogeneous Markov model or the non-homogeneous Markov model is established for each subsystem/system from bottom to top. In order to utilize the outcomes of the lower-level subsystem models as inputs to the upper-level subsystem model, an algorithm is proposed for converting the unavailability curve of a subsystem into its corresponding 2×2 dynamic state transition probability matrix (STPM). The STPM is then employed as an input to the upper-level system's non-homogeneous Markov model. A case study is presented using the reliability assessment of the Reactor Protection System (RPS) based on the proposed method, which is then compared with the models based on the other two contrast methods. This comparison verifies the effectiveness and accuracy of the proposed method.

The Time-Dependent Symbol of a Non-homogeneous Itô Process and Corresponding Maximal Inequalities

The probabilistic symbol is defined as the right-hand side derivative at time zero of the characteristic functions corresponding to the one-dimensional marginals of a time-homogeneous stochastic process. As described in various contributions to this topic, the symbol contains crucial information concerning the process. When leaving time-homogeneity behind, a modification of the symbol by inserting a time component is needed. In the present article, we show the existence of such a time-dependent symbol for non-homogeneous Itô processes. Moreover, for this class of processes, we derive maximal inequalities which we apply to generalize the Blumenthal–Getoor indices to the non-homogeneous case. These are utilized to derive several properties regarding the paths of the process, including the asymptotic behavior of the sample paths, the existence of exponential moments and the finiteness of p-variationa. In contrast to many situations where non-homogeneous Markov processes are involved, the space-time process cannot be utilized when considering maximal inequalities.

Operational subsea pipeline assessment affected by multiple defects of microbiologically influenced corrosion

This paper presents a systematic approach to evaluate the time interval of optimal maintenance strategy for the subsea process system influenced by Microbiological Influenced Corrosion (MIC) within multiple defects. The proposed method incorporates the non-homogeneous Poisson, homogeneous gamma, and non-homogeneous Markov processes for modeling the generation of multiple defects, the average pit depth growth, and maximum pit depth, respectively. The maintenance strategy comprises industrial procedure, probability of failure detection, errors sizing in-line inspection tools, management actions costs, and failure cost. The developed framework simulates maintenance strategies considering time interval, cost, probability of detection, average pit depth, and maximum pit depth and identifies the optimal strategy. The practical application is demonstrated in a North Sea subsea pipeline system under MIC’s influence. This work assists decision-makers in selecting the optimal conditioned-based maintenance strategy for the processing system. While the application is demonstrated to subsea process systems under MIC influence, the developed approach is equally applicable to other process systems.

Nonparametric tests for transition probabilities in nonhomogeneous Markov processes

This paper proposes nonparametric two-sample tests for the direct comparison of the probabilities of a particular transition between states of a continuous time nonhomogeneous Markov process with a finite state space. The proposed tests are a linear nonparametric test, an -norm-based test and a Kolmogorov–Smirnov-type test. Significance level assessment is based on rigorous procedures, which are justified through the use of modern empirical process theory. Moreover, the -norm and the Kolmogorov–Smirnov-type tests are shown to be consistent for every fixed alternative hypothesis. The proposed tests are also extended to more complex situations such as cases with incompletely observed absorbing states and non-Markov processes. Simulation studies show that the test statistics perform well even with small sample sizes. Finally, the proposed tests are applied to data on the treatment of early breast cancer from the European Organization for Research and Treatment of Cancer (EORTC) trial 10854, under an illness-death model.

Reliability and Survival Analysis for Drifting Markov Models: Modeling and Estimation

In this work we focus on multi-state systems modeled by means of a particular class of non-homogeneous Markov processes introduced in Vergne (Stat Appl Genet Mol Biol 7(1):1–45, 2008), called drifting Markov processes. The main idea behind this type of processes is to consider a non-homogeneity that is “smooth”, of a known shape. More precisely, the Markov transition matrix is assumed to be a linear (polynomial) function of two (several) Markov transition matrices. For this class of systems, we first obtain explicit expressions for reliability/survival indicators of drifting Markov models, like reliability, availability, maintainability and failure rates. Then, under different statistical settings, we estimate the parameters of the model, obtain plug-in estimators of the associated reliability/survival indicators and investigate the consistency of the estimators. The quality of the proposed estimators and the model validation is illustrated through numerical experiments.

The study of basic risk processes by discrete-time non-homogeneous Markov processes

The study of basic risk processes by discrete-time non-homogeneous Markov processes

The relationship between intensity and stochastic matrices for continuous-time discrete value stochastic non-homogeneous processes with Markov property

The matrices of non-homogeneous Markov processes consist of time-dependent functions whose values at time form typical intensity matrices. For solvingsome problems they must be changed into stochastic matrices. A stochas-tic matrix for non-homogeneous Markov process consists of time-dependent functions, whose values are probabilities and it depend on assumed time pe- riod. In this paper formulas for these functions are derived. Although the formula is not simple, it allows proving some theorems for Markov stochastic processes, well known for homogeneous processes, but for non-homogeneous ones the proofs of them turned out shorter.

Static output feedback control of nonhomogeneous Markovian jump systems with asynchronous time delays

This paper investigates the problem of static output feedback control for a class of nonhomogeneous Markovian jump system (NMJS) with asynchronous time delays (ATDs). Since the ATDs subject to uncertain transition probabilities (TPs) are taking into account in a practical phenomenon, new approaches are introduced to deal with the ATDs characterized by nonhomogeneous Markov processes. It is assumed that the communication links are not perfect due to its detrimental effect on the performance of systems. Stochastic variables are presented to characterize the data transmission, which are depending on operation modes and satisfying the Bernoulli distribution. Sets of slack variables are adopted to decouple the product terms between system matrices and Lyapunov matrices. Based on an extended Lyapunov function combined with Finsler inequality approach, the robust static output-feedback controller is designed for the closed-loop NMJS. Finally, a numerical example is provided to verify the design method.

Discrete-space partial dynamic equations on time scales and applications to stochastic processes

We consider a general class of discrete-space linear partial dynamic equations. The basic properties of solutions are provided (existence and uniqueness, sign preservation, maximum principle). Above all, we derive the following main results: first, we prove that the solutions depend continuously on the choice of the time scale. Second, we show that, under certain conditions, the solutions describe probability distributions of nonhomogeneous Markov processes, and that their time integrals remain the same for all underlying regular time scales.

TPmsm: Estimation of the Transition Probabilities in 3-State Models

One major goal in clinical applications of multi-state models is the estimation of transition probabilities. The usual nonparametric estimator of the transition matrix for non-homogeneous Markov processes is the Aalen-Johansen estimator (Aalen and Johansen 1978). However, two problems may arise from using this estimator: first, its standard error may be large in heavy censored scenarios; second, the estimator may be inconsistent if the process is non-Markovian. The development of the R package TPmsm has been motivated by several recent contributions that account for these estimation problems. Estimation and statistical inference for transition probabilities can be performed using TPmsm. The TPmsm package provides seven different approaches to three-state illness-death modeling. In two of these approaches the transition probabilities are estimated conditionally on current or past covariate measures. Two real data examples are included for illustration of software usage.

Large deviations for excursions of non-homogeneous Markov processes

In this paper, the large deviations at the trajectory level for ergodic Markov processes are studied. These processes take values in the non-negative quadrant of the two-dimensional lattice and are concentrated on step-wise functions. The rates of jumps towards the axes (downward jumps) depend on the position of the process - the higher the position, the greater the rate. The rates of jumps going in the same direction as the axes (upward jumps) are constants. Therefore the processes are ergodic. The large deviations are studied under equal scalings of both space and time. The scaled versions of the processes converge to 0. The main result is that the probabilities of excursions far from 0 tend to 0 exponentially fast with an exponent proportional to the square of the scaling parameter. The proportionality coefficient is an integral of a linear combination of path components. A rate function of the large deviation principle is calculated for continuous functions only.

Goodness-of-fit tests for parametric nonhomogeneous Markov processes

Tests for parametric nonhomogeneous and homogeneous Markov processes are given. Asymptotic distribution of test statistics is investigated. Tests for various well-known models are discussed as examples.

On Dobrushin Ergodicity Coefficient and weak ergodicity of Markov Chains on Jordan Algebras

In this paper we study certain properties of Dobrushin's ergodicity coefficient for stochastic operators defined on non-associative L1-spaces associated with semi-finite JBW-algebras. Such results extends the well-known classical ones to a non-associative setting. This allows us to investigate the weak ergodicity of nonhomogeneous discrete Markov processes (NDMP) by means of the ergodicity coefficient. We provide a necessary and sufficient conditions for such processes to satisfy the weak ergodicity.

A correlated random effects model for non-homogeneous Markov processes with nonignorable missingness

Life history data arising in clusters with pre-specified assessment time points for patients often feature incomplete data since patients may choose to visit the clinic based on their needs. Markov process models provide a useful tool describing disease progression for life history data. The literature mainly focuses on time homogeneous process. In this paper we develop methods to deal with non-homogeneous Markov process with incomplete clustered life history data. A correlated random effects model is developed to deal with the nonignorable missingness, and a time transformation is employed to address the non-homogeneity in the transition model. Maximum likelihood estimate based on the Monte-Carlo EM algorithm is advocated for parameter estimation. Simulation studies demonstrate that the proposed method works well in many situations. We also apply this method to an Alzheimer’s disease study.

Weak ergodicity of nonhomogeneous Markov chains on noncommutative $L^1$-spaces

In this paper we study certain properties of Dobrushin's ergod- icity coe�cient for stochastic operators de�ned on noncommutative L 1 -spaces associated with semi-�nite von Neumann algebras. Such results extends the well-known classical ones to a noncommutative setting. This allows us to in- vestigate the weak ergodicity of nonhomogeneous discrete Markov processes (NDMP) by means of the ergodicity coe�cient. We provide a su�cient condi- tions for such processes to satisfy the weak ergodicity. Moreover, a necessary and su�cient condition is given for the satisfaction of the L 1 -weak ergodicity of NDMP. It is also provided an example showing that L 1 -weak ergodicity is weaker that weak ergodicity. We applied the main results to several concrete examples of noncommutative NDMP.

Presmoothing the Aalen-Johansen estimator in the illness-death model

One major goal in clinical applications of multi-state models is the estimation of transition probabilities. The usual nonparametric estimator of the transition matrix for non-homogeneous Markov processes is the Aalen-Johansen estimator (Aalen and Johansen 1978 [1]). In this paper we propose a modification of the Aalen-Johansen estimator in the illness-death model based on presmoothing. The consistency of the proposed estimators is formally established. Simulations show that the presmoothed estimators may be much more efficient than the Aalen-Johansen estimator. A real data illustration is included.

Monounireducible Nonhomogeneous Continuous Time Semi-Markov Processes Applied to Rating Migration Models

Monounireducible nonhomogeneous semi- Markov processes are defined and investigated. The mono- unireducible topological structure is a sufficient condition that guarantees the absorption of the semi-Markov process in a state of the process. This situation is of fundamental importance in the modelling of credit rating migrations because permits the derivation of the distribution function of the time of default. An application in credit rating modelling is given in order to illustrate the results.

SIA Matrices and Non-Negative Subdivision

We reprove and extend results for uniform convergence of non-negative subdivision using results from infinite products of stochastic matrices as they appear in the study of non-homogeneous finite Markov processes. Geometric convergence of such products to rank one matrices, based on the notions of SIA matrices and of the so-called ergodic coefficient, is discussed. We also point to the properties of the directed graphs of such matrices. In this way the existing convergence results for non-negative subdivision are put into the context of such processes.

On L 1-weak ergodicity of nonhomogeneous discrete Markov processes and its applications

In the present paper we investigate the L 1-weak ergodicity of nonhomogeneous discrete Markov processes with general state spaces. Note that the L 1-weak ergodicity is weaker than well-known weak ergodicity. We provide a necessary and sufficient condition for such processes to satisfy the L 1-weak ergodicity. Moreover, we apply the obtained results to establish L 1-weak ergodicity of discrete time quadratic stochastic processes. As an application of the main result, certain concrete examples are also provided.

Bridges of quadratic harnesses

Quadratic harnesses are typically non-homogeneous Markov processes with time-dependent state space. Using an appropriately defined affine transformation we show that all bridges of a given quadratic harness can be transformed into other standard quadratic harnesses. Conversely, each such bridge is an affine transformation of a standard quadratic harness. We describe quadratic harnesses that correspond to bridges of some Levy processes. We determine all quadratic harnesses that may arise from stitching together a pair of q-Meixner processes.