Nonhomogeneous Transition Probabilities Research Articles

Simplified Exhaustive Search Approach for Estimating the Nonhomogeneous Transition Probabilities for Infrastructure Asset Management

AbstractA simplified exhaustive search approach is proposed to estimate the nonhomogeneous transition probabilities for a particular infrastructure element. The yearly nonhomogeneous transition pro...

A Hybrid Genetic Algorithm-Based Fuzzy Markovian Model for the Deterioration Modeling of Healthcare Facilities

Healthcare facilities are constantly deteriorating due to tight budgets allocated to the upkeep of building assets. This entails the need for improved deterioration modeling of such buildings in order to enforce a predictive maintenance approach that decreases the unexpected occurrence of failures and the corresponding downtime elapsed to repair or replace the faulty asset components. Currently, hospitals utilize subjective deterioration prediction methodologies that mostly rely on age as the sole indicator of degradation to forecast the useful lives of the building components. Thus, this paper aims at formulating a more efficient stochastic deterioration prediction model that integrates the latest observed condition into the forecasting procedure to overcome the subjectivity and uncertainties associated with the currently employed methods. This is achieved by means of developing a hybrid genetic algorithm-based fuzzy Markovian model that simulates the deterioration process given the scarcity of available data demonstrating the condition assessment and evaluation for such critical facilities. A nonhomogeneous transition probability matrix (TPM) based on fuzzy membership functions representing the condition, age and relative deterioration rate of the hospital systems is utilized to address the inherited uncertainties. The TPM is further calibrated by means of a genetic algorithm to circumvent the drawbacks of the expert-based models. A sensitivity analysis was carried out to analyze the possible changes in the output resulting from predefined modifications to the input parameters in order to ensure the robustness of the model. The performance of the deterioration prediction model developed is then validated through a comparison with a state-of-art stochastic model in contrast to real hospital datasets, and the results obtained from the developed model significantly outperformed the long-established Weibull distribution-based deterioration prediction methodology with mean absolute errors of 1.405 and 9.852, respectively. Therefore, the developed model is expected to assist decision-makers in creating more efficient maintenance programs as well as more data-driven capital renewal plans.

Constrained Model Predictive Control for Nonlinear Markov Jump System With Persistent Disturbance via Quadratic Boundedness

In this article, a robust quadratic-boundedness-based model predictive control (MPC) scheme, for a discrete-time nonlinear Markov jump system (MJS), is extended to the case with persistent bounded disturbance and nonhomogeneous transition probability. By applying the S-procedure, the constraint conditions, the persistent bounded disturbance and the sufficient stability conditions are all derived in term of a few linear matrix inequalities (LMIs), thus the original min-max optimization problem is transformed into a convex optimization problem in LMI paradigm. At each sampling time, the control moves satisfying the control constraint are obtained online and implemented in the nonlinear MJS. Quadratically boundedness and min-max MPC are combined to achieve the closed-loop stochastic stability of the controller with respect to the persistent bounded disturbance. A numerical example is presented to demonstrate the effectiveness of the proposed results.

Constrained MPC design of nonlinear Markov jump system with nonhomogeneous process

In this paper, a differential-inclusion-based MPC scheme is developed for the controller design for a discrete time nonlinear Markov jump system with nonhomogeneous transition probability. By adopting a differential-inclusion-based convex model predictive control mechanism, the nonlinear Markov jump system with nonhomogeneous transition probability is enclosed by a set of linear Markov jump systems. In this way, the controller design for the nonlinear Markov jump system can be solved via solving a set of linear Markov jump systems. Two numerical examples with different weighting parameters R are presented to illustrate the applicability of the results obtained.

Characterization of Time-Varying Regimes in Remote Sensing Time Series: Application to the Forecasting of Satellite-Derived Suspended Matter Concentrations

The spatial and temporal coverage of satellites provides data that are particularly well suited for the analysis and characterization of space–time-varying geophysical relationships. The latent-class models aim here to identify time-varying regimes within a dataset. This is of particular interest for geophysical processes driven by the seasonal variability. As a case example, we study the daily concentration of mineral suspended particulate matters estimated from satellite-derived datasets, in coastal waters adjacent to the French Gironde river mouth. We forecast this high-resolution dataset using environmental data (wave height, wind strength and direction, tides, and river outflow) and four latent-regime models: homogeneous and nonhomogeneous Markov-switching models, with and without an autoregressive term (i.e., the mineral suspended matter concentration observed the day before). Using a validation dataset, significant improvements are observed with the multiregime models compared to a classical multiregression and a state-of-the-art nonlinear model [support vector regression (SVR) model]. The best results are reported for a mixture of three regimes for the autoregressive model using nonhomogeneous transitions. With the autoregressive models, we obtain at day+1 for the mixture model forecasting performances of 93% of the explained variance, compared to 83% for a standard linear model and 85% using an SVR. These improvements are more important for the nonautoregressive models. These results stress the potential of the identification of geophysical regimes to improve the forecasting. We also show that nonhomogeneous transition probabilities and estimated autoregressive terms improve forecasting performances when observation data is lacking for short-time period of 1–15 days.

Bayesian estimation for jump Markov linear systems with non-homogeneous transition probabilities

This paper considers the state estimation problem for a class of discrete-time non-homogeneous jump Markov linear systems (JMLSs), where the transition probability matrix (TPM) is assumed to be time-variant and takes value in a finite set randomly at each time step. To show the simplicity brought by the finite-valued hypothesis, the optimal recursion for the posterior TPM probability density functions conditioned on that the TPM belongs to a continuous set is firstly derived. Then, we naturally incorporate the proposed TPM estimation into the recursion of system state. Two interacting multiple-model (IMM)-type approximation stages are employed to avoid the exponential computational requirements. The resulting filter reduces to the IMM filter when the number of candidate TPMs is unity. A meaningful example is presented to illustrate the effectiveness of our method.

State Estimation for Nonlinear Discrete-Time Systems with Markov Jumps and Nonhomogeneous Transition Probabilities

State estimation problem is addressed for a class of nonlinear discrete-time systems with Markov parameters and nonhomogeneous transition probabilities (TPs). In this paper, the optimal estimation mechanism of transition probability matrix is proposed in the minimum mean square error sense to show some critical points. Based on this mechanism, the extended Kalman filters are employed as the subfilters to obtain the subestimates with corresponding models. A novel operator which fuses the prior knowledge and the posterior information embedded in observations is developed to modify the posterior mode probabilities. A meaningful example is presented to illustrate the effectiveness of our method.

Bayesian Filtering for Non-linear Markov Jump Systems with Non-homogeneous Transition Probabilities

摘要: 针对具有时变不确定转移概率的非线性非齐次Markov跳变系统, 提出一种贝叶斯状态估计方法.该方法首次采用带约束高斯概率密度函数来刻画转移概率的真实特性. 然后,基于参考概率空间法, 将实际的概率测度投影到理想概率空间, 得出信息变量的递归表达式. 同时, 在贝叶斯框架内给出转移概率矩阵的最大后验估计式. 进一步, 采用粒子逼近法求解转移概率矩阵的最大后验估计, 解决非线性函数的多重积分问题, 进而获取状态估计值. 最后, 通过一个仿真示例表明该方法的有效性. 关键词: 非线性非齐次Markov跳变系统 / 贝叶斯估计 / 参考概率空间 / 粒子逼近

Monotonicity and comparability in ballistic missile defense systems

A general framework is provided for modeling Ballistic Missile Defense (BMD) systems that are driven by stochastically monotone decreasing discrete time Markov chains, not necessarily temporally homogeneous. Based on a lexicographical ordering of defense vectors, the notions of stochastic monotonicity and comparability for such systems are investigated. Then conditions under which these systems are monotone and comparable are established. For any such BMD system, it is shown that the number of missiles penetrating the defense is decreasing in the likelihood ratio, and the elements of the associated one-step nonhomogeneous transition probability matrix are totally positive of order 2. In addition, the number of missiles penetrating the defense under uniform firing doctrine is shown to be smaller in the reversed hazard rate order than the number of missiles penetrating the defense under random firing doctrine. Particular performance measures are also shown to be monotone and comparable.

Switching Varma Term Structure Models - Extended Version

The purpose of the paper is to propose a global discrete-time modeling of the term structure of interest rates able to capture simultaneously the following important features: (i) an historical dynamics of the factor driving term structure shapes involving several lagged values, and switching regimes; (ii) a specification of the stochastic discount factor (SDF) with time-varying and regime dependent risk-premia; (iii) explicit or quasi explicit formulas for zero-coupon bond and interest rate derivative prices; (iv) the positivity of the yields at each maturity. The first family of models we develop is given by the Switching Autoregressive Normal (SARN) and the Switching Vector Autoregressive Normal (SVARN) Factor-Based Term Structure Models of order p. The second family of models we study is given by the Switching Autoregressive Gamma (SARG) and the Switching Vector Autoregressive Gamma (SVARG) Factor-Based Term Structure Models of order p. Regime shifts are described by a Markov chain with (historical) non-homogeneous transition probabilities.

Switching Varma Term Structure Models - Extended Version

The purpose of the paper is to propose a global discrete-time modeling of the term structure of interest rates able to capture simultaneously the following important features : (i) an historical dynamics of the factor driving term structure shapes involving several lagged values, and switching regimes; (ii) a specification of the stochastic discount factor (SDF) with time-varying and regime dependent risk-premia; (iii) explicit or quasi explicit formulas for zero-coupon bond and interest rate derivative prices; (iv) the positivity of the yields at each maturity. The first family of models we develop is given by the Switching Autoregressive Normal (SARN) and the Switching Vector Autoregressive Normal (SVARN) Factor-Based Term Structure Models of order p. The second family of models we study is given by the Switching Autoregressive Gamma (SARG) and the Switching Vector Autoregressive Gamma (SVARG) Factor-Based Term Structure Models of order p. Regime shifts are described by a Markov chain with (historical) non-homogeneous transition probabilities.

Exponential bounds for discrete-time singularly perturbed Markov chains

This paper develops exponential type upper bounds for scaled occupation measures of singularly perturbed Markov chains in discrete time. By considering two-time scale in the Markov chains, asymptotic analysis is carried out. The cases of the fast changing transition probability matrix is irreducible and that are divisible into l ergodic classes are examined first; the upper bounds of a sequence of scaled occupation measures are derived. Then extensions to Markov chains involving transient states and/or nonhomogeneous transition probabilities are dealt with. The results enable us to further our understanding of the underlying Markov chains and related dynamic systems, which is essential for solving many control and optimization problems.

The Birth and Death Chain in a Random Environment: Instability and Extinction Theorems

Let $(Y_n)$ be a recurrent Markov chain with discrete or continuous state space. A model of a birth and death chain $(Z_n)$ controlled by a random environment $(Y_n)$ is formulated wherein the bivariate process $(Y_n, Z_n)$ is taken to be Markovian and the marginal process $(Z_n)$ is a birth and death chain on the nonnegative integers with absorbing state $z = 0$ when a fixed sequence of environmental states of $(Y_n)$ is specified. In this paper, the property of uniform $\phi$-recurrence of $(Y_n)$ is used to prove that with probability one the sequence $(Z_n)$ does not remain positive or bounded. An example is given to show that uniform $\phi$-recurrence of $(Y_n)$ is required to insure this instability property of $(Z_n)$. Conditions are given for the extinction of the process $(Z_n)$ when (i) $(Z_n)$ possesses homogeneous transition probabilities and $(Y_n)$ possesses an invariant measure on discrete state space, and (ii) $(Z_n)$ possesses nonhomogeneous transition probabilities and $(Y_n)$ has general state space.

Sur une caractérisation des vidanges fortes

This paper is concerned with a discrete Markov process related to an underlying weighted graph. With this graph we associate an urn containing p types of balls which correspond to the p vertices of the graph, and such that the number of balls of each type is proportional to the weight of the related vertex. A drawing scheme from the urn is defined which leads to Markovian non-homogeneous transition probabilities. Many kinds of weighted graphs are found to have a strong convergence property; this is that the number of balls drawn from the urn at the end of any single draw converges in probability to the total number of balls in the urn, as this tends to infinity.

Sur une caractérisation des vidanges fortes

This paper is concerned with a discrete Markov process related to an underlying weighted graph. With this graph we associate an urn containing p types of balls which correspond to the p vertices of the graph, and such that the number of balls of each type is proportional to the weight of the related vertex. A drawing scheme from the urn is defined which leads to Markovian non-homogeneous transition probabilities. Many kinds of weighted graphs are found to have a strong convergence property; this is that the number of balls drawn from the urn at the end of any single draw converges in probability to the total number of balls in the urn, as this tends to infinity.

Die Struktur der Ausscheideordnungen in der Invaliditätsversicherung im Lichte der modernen Mengen- und Wahrscheinlichkeitstheorie

The present treatise deals with the structure and functions of the multiple decrement tables in disability insurance on the basis of modern set and probability theories. In the first part, the author shows the set relations for the life table, the service table and the open disablement table. In this respect, two simple relations (see formula 1,39) are indicated for the two independent annual probabilities of exit in sense of the probability theory (probability of disablement). After elucidation of the pertinent theory, the process of decrement out of a closed service table is described as a continuous parameter Markov chain with non-homogeneous transition probability and finitely many states. The discussion of the homogeneous case leads further to an analysis of the mechanism of decrement due to two independent causes. Finally, the two distributions of the probabilities of remaining in the state or of leaving it are deduced.