Non-homogeneous Markov System Research Articles

Transient analysis of a SIQS model with state capacities using a non-homogeneous Markov system

In this paper, a novel stochastic model is proposed to model the spread of a virus in epidemics. The model is based on a discrete-time non-homogeneous Markov system with state capacities. In order to study the distributions of the state sizes, recursive formulae for their factorial and mixed factorial moments were derived in matrix form. As a consequence, the probability mass function of each state size can be evaluated in the transient period. To avoid the computational complexity of the proposed algorithm, an alternative method for the computation of the state size distributions was recommended. The proposed Markovian approach was then tailored to the characteristics of a SIQS (susceptible-infected-quarantined-susceptible) epidemic scheme, which took into account external infections and the potential for secondary infections. This epidemic model is well-suited for describing infections in computer networks, where the quarantine capacity can be likened to the number of working people (IT professionals) available to restore an infected computer. We presented numerical examples and sensitivity analysis to illustrate the behavior and performance of the system under different scenarios and parameter values. We show that the state capacities and the infection rates have significant effects on the evolution and extinction of the epidemic. We note that the optimal number of employed technicians can be identified, aiming to keep the computer network functional. Higher internal infection rates significantly affect the sustainability of the computer network, while controlling external infections is not always feasible. On the other hand, faster detection rates and higher malware elimination rates will considerably increase the number of computers that remain operational in the long term. Consequently, the quality of services provided by IT technicians plays a crucial role in the system’s viability.

Weak Ergodicity in G-NHMS

In the present we provide the definition of the new concept of the General Non-Homogeneous Markov System (G-NHMS) and establish the expected population structure of a NHMS in the various states. These results will be the basis to build on the new concepts and the basic theorems of what follows.We then establish the set of all possible expected relative distributions of the initial number of memberships at time t and all possible expected relative distributions of the expansion memberships at time t. We call this set the general expected relative population structure in the states of a G-NHMS. We then proceed by providing the new definitions of weak ergodicity in a G-NHMS and weak ergodicity with a geometrical rate of convergence. We then prove the Theorem 4 which is a new building block in the theory of G-NHMS. We also prove a similar theorem under the assumption that relative expansion of the population vanishes at infinity.We then provide a generalization of the coupling theorem for populations. We proceed then to study the asymptotic behavior of a G-NHMS when the input policy consists of independent non-homogeneous Poisson variates for each time interval t-1,t\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\left( t-1,t\\right] $$\\end{document}. It is founded in Theorem 7 that it displays a kind of weak ergodicity behavior, that is, it converges at each step to the row of a stable matrix. This row is independent of the initial distribution and of the asymptotic input policy unlike the results in previous works. Hence it generalizes the result in that works. Finally we illustrate our results numerically for a manpower system with three states.

Strong Ergodicity in Nonhomogeneous Markov Systems with Chronological Order

In the present, we study the problem of strong ergodicity in nonhomogeneous Markov systems. In the first basic theorem, we relax the fundamental assumption present in all studies of asymptotic behavior. That is, the assumption that the inherent inhomogeneous Markov chain converges to a homogeneous Markov chain with a regular transition probability matrix. In addition, we study the practically important problem of the rate of convergence to strong ergodicity for a nonhomogeneous Markov system (NHMS). In a second basic theorem, we provide conditions under which the rate of convergence to strong ergodicity is geometric. With these conditions, we in fact relax the basic assumption present in all previous studies, that is, that the inherent inhomogeneous Markov chain converges to a homogeneous Markov chain with a regular transition probability matrix geometrically fast. Finally, we provide an illustrative application from the area of manpower planning.

Limiting Distributions of a Non-Homogeneous Markov System in a Stochastic Environment in Continuous Time

The stochastic process non-homogeneous Markov system in a stochastic environment in continuous time (S-NHMSC) is introduced in the present paper. The ordinary non-homogeneous Markov process is a very special case of an S-NHMSC. I studied the expected population structure of the S-NHMSC, the first central classical problem of finding the conditions under which the asymptotic behavior of the expected population structure exists and the second central problem of finding which expected relative population structures are possible limiting ones, provided that the limiting vector of input probabilities into the population is controlled. Finally, the rate of convergence was studied.

Rate of convergence in fuzzy non homogeneous Markov systems

ABSTRACTCertain aspects of convergence rate in a non-homogeneous Markov system (NHMS) using fuzzy set theory and fuzzy reasoning are presented in this paper. More specifically, a fuzzy non-homogeneous Markov system is considered where fuzzy inference systems are used to estimate the transition probabilities, the input and loss probabilities and the population structure of the system. It is proved that under some conditions easily met in practice, the rate of convergence of the sequence of the relative population structures in such a system is geometric, i.e. it converges to its limit geometrically fast.

Adaptive sequential importance sampling technique for short‐term composite power system adequacy evaluation

The well-known sequential Monte Carlo simulation is a powerful tool to analyse short-term reliability of complicated composite power system. However, the corresponding computational burden is tremendous when applied to a highly reliable system. Aiming at improving the simulation efficiency, an adaptive importance sampling technology is proposed in this study. The proposed method, on the basis of component state duration sampling technique combined with cross-entropy method, is able to provide reliability evaluation of highly reliable non-homogeneous Markov system. The Weibull and lognormal distributions are considered to describe repair time of individual components comprising a system. Through the case studies conducted on a reinforced Roy Billinton reliability test system, it is validated that the proposed method is effective and of high efficiency and the efficiency gain against the crude sequential Monte Carlo simulation is robust to variation of load level and lead timescale. The proposed method is useful and efficient to timely monitor the system operating pressure under different lead timescales.

Markov Systems in a General State Space

In this article, we introduce and study Markov systems on general spaces (MSGS) as a first step of an entire theory on the subject. Also, all the concepts and basic results needed for this scope are given and analyzed. This could be thought of as an extension of the theory of a non homogeneous Markov system (NHMS) and that of a non homogeneous semi-Markov system on countable spaces, which has realized an interesting growth in the last thirty years. In addition, we study the asymptotic behaviour or ergodicity of Markov systems on general state spaces. The problem of asymptotic behaviour of Markov chains has been central for finite or countable spaces since the foundation of the subject. It has also been basic in the theory of NHMS and NHSMS. Two basic theorems are provided in answering the important problem of the asymptotic distribution of the population of the memberships of a Markov system that lives in the general space (X, ℬ(X)). Finally, we study the total variability from the invariant measure of the Markov system given that there exists an asymptotic behaviour. We prove a theorem which states that the total variation is finite. This problem is known also as the coupling problem.

Rate of convergence, asymptotically attainable structures and sensitivity in non-homogeneous Markov systems with fuzzy states

This paper is concerned with the rate of convergence, the asymptotically attainable structures and the sensitivity of a non-homogeneous Markov system (NHMS, in short) with fuzzy states. More specifically, it is proved that under some realistic assumptions easily met in practice, the convergence rate of the relative population structure to its limit is geometric. Furthermore, the asymptotically attainable structures in a NHMS with fuzzy states are given. More precisely, the problem of finding which states are possible as limiting ones in a NHMS with fuzzy states is studied, provided that the limit of the sequence of the input probability vectors is controlled. Moreover, the sensitivity of the limiting expected population structure on perturbations of the limiting input probabilities for the system is also studied. Two applications (in Cognitive Science and in Manpower Planning) are given, where the sensitivity of two NHMS with three states is studied, provided that the limiting input probability vector is perturbed.

Theory of Markov systems with fuzzy states

In this paper the theory of fuzzy logic is combined with the theory of Markov systems and the concept of a Markov system with fuzzy states is introduced for the first time. The main objective is to develop a methodology of describing a Markov population system with fuzzy states. This is an effort to deal with the fact that in real applications one is often faced with fuzzy states. This paper investigates the properties of a Markov system with states that have imprecise boundaries. A full description of a nonhomogeneous Markov system with fuzzy states is outlined and the basic parameters of the system are provided. More specifically, the expected population structure of the system is found and given in a closed analytic form and the asymptotic behaviour of the system is provided under the more realistic assumption of a fuzzy set space.

Fuzzy Non-Homogeneous Markov Systems

In this paper the theory of fuzzy logic and fuzzy reasoning is combined with the theory of Markov systems and the concept of a fuzzy non-homogeneous Markov system is introduced for the first time. This is an effort to deal with the uncertainty introduced in the estimation of the transition probabilities and the input probabilities in Markov systems. The asymptotic behaviour of the fuzzy Markov system and its asymptotic variability is considered and given in closed analytic form. Moreover, the asymptotically attainable structures of the system are estimated also in a closed analytic form under some realistic assumptions. The importance of this result lies in the fact that in most cases the traditional methods for estimating the probabilities can not be used due to lack of data and measurement errors. The introduction of fuzzy logic into Markov systems represents a powerful tool for taking advantage of the symbolic knowledge that the experts of the systems possess.

Aspirations and priorities in a three phase approach of a nonhomogeneous Markov system

Aspiration levels and priorities using goal programming are employed in a manpower nonhomogeneous Markov system (NHMS) which evolves in three phases, the transient, the semi-transient and the equilibrium phase. The general goal programming framework is used in several variations which depend on the phase, in order to detect appropriate input policies that can achieve a satisfactory trade off between operational cost and target attainability. The ultimate target is to guide the system towards an asymptotically attainable structure that is considered to be satisfactory.

The perturbed nonhomogeneous Markov system

In this paper we introduce for the first time the concept of a perturbed nonhomogeneous Markov system (P-NHMS). The expected population structure is found and its asymptotic behavior is provided under more realistic assumptions than previous studies, by relaxing the assumption that the imbedded nonhomogeneous Markov chain of a NHMS is converging to a homogeneous Markov chain. Also we study the sensitivity of the limiting expected structure on perturbations of the limiting input probabilities for a NHMS. Moreover, the asymptotic behavior of the variances and covariances of the class sizes of a P-NHMS is considered and found in an elegant closed analytic form.

Partial maintainability of a population model in a stochastic environment

We consider a non-homogeneous Markov system in a stochastic environment. Concepts of recruitment control are empoyed in order to study the probabilities of partially maintaining population structures under this establishment. Two conditions taken from the deterministic case of the partial control problem are further refined, providing the basis of the calculation of these probabilities. The stochastic environment is realized by pools of alternative transitions. Expected regions of partial maintainability are determined through the alternative transition policies calculated by a selection mechanism, the compromise Markov chain. © 1998 John Wiley & Sons, Ltd.

Partial maintainability of a population model in a stochastic environment

We consider a non-homogeneous Markov system in a stochastic environment. Concepts of recruitment control are empoyed in order to study the probabilities of partially maintaining population structures under this establishment. Two conditions taken from the deterministic case of the partial control problem are further refined, providing the basis of the calculation of these probabilities. The stochastic environment is realized by pools of alternative transitions. Expected regions of partial maintainability are determined through the alternative transition policies calculated by a selection mechanism, the compromise Markov chain. © 1998 John Wiley & Sons, Ltd.

The perturbed non-homogeneous Markov system in continuous time

In the present paper, we present the concept of a perturbed non-homogeneous Markov system in continuous time. The expected population structure of the system is found and its asymptotic behaviour is provided under more realistic assumptions than in previous studies, by relaxing the assumption that the imbedded non-homogeneous Markov chain of the NHMS converges to a homogeneous Markov chain. Finally we provide the variances and convariances of the state sizes and we find their asymptotic behaviour as t∞ in closed analytic form. © 1998 John Wiley & Sons, Ltd.

The perturbed non‐homogeneous Markov system in continuous time

In the present paper, we present the concept of a perturbed non-homogeneous Markov system in continuous time. The expected population structure of the system is found and its asymptotic behaviour is provided under more realistic assumptions than in previous studies, by relaxing the assumption that the imbedded non-homogeneous Markov chain of the NHMS converges to a homogeneous Markov chain. Finally we provide the variances and convariances of the state sizes and we find their asymptotic behaviour as t→∞ in closed analytic form. © 1998 John Wiley & Sons, Ltd.

Cost models in nonhomogeneous Markov systems

We study the problem of introducing the phases through which a nonhomogeneous Markov system (NHMS) passes, as time goes to infinity. For each phase an appropriate objective function is introduced and a cost problem is formulated. Appropriate input policies subject to the cost objective functions which are established on the Markov manpower system are investigated. Finally, a procedure is provided that summarises the phases, the objective functions and the control policies commencing from the initial structure resulting to its stationary form.

Nonstationary cyclic behavior in Markov systems

We introduce the concept of the nonstationary cyclic Markov system as a natural extension of the cyclic systems found in recent literature. We find the sets of d-cyclic asymptotically attainable structures and establish the relationships that exist between these structures for a nonhomogeneous Markov system in a deterministic or stochastic environment. The inherent connection between nonstationary cyclicity and stochastic environment is examined. Also we investigate the conditions under which there exists a geometric convergence to the equilibrium.

Performability of electric-power systems modeled by non-homogeneous Markov chains

This paper treats a particular type of Markov system: time nonhomogeneous Markov system in discrete time. In order to adapt measures to this kind of system, performability measures are formulated, such as new indicators more dedicated to electrical power systems like 'instantaneous mean load curtailed' and 'mean energy not supplied on a time interval'. This method considers hazard-rate time variation in order to: 1) obtain more-accurate and more-instructive indicators; and 2) access new performability indicators that cannot be obtained by classical methods. An example from an Electricite De France electrical substation illustrates this.

Stochastic analysis of a non-homogeneous Markov system

3 stochastic analysis on the non-homogeneous Markov system is outlined firstly in this paper. A method is given for computing the expectations and central quadratic moments of the number of individuals in the various classes at discrete points of time. We also discuss the probability of attaining goal-structures and paths using recruitment control. Finally an illustration is provided of the present results in a manpower system