Markov Model Formulation Research Articles

Performance analysis and control F-policy for fault-tolerant system with working vacation

This investigation presents a Markov model for the performance analysis of the fault tolerant machining system with failure-prone server and supported by warm standbys. To utilize the server’s idle time, provision of server’s working vacation has been done which make the system cost effective. The online and warm standby machines may fail and can be repaired by a single skilled repairman. Due to capacity constraint, when the system reaches its full capacity, no more jobs for repairing of failed machines are allowed until the workload of repair jobs reduces to a threshold level ‘F’. Before initiating the repair of the failed machines in case of coming back from the vacation state, the server requires the setup time. To make system fault tolerable, apart from standby provisioning and repairing of failed machines, the concepts of reboot and recovery are included for the formulation of Markov model. The various performance measures including the reliability indices are derived by using the transient probabilities which are computed using Runge–Kutta method. By taking a suitable numerical illustration, various system indices are examined with respect to different parameters. The computational tractability and sensitivity analysis carried out for the established metrics will provides valuable insights for the maintainability and up-gradation of the existing machining systems.

Estimating recurrence and incidence of preterm birth subject to measurement error in gestational age: A hidden Markov modeling approach.

Prediction of preterm birth as well as characterizing the etiological factors affecting both the recurrence and incidence of preterm birth (defined as gestational age at birth≤37wk) are important problems in obstetrics. The National Institute of Child Health and Human Development (NICHD) consecutive pregnancy study recently examined this question by collecting data on a cohort of women with at least 2 pregnancies over a fixed time interval. Unfortunately, measurement error due to the dating of conception may induce sizable error in computing gestational age at birth. This article proposes a flexible approach that accounts for measurement error in gestational age when making inference. The proposed approach is a hidden Markov model that accounts for measurement error in gestational age by exploiting the relationship between gestational age at birth and birth weight. We initially model the measurement error as being normally distributed, followed by a mixture of normals that has been proposed on the basis of biological considerations. We examine the asymptotic bias of the proposed approach when measurement error is ignored and also compare the efficiency of this approach to a simpler hidden Markov model formulation where only gestational age and not birth weight is incorporated. The proposed model is compared with alternative models for estimating important covariate effects on the risk of subsequent preterm birth using a unique set of data from the NICHD consecutive pregnancy study.

Markovian analysis of unreliable multi-components redundant fault tolerant system with working vacation and F-policy

AbstractThis investigation presents admission control policy for the fault tolerant system comprising of multi-components operating machines and multiple types of warm standbys under the maintenance of single unreliable server. The concepts of F-policy which deals with the controlling of inflow of failed machines and imperfect coverage are incorporated to make Markov model formulation more realistic. The unreliable repairman takes some time before allowing the failed machines to wait in the system for the repair job. The successive over relaxation (SOR) method is used to obtain the system state probabilities at the steady state which are further used to evaluate other system indices including the mean queue length of failed machines, mean number of standby machines, throughput, etc. By constructing the cost function in terms of various performance indices and associated cost elements, the optimal repair rate is determined so that the maintenance of the concerned failure prone FTS can be done in an economi...

One-Shot Learning of Human Activity With an MAP Adapted GMM and Simplex-HMM.

This paper presents a novel activity class representation using a single sequence for training. The contribution of this representation lays on the ability to train an one-shot learning recognition system, useful in new scenarios where capturing and labeling sequences is expensive or impractical. The method uses a universal background model of local descriptors obtained from source databases available on-line and adapts it to a new sequence in the target scenario through a maximum a posteriori adaptation. Each activity sample is encoded in a sequence of normalized bag of features and modeled by a new hidden Markov model formulation, where the expectation-maximization algorithm for training is modified to deal with observations consisting in vectors in a unit simplex. Extensive experiments in recognition have been performed using one-shot learning over the public datasets Weizmann, KTH, and IXMAS. These experiments demonstrate the discriminative properties of the representation and the validity of application in recognition systems, achieving state-of-the-art results.

Analysis of blood transfused in a city hospital with the principle of Markov-dependence

The management of blood bank is an integral part of health delivery system and it is centred on the donation and transfusion of blood. The four major classification of blood is considered in the formulation of Markov model with a first- order dependence. The data of blood transfused in a city hospital was studied and the result revealed that the blood that is donated and received by the same blood group constitute the major blood needed in the hospital with 100%, 99.1%, 98.3%, and 98.5% for blood groups AB,B,A and O respectively. There will be a little variation in the blood needed for transfusion in the future on the basis of the present when the initial values of the transition probabilities P12 (t), P13 (t), P14 (t), P24 (t), and P34 (t) (t = 0) are 0.00354, 0.00202, 0.00865, 0.01653 and 0.00884 respectively. The information about blood bank inventory is important for a successful health care delivery. If such information is available in every city hospital, it may be possible to move blood from one hospital to another during emergency. Mathematics Subject Classification: 60J28

Ordinal latent variable models and their application in the study of newly licensed teenage drivers.

In a unique longitudinal study of teen driving, risky driving behavior and the occurrence of crashes or near crashes are measured prospectively over the first 18 months of licensure. Of scientific interest is relating the two processes and developing a predictor of crashes from previous risky driving behavior. In this work, we propose two latent class models for relating risky driving behavior to the occurrence of a crash or near crash event. The first approach models the binary longitudinal crash/near crash outcome using a binary latent variable which depends on risky driving covariates and previous outcomes. A random effects model introduces heterogeneity among subjects in modeling the mean value of the latent state. The second approach extends the first model to the ordinal case where the latent state is composed of K ordinal classes. Additionally, we discuss an alternate hidden Markov model formulation. Estimation is performed using the expectation-maximization (EM) algorithm and Monte Carlo EM. We illustrate the importance of using these latent class modeling approaches through the analysis of the teen driving behavior.

Operational Prediction of Groundwater Fluctuation in South Florida using Sequence Based Markovian Stochastic Model

The ecosystem of South Florida is characterized by a vast wetland system, karst surficial hydrogeology, and extended coastal boundary. The ecosystem is poised under risks of: ecological failure due to increased fragmentation by urbanization; groundwater flow disruption because of sinkhole formation; and intrusion of oceanic water with decreasing water table head because of drought or over pumping. It was found important to synthesize the spatiotemporal state of the groundwater hydrology and also develop a forecasting model to support the intensive management and monitoring in place. In this study, an objective was set to develop a stochastic sequence model capable of forecasting groundwater levels on a monthly span at a daily time scale. The groundwater level simulation was conceptualized as a sequence of daily fluctuating states of magnitudes and patterns that has a defined probability of occurrence. The model setup involved representation of daily fluctuation magnitudes in ten states and pattern changes in three states. The sequential occurrence of states of magnitudes and patterns at each time step was used for estimation of the transitional probabilities and employed in a hidden Markov model frame work for ensemble generation and estimation of posterior probabilities. A realization was chosen based on the highest maximum likelihood ratio of 90% and smallest root mean square error of 0.05–0.12 m against the historical data. A monthly forecasting at daily time step was done dynamically incorporating observed data at each time step and revising prior and posterior probability estimation in the hidden Markov model formulation. A case study was conducted at three well sites, which are situated at three different hydrogeologic settings. The model not only reproduced annual groundwater fluctuation patterns but also forecasted preceding monthly fluctuations at maximum likelihood ratio above 90% and root mean square error below 0.15 m. A further study was recommended first to analyze break point parametric estimation for seasonal analysis, and secondly to integrate the approach in other hydrological models for the purpose of synthetic groundwater fluctuation generation.

A Stochastic Approach for Blind Estimation of Multiple Co-Channel Signals Received at an Antenna Array

A stochastic maximum likelihood approach for blind estimation of co-channel signals received at an antenna array is proposed in this letter. A hidden Markov model formulation of the problem is introduced and the Baum-Welch algorithm for the associated stochastic maximum likelihood estimation procedure is modified. The performance of the proposed algorithm based on the evaluation of approximate Cramer-Rao bounds is studied. Finally, some simulation results are presented.

Selection of the strategy of plant control based on recognition of its state patterns

The realization of the optimum strategy of control of complex dynamic plants is connected with necessity of solving the problem of the plant state classification and decision making about its membership to some class by the results of observation of its inspected parameters. At arbitrary assigning the boundaries of state classes, the classification of the states is carried out with the errors influencing to the quality of control. The probabilistic models of plant state classes and the methods of determination of classes boundaries based on the representation of knowledge about the plant behaviour in the form of Markov model are offered in the paper.

A statistical model of possible future human actions

Abstract A critical review has been carried out of models of possible future human actions during the long-term post-closure period of a radioactive waste repository. Various Markov models have been considered as alternatives to the more conventional Poisson model, and the problems of parameterisation have been addressed. Where the Poisson model unduly exaggerates the intrusion risk, some form of Markov model may have to be introduced. This situation may well arise for shallow repositories, but it is thought less likely for deep repositories. Consideration of state transitions within a Markov model highlights the value of increasing the availability of site information.