Hidden Markov Process Research Articles

Automatic Segmentation of Stabilometric Signals Using Hidden Markov Model Regression

Posture analysis in quiet standing is an essential element in evaluating human balance control. Many factors enhance the human control system’s ability to maintain stability, such as the visual system and base of support (feet) placement. In contrast, many neural pathologies, such as Parkinson’s disease (PD) and cerebellar disorder, disturb human stability. This paper addresses the problem of the automatic segmentation of stabilometric signals recorded under four different conditions related to vision and foot position. This is achieved for both control subjects and PD subjects. A hidden Markov model (HMM)-regression-based approach is used to carry out the segmentation between the different conditions using simple and multiple regression processes. Twenty-eight control subjects and thirty-two PD subjects participated in this study. They were asked to stand upright while recording stabilometric signals in mediolateral and anteroposterior directions under two permutations: feet apart and together with eyes open or closed. The results show high values for the correct segmentation rates, up to 98%, for the separation between the different conditions. The present findings could help clinicians better understand the motor strategies used by the patients during their orthostatic postures and may guide the rehabilitation process. The proposed method compares favorably with standard segmentation approaches. Note to Practitioners —In this paper, the problem of human balance control assessment is analyzed through the segmentation of the multidimensional time series of the center of pressure (CoP) displacement measurements during orthostatic postures of healthy and Parkinsonian subjects. The proposed model for automatic temporal segmentation is a specific statistical latent process model that assumes that the observed stabilometric sequence is governed by a sequence of hidden (unobserved) states or conditions. More specifically, the proposed approach is based on a specific multiple regression model that incorporates a hidden Markov process that governs the switching from one condition to another over time. The model is learned in an unsupervised context by maximizing the observed data log-likelihood via a dedicated expectation–maximization algorithm. We applied it on a real-world automatic CoP displacement excursion segmentation problem and assessed its performance by performing comparisons with alternative approaches, including well-known supervised static classifiers and the standard HMM. The results obtained are very encouraging and show that the proposed approach is quite competitive, though it works in an entirely unsupervised fashion and does not requires a feature extraction preprocessing step. The present findings could help clinicians to better understand the motor strategies used by the patients during their orthostatic postures and may guide the rehabilitation process.

Estimating Density and Temperature Dependence of Juvenile Vital Rates Using a Hidden Markov Model.

Organisms in the wild have cryptic life stages that are sensitive to changing environmental conditions and can be difficult to survey. In this study, I used mark-recapture methods to repeatedly survey Anaea aidea (Nymphalidae) caterpillars in nature, then modeled caterpillar demography as a hidden Markov process to assess if temporal variability in temperature and density influence the survival and growth of A. aidea over time. Individual encounter histories result from the joint likelihood of being alive and observed in a particular stage, and I have included hidden states by separating demography and observations into parallel and independent processes. I constructed a demographic matrix containing the probabilities of all possible fates for each stage, including hidden states, e.g., eggs and pupae. I observed both dead and live caterpillars with high probability. Peak caterpillar abundance attracted multiple predators, and survival of fifth instars declined as per capita predation rate increased through spring. A time lag between predator and prey abundance was likely the cause of improved fifth instar survival estimated at high density. Growth rates showed an increase with temperature, but the preferred model did not include temperature. This work illustrates how state-space models can include unobservable stages and hidden state processes to evaluate how environmental factors influence vital rates of cryptic life stages in the wild.

A Robbins–Monro Algorithm for Non‐Parametric Estimation of NAR Process with Markov Switching: Consistency

We approach the problem of non‐parametric estimation for autoregressive Markov switching processes. In this context, the Nadaraya–Watson‐type regression functions estimator is interpreted as a solution of a local weighted least‐square problem, which does not admit a closed‐form solution in the case of hidden Markov switching. We introduce a non‐parametric recursive algorithm to approximate the estimator. Our algorithm restores the missing data by means of a Monte Carlo step and estimates the regression function via a Robbins–Monro step. We prove that non‐parametric autoregressive models with Markov switching are identifiable when the hidden Markov process has a finite state space. Consistency of the estimator is proved using the strong α‐mixing property of the model. Finally, we present some simulations illustrating the performances of our non‐parametric estimation procedure.

Full-Duplex-Based Rate/Mode Adaptation Strategies for Wi-Fi/LTE-U Coexistence: A POMDP Approach

The rapid increase in wireless demand prompted the FCC to open up parts of the 5-GHz band for unlicensed access. This caught the interest of 4G/LTE providers, who wish to extend their LTE-A services to the unlicensed spectrum (LTE-U). In LTE-U, small-cell base stations aggregate unlicensed and licensed bands to increase the throughput. Wi-Fi/LTE-U coexistence is a challenging issue due to the different access mechanisms of these two systems, which may cause high collision rates and delays. By leveraging self-interference-suppression techniques, we propose joint mode/rate adaptation strategies for Wi-Fi/LTE-U coexistence. Specifically, a full-duplex enabled Wi-Fi station can transmit and receive data simultaneously to increase the throughput, or transmit and sense (TS mode) simultaneously to monitor the LTE-U activity. We model the LTE-U interference as a hidden Markov process, and solve the problem of jointly adapting Wi-Fi rates/modes using a framework of partially observable Markov decision process. A detection approach based on the sliding window correlator is analyzed for the TS mode, which can differentiate between Wi-Fi and LTE-U signals. Our results indicate that our scheme provides 1.5x (1.9x) average throughput gain for Wi-Fi system in the low (high) signal-to-interference-and-noise regime relative to a half-duplex-based scheme.

Interacting default intensity with a hidden Markov process

In this paper we consider a reduced-form intensity-based credit risk model with a hidden Markov state process. A filtering method is proposed for extracting the underlying state given the observation processes. The method can be applied to a wide range of problems. Based on this model, we derive the joint distribution of multiple default times without imposing stringent assumptions on the form of default intensities. Closed-form formulas for the distribution of default times are obtained which are then applied to solve a number of practical problems such as hedging and pricing credit derivatives. The method and numerical algorithms presented can be applicable to various forms of default intensities.

Using Simulation to Model and Validate Invasive Breast Cancer Progression in Women in the Study and Control Groups of the Canadian National Breast Screening Studies I and II.

Modeling breast cancer progression and the effect of various risk is helpful in deciding when a woman should start and end screening, and how often the screening should be undertaken. We modeled the natural progression of breast cancer using a hidden Markov process, and incorporated the effects of covariates. Patients are women aged 50-59 (older) and 40-49 (younger) years from the Canadian National Breast Screening Studies. We included prevalent cancers, estimated the screening sensitivities and rates of over-diagnosis, and validated the models using simulation. We found that older women have a higher rate of transition from a healthy to preclinical state and other causes of death but a lower rate of transition from preclinical to clinical state. Reciprocally, younger women have a lower rate of transition from a healthy to preclinical state and other causes of death but a higher rate of transition from a preclinical to clinical state. Different risk factors were significant for the age groups. The mean sojourn times for older and younger women were 2.53 and 2.96 years, respectively. In the study group, the sensitivities of the initial physical examination and mammography for older and younger women were 0.87 and 0.81, respectively, and the sensitivity of the subsequent screens were 0.78 and 0.53, respectively. In the control groups, the sensitivities of the initial physical examination for older and younger women were 0.769 and 0.671, respectively, and the sensitivity of the subsequent physical examinations for the control group aged 50-59 years was 0.37. The upper-bounds for over-diagnosis in older and younger women were 25% and 27%, respectively. The present work offers a basis for the better modeling of cancer incidence for a population with the inclusion of prevalent cancers.

Pricing Sovereign Contingent Convertible Debt

We develop a pricing model for sovereign contingent convertible bonds (S-CoCo) with payment standstills triggered by a sovereign's credit default swap CDS spread. One innovation is the modeling of CDS spread regime switching which is prevalent during crises. Regime switching is modeled as a hidden Markov process and is integrated with a stochastic process of spread levels to obtain S-CoCo prices through simulation. The paper goes a step further and uses the pricing model in a Longstaff-Schwartz. American option pricing framework to compute state contingent S-CoCo prices at some risk horizon, thus facilitating risk management. Dual trigger pricing is also discussed using the idiosyncratic CDS spread for the sovereign debt together with a broad market index. Extensive numerical results are reported using S-CoCo designs for Greece, Italy and Germany with both the pricing and contingent pricing models.

Fatigue damage diagnostics and prognostics of composites utilizing structural health monitoring data and stochastic processes

The procedure of damage accumulation in composites, especially during fatigue loading, is a complex phenomenon of stochastic nature which depends on a number of parameters such as type and frequency of loading, stacking sequence, material properties, and so on. Toward condition-based health monitoring and decision making, the need for not only diagnostic but also prognostic tools rises and draws increasing attention in the last few years. To this direction, we model the damage evolution in composites as a doubly stochastic hidden Markov process that manifests itself via structural health monitoring observations, that is, acoustic emission data. The damage process is modeled via an extension of the classic hidden Markov models to account for nonhomogeneity, that is, age dependence in state transitions. The observations come from acoustic emission data recorded throughout fatigue testing of open-hole carbon–epoxy coupons. A procedure that utilizes multiple observation sequences from a training dataset and estimates in a maximum likelihood sense the optimal model parameters is presented and applied in unseen data via a cross-validation rationale. Diagnostics of the most likely health state determination, average degradation level, and prognostics of the remaining useful life are among the capabilities of the presented stochastic model.

Granger Causality and Regime Inference in Markov Switching VAR Models with Bayesian Methods

Summary In this paper, we derive restrictions for Granger noncausality in MS-VAR models and show under what conditions a variable does not affect the forecast of the hidden Markov process. To assess the noncausality hypotheses, we apply Bayesian inference. The computational tools include a novel block Metropolis–Hastings sampling algorithm for the estimation of the underlying models. We analyze a system of monthly US data on money and income. The results of testing in MS-VARs contradict those obtained with linear VARs: the money aggregate M1 helps in forecasting industrial production and in predicting the next period's state. Copyright © 2016 John Wiley & Sons, Ltd.

Dynamic Detection of Transmission Line Outages Using Hidden Markov Models

We study the problem of detecting transmission line outages in power grids. We model the time series of power network measurements as a hidden Markov process, and formulate the line outage detection problem as an inference problem. Due to the physical nature of the line failure dynamics, the transition probabilities of the hidden Markov Model are sparse. Taking advantage of this fact, we further propose an approximate inference algorithm using particle filtering, which takes in the times series of power network measurements and produces a probabilistic estimation of the status of the transmission line status. We then assess the performance of the proposed algorithm with case studies. We show that it outperforms the conventional static line outage detection algorithms and is robust to both measurement noise and model parameter errors.

Thermodynamics of the binary symmetric channel

We study a hidden Markov process which is the result of a transmission of the binary symmetric Markov source over the memoryless binary symmetric channel. This process has been studied extensively in Information Theory and is often used as a benchmark case for the so-called denoising algorithms. Exploiting the link between this process and the 1D Random Field Ising Model (RFIM), we are able to identify the Gibbs potential of the resulting Hidden Markov process. Moreover, we obtain a stronger bound on the memory decay rate. We conclude with a discussion on implications of our results for the development of denoising algorithms.

Joint optimization of lot-sizing and maintenance policy for a partially observable two-unit system

In this paper, we present a new model to find the jointly optimal economic manufacturing quantity (EMQ) and preventive maintenance (PM) policy for a complex production facility. Unlike the previous joint models which dealt with EMQ and maintenance policy considering a single unit production facility and traditional maintenance approaches, we consider a production facility which consists of two modules with economic dependence. The more expensive module (unit 1) is subject to condition monitoring (CM), and only the age information of the second module (unit 2) is available, which follows a general distribution. The deterioration process of unit 1 is modeled as a continuous time hidden-Markov process. CM data is available at the end of each production run, and it provides only partial information about the hidden state of unit 1. The failure state of unit 1 is observable at any time. The objective is to develop a jointly optimal lot sizing and maintenance policy for a two-unit production facility using multivariate Bayesian control approach by minimizing the long-run expected average cost per unit time. The problem is formulated and solved in the semi-Markov decision process (SMDP) framework. Also, a formula for the mean residual life (MRL) of the production facility is derived, which is an important statistic for practical applications. A practical example of the wind turbine CM and maintenance is provided and a comparison with other policies shows an outstanding performance of the new model and the control policy proposed in this paper.

Minimum MS. E. Gerber’s Lemma

Mrs. Gerber’s Lemma lower bounds the entropy at the output of a binary symmetric channel in terms of the entropy of the input process. In this paper, we lower bound the output entropy via a different measure of input uncertainty, pertaining to the minimum mean squared error prediction cost of the input process. We show that in many cases our bound is tighter than the one obtained from Mrs. Gerber’s Lemma. As an application, we evaluate the bound for binary hidden Markov processes, and obtain new estimates for the entropy rate.

Effectiveness of Bayesian filters: An information fusion perspective

The general solution for dynamic state estimation is to model the system as a hidden Markov process and then employ a recursive estimator of the prediction–correction format (of which the best known is the Bayesian filter) to statistically fuse the time-series observations via models. The performance of the estimator greatly depends on the quality of the statistical mode assumed. In contrast, this paper presents a modeling-free solution, referred to as the observation-only (O2) inference, which infers the state directly from the observations. A Monte Carlo sampling approach is correspondingly proposed for unbiased nonlinear O2 inference. With faster computational speed, the performance of the O2 inference has identified a benchmark to assess the effectiveness of conventional recursive estimators where an estimator is defined as effective only when it outperforms on average the O2 inference (if applicable). It has been quantitatively demonstrated, from the perspective of information fusion, that a prior “biased” information (which inevitably accompanies inaccurate modelling) can be counterproductive for a filter, resulting in an ineffective estimator. Classic state space models have shown that a variety of Kalman filters and particle filters can easily be ineffective (inferior to the O2 inference) in certain situations, although this has been omitted somewhat in the literature.

Sequential Bayesian inference for implicit hidden Markov models and current limitations

Hidden Markov models can describe time series arising in various fields of science, by treating the data as noisy measurements of an arbitrarily complex Markov process. Sequential Monte Carlo (SMC) methods have become standard tools to estimate the hidden Markov process given the observations and a fixed parameter value. We review some of the recent developments allowing the inclusion of parameter uncertainty as well as model uncertainty. The shortcomings of the currently available methodology are emphasised from an algorithmic complexity perspective. The statistical objects of interest for time series analysis are illustrated on a toy “Lotka-Volterra” model used in population ecology. Some open challenges are discussed regarding the scalability of the reviewed methodology to longer time series, higher-dimensional state spaces and more flexible models.

Option Pricing Under Jump-Diffusion Processes with Regime Switching

We study an incomplete market model, based on jump-diffusion processes with parameters that are switched at random times. The set of equivalent martingale measures is determined. An analogue of the fundamental equation for the option price is derived. In the case of the two-state hidden Markov process we obtain explicit formulae for the option prices. Furthermore, we numerically compare the results corresponding to different equivalent martingale measures.

On the use of Hidden Markov Processes and auto-regressive filters to incorporate indoor bursty wireless channels into network simulation platforms

In this paper we thoroughly analyze two alternatives to replicate the bursty behavior that characterizes real indoor wireless channels within Network Simulation platforms. First, we study the performance of an improved Hidden Markov Process model, based on a time-wise configuration so as to decouple its operation from any particular traffic pattern. We compare it with the behavior of Bursty Error Model Based on an Auto-Regressive Filter, a previous proposal of ours that emulates the received Signal to Noise Ratio by means of an auto-regressive filter that captures the “memory” assessed in real measurements. We also study the performance of one of the legacy approaches intrinsically offered by most network simulation frameworks. By means of a thorough simulation campaign, we demonstrate that our two models are able to offer a much more realistic behavior, yet maintaining an affordable response in terms of computational complexity.

Novel Strategy to Improve the Performance of Localization in WSN.

A novel strategy of discrete energy consumption model for WSN based on quasi Monte Carlo and crude Monte Carlo method is developed. In our model the discrete hidden Markov process plays a major role in analyzing the node location in heterogeneous media. In this energy consumption model we use both static and dynamic sensor nodes to monitor the optimized energy of all sensor nodes in which every sensor state can be considered as the dynamic Bayesian network. By using this method the power is assigned in terms of dynamic manner to each sensor over discrete time steps to control the graphical structure of our network. The simulation and experiment result shows that our proposed methods are better in terms of localization accuracy and possess minimum computational time over existing localization method.

Granger Causality and Regime Inference in Bayesian Markov-Switching VARs

We derive restrictions for Granger noncausality in Markov-switching vector autoregressive models and also show under which conditions a variable does not affect the forecast of the hidden Markov process. Based on Bayesian approach to evaluating the hypotheses, the computational tools for posterior inference include a novel block Metropolis-Hastings sampling algorithm for the estimation of the restricted models. We analyze a system of monthly US data on money and income. The test results in MS-VARs contradict those in linear VARs: the money aggregate M1 is useful for forecasting income and for predicting the next period’s state.

Optimal condition-based maintenance policy for a partially observable system with two sampling intervals

In this paper, we propose an optimal Bayesian control policy with two sampling intervals minimizing the long-run expected average maintenance cost per unit time for a partially observable deteriorating system. Unlike the previous optimal Bayesian approaches which used periodic sampling models with equidistant intervals, a novel sampling methodology is proposed which is characterized by two sampling intervals and two control thresholds. The deterioration process is modeled as a 3-state continuous time hidden-Markov process with two unobservable operating-states and an observable failure state. At each sampling epoch, the multivariate observation data provides only partial information about the actual state of the system. We start observing the system with a longer sampling interval. If the posterior probability that the system is in the warning state exceeds a warning limit, observations are taken more frequently, i.e., the sampling interval changes to a shorter one, and if the posterior probability exceeds a maintenance limit, the full inspection is performed, followed possibly by preventive maintenance. We formulate the maintenance control problem in a partially observable Markov decision process (POMDP) framework to find the two optimal control limits and two sampling intervals. Also, the mean residual life (MRL) of the system is calculated as a function of the posterior probability. A numerical example is provided and comparison of the proposed scheme with several alternative sampling and maintenance control strategies is carried out.