Mixed Hidden Markov Model Research Articles

Immunogenicity dynamics and covariate effects after satralizumab administration predicted with a hidden Markov model.

Immunogenicity is the propensity of a therapeutic protein to generate an immune response to itself. While reporting of antidrug antibodies (ADAs) is increasing, model-based analysis of such data is seldom performed. Model-based characterization of factors affecting the emergence and dissipation of ADAs may inform drug development and/or improve understanding in clinical practice. This analysis aimed to predict ADA dynamics, including the potential influence of individual covariates, following subcutaneous satralizumab administration. Satralizumab is a humanized IgG2 monoclonal recycling IL-6 receptor antagonist antibody approved for treating neuromyelitis optica spectrum disorder (NMOSD). Longitudinal pharmacokinetic (PK) and ADA data from 154 NMOSD patients in two pivotal Phase 3 studies (NCT02028884, NCT02073279) and PK data from one Phase 1 study (SA-001JP) in 72 healthy volunteers were available for this analysis. An existing population PK model was adapted to derive steady-state concentration without ADA for each patient. A mixed hidden Markov model (mHMM) was developed whereby three different states were identified: one absorbing Markov state for non-ADA developer, and two dynamic and inter-connected Markov states-transient ADA negative and positive. Satralizumab exposure and body mass index impacted transition probabilities and, therefore, the likelihood of developing ADAs. In conclusion, the mHMM model was able to describe the time course of ADA development and identify patterns of ADA development in NMOSD patients following treatment with satralizumab, which may allow for the formulation of strategies to reduce the emergence or limit the impact of ADA in the clinical setting.

Characterization of anti-drug antibody dynamics using a bivariate mixed hidden-markov model by nonlinear-mixed effects approach

Biological therapies may act as immunogenic triggers leading to the formation of anti-drug antibodies (ADAs). Population pharmacokinetic (PK) models can be used to characterize the relationship between ADA and drug disposition but often rely on the ADA bioassay results, which may not be sufficiently sensitive to inform on this characterization.In this work, a methodology that could help to further elucidate the underlying ADA production and impact on the drug disposition was explored. A mixed hidden-Markov model (MHMM) was developed to characterize the underlying (hidden) formation of ADA against the biologic, using certolizumab pegol (CZP), as a test drug. CZP is a PEGylated Fc free TNF-inhibitor used in the treatment of rheumatoid arthritis and other chronic inflammatory diseases.The bivariate MHMM used information from plasma drug concentrations and ADA measurements, from six clinical studies (n = 845), that were correlated through a bivariate Gaussian function to infer about two hidden states; production and no-production of ADA influencing PK. Estimation of inter-individual variability was not supported in this case. Parameters associated with the observed part of the model were reasonably well estimated while parameters associated with the hidden part were less precise. Individual state sequences obtained using a Viterbi algorithm suggested that the model was able to determine the start of ADA production for each individual, being a more assay-independent methodology than traditional population PK. The model serves as a basis for identification of covariates influencing the ADA formation, and thus has the potential to identify aspects that minimize its impact on PK and/or efficacy.

Joint Estimation of Behind-the-Meter Solar Generation in a Community

Distribution grid planning, control, and optimization require accurate estimation of solar photovoltaic (PV) generation and electric load in the system. Most of the small residential solar PV systems are installed behind-the-meter making only the net load readings available to the utilities. This paper presents an unsupervised framework for joint disaggregation of the net load readings of a group of customers into the solar PV generation and electric load. Our algorithm synergistically combines a physical PV system performance model for individual solar PV generation estimation with a statistical model for joint load estimation. The electric loads for a group of customers are estimated jointly by a mixed hidden Markov model (MHMM) which enables modeling the general load consumption behavior present in all customers while acknowledging the individual differences. At the same time, the model can capture the change in load patterns over a time period by the hidden Markov states. The proposed algorithm is also capable of estimating the key technical parameters of the solar PV systems. Our proposed method is evaluated using the net load, electric load, and solar PV generation data gathered from residential customers located in Austin, Texas. Testing results show that our proposed method reduces the mean squared error of state-of-the-art net-load disaggregation algorithms by 67%.

Mixed location scale hidden Markov model for the analysis of intensive longitudinal data

Hidden Markov models (HMM) presents an attractive analytical framework for capturing the state-switching process for auto-correlated data. These models have been extended to longitudinal data setting where simultaneous multiple processes are observed by including subject specific random effects. However, application of HMMs for intensive longitudinal data, where each subject gets measured intensively over relatively short period of time, has not been widely studied. In this paper, we extend the mixed hidden Markov model and allow subject heterogeneity with respect to the mean and within subject variance by including subject random effects in both perspectives. We focus on the application of this model to intensive longitudinal studies in psychological and behavioral research where individual’s latent states and state-switching process are of interest. Models are estimated using forward–backward algorithm via Bayesian sampling approach. Advantages over regular HMM and mixed HMM that only accounts for the subjects’ mean heterogeneity are illustrated through a series of simulation studies. Finally, models are applied to an adolescent mood study data set and results show that the proposed mixed location scale HMM achieves better model fit and more interpretative mood state identification in terms of state specific covariate effects compared to regular HMM and mixed HMM.

Handling underlying discrete variables with bivariate mixed hidden Markov models in NONMEM

Non-linear mixed effects models typically deal with stochasticity in observed processes but models accounting for only observed processes may not be the most appropriate for all data. Hidden Markov models (HMMs) characterize the relationship between observed and hidden variables where the hidden variables can represent an underlying and unmeasurable disease status for example. Adding stochasticity to HMMs results in mixed HMMs (MHMMs) which potentially allow for the characterization of variability in unobservable processes. Further, HMMs can be extended to include more than one observation source and are then multivariate HMMs. In this work MHMMs were developed and applied in a chronic obstructive pulmonary disease example. The two hidden states included in the model were remission and exacerbation and two observation sources were considered, patient reported outcomes (PROs) and forced expiratory volume (FEV1). Estimation properties in the software NONMEM of model parameters were investigated with and without random and covariate effect parameters. The influence of including random and covariate effects of varying magnitudes on the parameters in the model was quantified and a power analysis was performed to compare the power of a single bivariate MHMM with two separate univariate MHMMs. A bivariate MHMM was developed for simulating and analysing hypothetical COPD data consisting of PRO and FEV1 measurements collected every week for 60 weeks. Parameter precision was high for all parameters with the exception of the variance of the transition rate dictating the transition from remission to exacerbation (relative root mean squared error [RRMSE] > 150%). Parameter precision was better with higher magnitudes of the transition probability parameters. A drug effect was included on the transition rate probability and the precision of the drug effect parameter improved with increasing magnitude of the parameter. The power to detect the drug effect was improved by utilizing a bivariate MHMM model over the univariate MHMM models where the number of subject required for 80% power was 25 with the bivariate MHMM model versus 63 in the univariate MHMM FEV1 model and > 100 in the univariate MHMM PRO model. The results advocates for the use of bivariate MHMM models when implementation is possible.

Mixed Hidden Markov Models for Clinical Research with Discrete Repeated Measurements

A hidden Markov model (HMM) is a method for analyzing a sequence of transitions for a set of data by considering the outcomes Y to be output from latent state X, which has the Markov property. The HMM has been widely applied, with applications that include speech recognition, genomic analysis, and finance forecasting. The HMM was originally a method for dealing with single-process data. Thus, it is a natural extension to apply it to data with a repeated measure structure by incorporating random effects in it. This is called the mixed hidden Markov model (MHMM). With this extension, the MHMM was recently applied to clinical research data with repeated measurements, e.g. multiple sclerosis, alcohol consumption, and primary biliary cirrhosis. In relation to parameter inference, because regular HMM methods can be used in an MHMM framework, some legacy knowledge is applicable. The likelihood can be obtained by simply adding a random effect parameter to a single process HMM, and the conventional maximum-likelihood method can be used for parameter estimation. On the other hand, much work must still be performed. For instance, the mathematical property of the maximum likelihood estimator has not yet been thoroughly examined. In this study, the asymptotic normality and consistency of the maximum likelihood estimator of the MHMM concerned with time points are examined via simulation, and found that these properties were almost fine. These methods are applied to actual study data, and future perspectives are provided in the conclusion.

Dynamic latent trait models with mixed hidden Markov structure for mixed longitudinal outcomes

We propose a general Bayesian joint modeling approach to model mixed longitudinal outcomes from the exponential family for taking into account any differential misclassification that may exist among categorical outcomes. Under this framework, outcomes observed without measurement error are related to latent trait variables through generalized linear mixed effect models. The misclassified outcomes are related to the latent class variables, which represent unobserved real states, using mixed hidden Markov models (MHMMs). In addition to enabling the estimation of parameters in prevalence, transition and misclassification probabilities, MHMMs capture cluster level heterogeneity. A transition modeling structure allows the latent trait and latent class variables to depend on observed predictors at the same time period and also on latent trait and latent class variables at previous time periods for each individual. Simulation studies are conducted to make comparisons with traditional models in order to illustrate the gains from the proposed approach. The new approach is applied to data from the Southern California Children Health Study to jointly model questionnaire-based asthma state and multiple lung function measurements in order to gain better insight about the underlying biological mechanism that governs the inter-relationship between asthma state and lung function development.

A Bayesian HMM with random effects and an unknown number of states for DNA copy number analysis

Hidden Markov models (HMMs) have been shown to be a flexible tool for modelling complex biological processes. However, choosing the number of hidden states remains an open question and the inclusion of random effects also deserves more research, as it is a recent addition to the fixed-effect HMM in many application fields. We present a Bayesian mixed HMM with an unknown number of hidden states and fixed covariates. The model is fitted using reversible-jump Markov chain Monte Carlo, avoiding the need to select the number of hidden states. We show through simulations that the estimations produced are more precise than those from a fixed-effect HMM and illustrate its practical application to the analysis of DNA copy number data, a field where HMMs are widely used.

The analysis of disease biomarker data using a mixed hidden Markov model (Open Access publication)

A mixed hidden Markov model (HMM) was developed for predicting breeding values of a biomarker (here, somatic cell score) and the individual probabilities of health and disease (here, mastitis) based upon the measurements of the biomarker. At a first level, the unobserved disease process (Markov model) was introduced and at a second level, the measurement process was modeled, making the link between the unobserved disease states and the observed biomarker values. This hierarchical formulation allows joint estimation of the parameters of both processes. The flexibility of this approach is illustrated on the simulated data. Firstly, lactation curves for the biomarker were generated based upon published parameters (mean, variance, and probabilities of infection) for cows with known clinical conditions (health or mastitis due to Escherichia coli or Staphylococcus aureus). Next, estimation of the parameters was performed via Gibbs sampling, assuming the health status was unknown. Results from the simulations and mathematics show that the mixed HMM is appropriate to estimate the quantities of interest although the accuracy of the estimates is moderate when the prevalence of the disease is low. The paper ends with some indications for further developments of the methodology.

The analysis of disease biomarker data using a mixed hidden Markov model (Open Access publication)

The analysis of disease biomarker data using a mixed hidden Markov model (Open Access publication)

Mixed Hidden Markov Models

Hidden Markov models (HMMs) are a useful tool for capturing the behavior of overdispersed, autocorrelated data. These models have been applied to many different problems, including speech recognition, precipitation modeling, and gene finding and profiling. Typically, HMMs are applied to individual stochastic processes; HMMs for simultaneously modeling multiple processes—as in the longitudinal data setting—have not been widely studied. In this article I present a new class of models, mixed HMMs (MHMMs), where I use both covariates and random effects to capture differences among processes. I define the models using the framework of generalized linear mixed models and discuss their interpretation. I then provide algorithms for parameter estimation and illustrate the properties of the estimators via a simulation study. Finally, to demonstrate the practical uses of MHMMs, I provide an application to data on lesion counts in multiple sclerosis patients. I show that my model, while parsimonious, can describe the heterogeneity among such patients.