Monte Carlo Expectation Research Articles

Copula Gaussian graphical models with penalized ascent Monte Carlo EM algorithm

Typical data that arise from surveys, experiments, and observational studies include continuous and discrete variables. In this article, we study the interdependence among a mixed (continuous, count, ordered categorical, and binary) set of variables via graphical models. We propose an ℓ1‐penalized extended rank likelihood with an ascent Monte Carlo expectation maximization approach for the copula Gaussian graphical models and establish near conditional independence relations and zero elements of a precision matrix. In particular, we focus on high‐dimensional inference where the number of observations are in the same order or less than the number of variables under consideration. To illustrate how to infer networks for mixed variables through conditional independence, we consider two datasets: one in the area of sports and the other concerning breast cancer.

A mixture hierarchical model for response times and response accuracy

In real testing, examinees may manifest different types of test-taking behaviours. In this paper we focus on two types that appear to be among the more frequently occurring behaviours – solution behaviour and rapid guessing behaviour. Rapid guessing usually happens in high-stakes tests when there is insufficient time, and in low-stakes tests when there is lack of effort. These two qualitatively different test-taking behaviours, if ignored, will lead to violation of the local independence assumption and, as a result, yield biased item/person parameter estimation. We propose a mixture hierarchical model to account for differences among item responses and response time patterns arising from these two behaviours. The model is also able to identify the specific behaviour an examinee engages in when answering an item. A Monte Carlo expectation maximization algorithm is proposed for model calibration. A simulation study shows that the new model yields more accurate item and person parameter estimates than a non-mixture model when the data indeed come from two types of behaviour. The model also fits real, high-stakes test data better than a non-mixture model, and therefore the new model can better identify the underlying test-taking behaviour an examinee engages in on a certain item.

Segmented classification analysis with a class of rectangle-screened elliptical populations

In many practical situations, a statistical practitioner often faces a problem of classifying an object from one of the segmented (or screened) populations where the segmentation was conducted by a set of screening variables. This paper addresses this problem, proposing and studying yet another optimal rule for classification with segmented populations. A class of q-dimensional rectangle-screened elliptically contoured (RSEC) distributions is considered for flexibly modeling the segmented populations. Based on the properties of the RSEC distributions, a parametric procedure for the segmented classification analysis (SCA) is proposed. This includes motivation for the SCA as well as some theoretical propositions regarding its optimal rule and properties. These properties allow us to establish other important results which include an efficient estimation of the rule by the Monte Carlo expectation–conditional maximization algorithm and an optimal variable selection procedure. Two numerical examples making use of utilizing a simulation study and a real dataset application and advocating the SCA procedure are also provided.

Longitudinal quantile regression in the presence of informative dropout through longitudinal-survival joint modeling.

We propose a joint model for a time-to-event outcome and a quantile of a continuous response repeatedly measured over time. The quantile and survival processes are associated via shared latent and manifest variables. Our joint model provides a flexible approach to handle informative dropout in quantile regression. A Monte Carlo expectation maximization strategy based on importance sampling is proposed, which is directly applicable under any distributional assumption for the longitudinal outcome and random effects. We consider both parametric and nonparametric assumptions for the baseline hazard. We illustrate through a simulation study and an application to an original data set about dilated cardiomyopathies.

A pseudo-likelihood approach for estimating diagnostic accuracy of multiple binary medical tests

Latent class models with crossed subject-specific and test(rater)-specific random effects have been proposed to estimate the diagnostic accuracy (sensitivity and specificity) of a group of binary tests or binary ratings. However, the computation of these models are hindered by their complicated Monte Carlo Expectation–Maximization (MCEM) algorithm. In this article, a class of pseudo-likelihood functions is developed for conducting statistical inference with crossed random-effects latent class models in diagnostic medicine. Theoretically, the maximum pseudo-likelihood estimation is still consistent and has asymptotic normality. Numerically, our results show that not only the pseudo-likelihood approach significantly reduces the computational time, but it has comparable efficiency relative to the MCEM algorithm. In addition, dimension-wise likelihood, one of the proposed pseudo-likelihoods, demonstrates its superior performance in estimating sensitivity and specificity.

An Automated MCEM Algorithm for Hierarchical Models with Multivariate and Multitype Response Variables

In this article, we consider a model allowing the analysis of multivariate data, which can contain data attributes of different types (e.g., continuous, discrete, binary). This model is a two-level hierarchical model which supports a wide range of correlation structures and can accommodate overdispersed data. Maximum likelihood estimation of the model parameters is achieved with an automated Monte Carlo expectation maximization algorithm. Our method is tested in a simulation study in the bivariate case and applied to a data set dealing with beehive activity.

Performance tests of a full scale prototype of the Belle II TOP counter with cosmic muons and 2.1 GeV/c positron beam

A time-of-propagation detector named TOP is a hadronic particle identification system for the Belle II experiment. We have produced a full scale prototype of the Belle II TOP counter and tested with cosmic muons at KEK and the 2.1GeV/c positrons at SPring-8 LEPS beamline. The procedures for the quartz acceptance test and assembly worked well and the first quartz radiator was successfully fabricated. The obtained test data shows good agreement with Monte Carlo expectation.

A functional network estimation method of resting-state fMRI using a hierarchical Markov random field

We propose a hierarchical Markov random field model for estimating both group and subject functional networks simultaneously. The model takes into account the within-subject spatial coherence as well as the between-subject consistency of the network label maps. The statistical dependency between group and subject networks acts as a regularization, which helps the network estimation on both layers. We use Gibbs sampling to approximate the posterior density of the network labels and Monte Carlo expectation maximization to estimate the model parameters. We compare our method with two alternative segmentation methods based on K-Means and normalized cuts, using synthetic and real fMRI data. The experimental results show that our proposed model is able to identify both group and subject functional networks with higher accuracy on synthetic data, more robustness, and inter-session consistency on the real data.

Robust inference for generalized partially linear mixed models that account for censored responses and missing covariates – an application to Arctic data analysis

In this article, we propose a family of bounded influence robust estimates for the parametric and non-parametric components of a generalized partially linear mixed model that are subject to censored responses and missing covariates. The asymptotic properties of the proposed estimates have been looked into. The estimates are obtained by using Monte Carlo expectation–maximization algorithm. An approximate method which reduces the computational time to a great extent is also proposed. A simulation study shows that performances of the two approaches are similar in terms of bias and mean square error. The analysis is illustrated through a study on the effect of environmental factors on the phytoplankton cell count.

Regression for skewed biomarker outcomes subject to pooling

Epidemiological studies involving biomarkers are often hindered by prohibitively expensive laboratory tests. Strategically pooling specimens prior to performing these lab assays has been shown to effectively reduce cost with minimal information loss in a logistic regression setting. When the goal is to perform regression with a continuous biomarker as the outcome, regression analysis of pooled specimens may not be straightforward, particularly if the outcome is right-skewed. In such cases, we demonstrate that a slight modification of a standard multiple linear regression model for poolwise data can provide valid and precise coefficient estimates when pools are formed by combining biospecimens from subjects with identical covariate values. When these x-homogeneous pools cannot be formed, we propose a Monte Carlo expectation maximization (MCEM) algorithm to compute maximum likelihood estimates (MLEs). Simulation studies demonstrate that these analytical methods provide essentially unbiased estimates of coefficient parameters as well as their standard errors when appropriate assumptions are met. Furthermore, we show how one can utilize the fully observed covariate data to inform the pooling strategy, yielding a high level of statistical efficiency at a fraction of the total lab cost.

Measurement and verification of building systems under uncertain data: A Gaussian process modeling approach

Uncertainty in sensor data (e.g., weather, occupancy) complicates the construction of baseline models for measurement and verification (M&V). We present a Monte Carlo expectation maximization (MCEM) framework for constructing baseline Gaussian process (GP) models under uncertain input data. We demonstrate that the GP-MCEM framework yields more robust predictions and confidence levels compared with standard GP training approaches that neglect uncertainty. We argue that the approach can also reduce data needs because it implicitly expands the data range used for training and can thus be used as a mechanism to reduce data collection and sensor installation costs in M&V processes. We analyze the numerical behavior of the framework and conclude that robust predictions can be obtained with relatively few samples.

Employing a Monte Carlo Algorithm in Newton-Type Methods for Restricted Maximum Likelihood Estimation of Genetic Parameters

Estimation of variance components by Monte Carlo (MC) expectation maximization (EM) restricted maximum likelihood (REML) is computationally efficient for large data sets and complex linear mixed effects models. However, efficiency may be lost due to the need for a large number of iterations of the EM algorithm. To decrease the computing time we explored the use of faster converging Newton-type algorithms within MC REML implementations. The implemented algorithms were: MC Newton-Raphson (NR), where the information matrix was generated via sampling; MC average information(AI), where the information was computed as an average of observed and expected information; and MC Broyden's method, where the zero of the gradient was searched using a quasi-Newton-type algorithm. Performance of these algorithms was evaluated using simulated data. The final estimates were in good agreement with corresponding analytical ones. MC NR REML and MC AI REML enhanced convergence compared to MC EM REML and gave standard errors for the estimates as a by-product. MC NR REML required a larger number of MC samples, while each MC AI REML iteration demanded extra solving of mixed model equations by the number of parameters to be estimated. MC Broyden's method required the largest number of MC samples with our small data and did not give standard errors for the parameters directly. We studied the performance of three different convergence criteria for the MC AI REML algorithm. Our results indicate the importance of defining a suitable convergence criterion and critical value in order to obtain an efficient Newton-type method utilizing a MC algorithm. Overall, use of a MC algorithm with Newton-type methods proved feasible and the results encourage testing of these methods with different kinds of large-scale problem settings.

A fast Monte Carlo expectation–maximization algorithm for estimation in latent class model analysis with an application to assess diagnostic accuracy for cervical neoplasia in women with atypical glandular cells

In this article, we use a latent class model (LCM) with prevalence modeled as a function of covariates to assess diagnostic test accuracy in situations where the true disease status is not observed, but observations on three or more conditionally independent diagnostic tests are available. A fast Monte Carlo expectation–maximization (MCEM) algorithm with binary (disease) diagnostic data is implemented to estimate parameters of interest; namely, sensitivity, specificity, and prevalence of the disease as a function of covariates. To obtain standard errors for confidence interval construction of estimated parameters, the missing information principle is applied to adjust information matrix estimates. We compare the adjusted information matrix-based standard error estimates with the bootstrap standard error estimates both obtained using the fast MCEM algorithm through an extensive Monte Carlo study. Simulation demonstrates that the adjusted information matrix approach estimates the standard error similarly with the bootstrap methods under certain scenarios. The bootstrap percentile intervals have satisfactory coverage probabilities. We then apply the LCM analysis to a real data set of 122 subjects from a Gynecologic Oncology Group study of significant cervical lesion diagnosis in women with atypical glandular cells of undetermined significance to compare the diagnostic accuracy of a histology-based evaluation, a carbonic anhydrase-IX biomarker-based test and a human papillomavirus DNA test.

Tuned iterated filtering

Iterated filtering is an algorithm for estimating parameters in partially observed Markov process (POMP) models. The real-world performance of the algorithm depends on several tuning parameters. We propose a simple method for optimizing the parameter governing the joint dynamics of the hidden parameter process (called the Σ matrix).The tuning is implemented using a fixed-lag sequential Monte Carlo expectation–maximization (EM) algorithm. We introduce two different versions of the tuning parameter, the approximately estimated Σ matrix, and a normalized version of the same matrix.Our simulations show that the finite-sample performance for the normalized matrix outperform the standard iterated filter, while the naive version is doing more harm than good.

An efficient Monte Carlo EM algorithm for Bayesian lasso

The lasso is a popular technique of simultaneous estimation and variable selection in many research areas. The marginal posterior mode of the regression coefficients is equivalent to estimates given by the non-Bayesian lasso when the regression coefficients have independent Laplace priors. Because of its flexibility of statistical inferences, the Bayesian approach is attracting a growing body of research in recent years. Current approaches are primarily to either do a fully Bayesian analysis using Markov chain Monte Carlo (MCMC) algorithm or use Monte Carlo expectation maximization (MCEM) methods with an MCMC algorithm in each E-step. However, MCMC-based Bayesian method has much computational burden and slow convergence. Tan et al. [An efficient MCEM algorithm for fitting generalized linear mixed models for correlated binary data. J Stat Comput Simul. 2007;77:929–943] proposed a non-iterative sampling approach, the inverse Bayes formula (IBF) sampler, for computing posteriors of a hierarchical model in the structure of MCEM. Motivated by their paper, we develop this IBF sampler in the structure of MCEM to give the marginal posterior mode of the regression coefficients for the Bayesian lasso, by adjusting the weights of importance sampling, when the full conditional distribution is not explicit. Simulation experiments show that the computational time is much reduced with our method based on the expectation maximization algorithm and our algorithms and our methods behave comparably with other Bayesian lasso methods not only in prediction accuracy but also in variable selection accuracy and even better especially when the sample size is relatively large.

Maximum likelihood estimation of the Markov-switching GARCH model

The Markov-switching GARCH model offers rich dynamics to model financial data. Estimating this path dependent model is a challenging task because exact computation of the likelihood is infeasible in practice. This difficulty led to estimation procedures either based on a simplification of the model or not dependent on the likelihood. There is no method available to obtain the maximum likelihood estimator without resorting to a modification of the model. A novel approach is developed based on both the Monte Carlo expectation–maximization algorithm and importance sampling to calculate the maximum likelihood estimator and asymptotic variance–covariance matrix of the Markov-switching GARCH model. Practical implementation of the proposed algorithm is discussed and its effectiveness is demonstrated in simulation and empirical studies.

A Dynamic State-Space Model of Coded Political Texts

This article presents a new method of reconstructing actors' political positions from coded political texts. It is based on a model that combines a dynamic perspective on actors' political positions with a probabilistic account of how these positions are translated into emphases of policy topics in political texts. In the article it is shown how model parameters can be estimated based on a maximum marginal likelihood principle and how political actors' positions can be reconstructed using empirical Bayes techniques. For this purpose, a Monte Carlo Expectation Maximization algorithm is used that employs independent sample techniques with automatic Monte Carlo sample size adjustment. An example application is given by estimating a model of an economic policy space and a noneconomic policy space based on the data from the Comparative Manifesto Project. Parties' positions in policy spaces reconstructed using these models are made publicly available for download.

Ordinal latent variable models and their application in the study of newly licensed teenage drivers.

In a unique longitudinal study of teen driving, risky driving behavior and the occurrence of crashes or near crashes are measured prospectively over the first 18 months of licensure. Of scientific interest is relating the two processes and developing a predictor of crashes from previous risky driving behavior. In this work, we propose two latent class models for relating risky driving behavior to the occurrence of a crash or near crash event. The first approach models the binary longitudinal crash/near crash outcome using a binary latent variable which depends on risky driving covariates and previous outcomes. A random effects model introduces heterogeneity among subjects in modeling the mean value of the latent state. The second approach extends the first model to the ordinal case where the latent state is composed of K ordinal classes. Additionally, we discuss an alternate hidden Markov model formulation. Estimation is performed using the expectation-maximization (EM) algorithm and Monte Carlo EM. We illustrate the importance of using these latent class modeling approaches through the analysis of the teen driving behavior.

Maximum likelihood estimation of multinomial probit factor analysis models for multivariate t-distribution

We propose a model for multinomial probit factor analysis by assuming t-distribution error in probit factor analysis. To obtain maximum likelihood estimation, we use the Monte Carlo expectation maximization algorithm with its M-step greatly simplified under conditional maximization and its E-step made feasible by Monte Carlo simulation. Standard errors are calculated by using Louis's method. The methodology is illustrated with numerical simulations.

Estimation and inference on correlations between biomarkers with repeated measures and left‐censoring due to minimum detection levels

Statistical approaches for estimating and drawing inference on the correlation between two biomarkers that are repeatedly assessed over time and subject to left-censoring because minimum detection levels are lacking. We propose a linear mixed-effects model and estimate the parameters with the Monte Carlo expectation maximization (MCEM) method. Inferences regarding the model parameters and the correlation between the biomarkers are performed by applying Louis's method and the delta method. Simulation studies were conducted to compare the proposed MCEM method with existing methods including the maximum likelihood estimation method, the multiple imputation method, and two widely used ad hoc approaches: replacing the censored values with the detection limit or with half of the detection limit. The results show that the performance of the MCEM with respect to relative bias and coverage probability for the 95% confidence interval is superior to the detection limit and half of the detection limit approaches and exceeds that of the multiple imputation method at medium to high levels of censoring, and the standard error estimates from the MCEM method are close to ideal. The maximum likelihood estimation method can estimate the parameters accurately; however, a nonpositive definite information matrix can occur so that the variances are not estimable. These five methods are illustrated with data from a longitudinal human immunodeficiency virus study to estimate and draw inference on the correlation between human immunodeficiency virus RNA levels measured in plasma and in cervical secretions at multiple time points.