Fitting Linear Mixed-Effects Models Using lme4

In clinical data analysis, the restricted maximum likelihood (REML) method has been commonly used for estimating variance components in the linear mixed effects model. Under the REML estimation, however, it is not straightforward to compare several linear mixed effects models with different mean and covariance structures. In particular, few approaches have been proposed for the comparison of linear mixed effects models with different mean structures under the REML estimation. We propose an approach using extended information criterion (EIC), which is a bootstrap-based extension of AIC, for comparing linear mixed effects models with different mean and covariance structures under the REML estimation. We present simulation studies and applications to two actual clinical data sets.

On the effect of ignoring correlation in the covariates when fitting linear mixed models

For a balanced two‐way mixed model, the maximum likelihood (ML) and restricted ML (REML) estimators of the variance components were obtained and compared under the non‐negativity requirements of the variance components by Lee and Kapadia (1984). In this note, for a mixed (random blocks) incomplete block model, explicit forms for the REML estimators of variance components are obtained. They are always non‐negative and have smaller mean squared error (MSE) than the analysis of variance (AOV) estimators. The asymptotic sampling variances of the maximum likelihood (ML) estimators and the REML estimators are compared and the balanced incomplete block design (BIBD) is considered as a special case. The ML estimators are shown to have smaller asymptotic variances than the REML estimators, but a numerical result in the randomized complete block design (RCBD) demonstrated that the performances of the REML and ML estimators are not much different in the MSE sense.

This chapter expands on linear models through the introduction of random effects, where the independent variables in the model include those variables that are fixed and those that vary across subjects. The general linear mixed effects model (LMEM) is introduced as methods for parameter estimation – maximum likelihood and restricted maximum likelihood. Model selection and goodness of fit in the context of LMEMs are discussed. The concept of empirical Bayes estimates and how shrinkage affects these estimates are also discussed. Building upon LMEMs, partial LMEMs are introduced, which use penalized spline regression to obtain a nonparametric-type smooth to the data, and how these might be used for covariate selection when knowing the exact nature of the effect of time is not needed. Three examples of LMEMs are provided: results from a food effect study, modeling tumor growth in a mouse xenograft model, and a detailed analysis of QT prolongation in clinical studies.

Assumptions in the classical linear regression model include that of lack of autocorrelation of the error terms and the zero covariance between the explanatory variable and the error terms. This study is channeled towards the estimation of the parameters of the linear models for both time series and cross-sectional data when the above two assumptions are violated. The study used the Monte-Carlo simulation method to investigate the performance of six estimators: ordinary least square (OLS), Prais-Winsten (PW), Cochrane-Orcutt (CC), Maximum Likelihood (MLE), Restricted Maximum- Likelihood (RMLE) and the Weighted Least Square (WLS) in estimating the parameters of a single linear model in which the explanatory variable is also correlated with the autoregressive error terms. Using the models’ finite properties(mean square error) to measure the estimators’ performance, the results shows that OLS should be preferred when autocorrelation level is relatively mild (ρ = 0.3) and the PW, CC, RMLE, and MLE estimator will perform better with the presence of any level of AR (1) disturbance between 0.4 to 0.8 level, while WLS shows better performance at 0.9 level of autocorrelation and above. The study thus recommended the application of the various estimators considered to real-life data to affirm the results of this simulation study.

This work describes a Gaussian Markov random field model that includes several previously proposed models, and studies properties of its maximum likelihood (ML) and restricted maximum likelihood (REML) estimators in a special case. Specifically, for models where a particular relation holds between the regression and precision matrices of the model, we provide sufficient conditions for existence and uniqueness of ML and REML estimators of the covariance parameters, and provide a straightforward way to compute them. It is found that the ML estimator always exists while the REML estimator may not exist with positive probability. A numerical comparison suggests that for this model ML estimators of covariance parameters have, overall, better frequentist properties than REML estimators.

Longitudinal data are measurements or observations taken from multiple subjects repeatedly over time. The main theme of this book is to describe autoregressive linear mixed effects models for longitudinal data analysis. This model is an extension of linear mixed effects models and autoregressive models. This chapter introduces longitudinal data and linear mixed effects models before the main theme in the following chapters. Linear mixed effects models are popularly used for the analysis of longitudinal data of a continuous response variable. They are an extension of linear models by including random effects and variance covariance structures for random errors. Marginal models, which do not include random effects, are also introduced in the same framework. This chapter explains examples of popular linear mixed effects models and marginal models: means at each time point with a random intercept, means at each time point with an unstructured variance covariance, and linear time trend models with a random intercept and a random slope. The corresponding examples of group comparisons are also provided. This chapter also discusses the details of mean structures and variance covariance structures and provides estimation methods based on maximum likelihood and restricted maximum likelihood.

A one-stage individual participant data (IPD) meta-analysis synthesizes IPD from multiple studies using a general or generalized linear mixed model. This produces summary results (eg, about treatment effect) in a single step, whilst accounting for clustering of participants within studies (via a stratified study intercept, or random study intercepts) and between-study heterogeneity (via random treatment effects). We use simulation to evaluate the performance of restricted maximum likelihood (REML) and maximum likelihood (ML) estimation of one-stage IPD meta-analysis models for synthesizing randomized trials with continuous or binary outcomes. Three key findings are identified. First, for ML or REML estimation of stratified intercept or random intercepts models, a t-distribution based approach generally improves coverage of confidence intervals for the summary treatment effect, compared with a z-based approach. Second, when using ML estimation of a one-stage model with a stratified intercept, the treatment variable should be coded using "study-specific centering" (ie, 1/0 minus the study-specific proportion of participants in the treatment group), as this reduces the bias in the between-study variance estimate (compared with 1/0 and other coding options). Third, REML estimation reduces downward bias in between-study variance estimates compared with ML estimation, and does not depend on the treatment variable coding; for binary outcomes, this requires REML estimation of the pseudo-likelihood, although this may not be stable in some situations (eg, when data are sparse). Two applied examples are used to illustrate the findings.

Genetic correlations (rho ( g )) are frequently estimated from natural and experimental populations, yet many of the statistical properties of estimators of rho ( g ) are not known, and accurate methods have not been described for estimating the precision of estimates of rho ( g ). Our objective was to assess the statistical properties of multivariate analysis of variance (MANOVA), restricted maximum likelihood (REML), and maximum likelihood (ML) estimators of rho ( g ) by simulating bivariate normal samples for the one-way balanced linear model. We estimated probabilities of non-positive definite MANOVA estimates of genetic variance-covariance matrices and biases and variances of MANOVA, REML, and ML estimators of rho ( g ), and assessed the accuracy of parametric, jackknife, and bootstrap variance and confidence interval estimators for rho ( g ). MANOVA estimates of rho ( g ) were normally distributed. REML and ML estimates were normally distributed for rho ( g ) = 0.1, but skewed for rho ( g ) = 0.5 and 0.9. All of the estimators were biased. The MANOVA estimator was less biased than REML and ML estimators when heritability (H), the number of genotypes (n), and the number of replications (r) were low. The biases were otherwise nearly equal for different estimators and could not be reduced by jackknifing or bootstrapping. The variance of the MANOVA estimator was greater than the variance of the REML or ML estimator for most H, n, and r. Bootstrapping produced estimates of the variance of rho ( g ) close to the known variance, especially for REML and ML. The observed coverages of the REML and ML bootstrap interval estimators were consistently close to stated coverages, whereas the observed coverage of the MANOVA bootstrap interval estimator was unsatisfactory for some H, rho ( g ), n, and r. The other interval estimators produced unsatisfactory coverages. REML and ML bootstrap interval estimates were narrower than MANOVA bootstrap interval estimates for most H, rho ( g ), n, and r.

Structural equation modeling (SEM) is now a generic modeling framework for many multivariate techniques applied in the social and behavioral sciences. Many statistical models can be considered either as special cases of SEM or as part of the latent variable modeling framework. One popular extension is the use of SEM to conduct linear mixed-effects modeling (LMM) such as cross-sectional multilevel modeling and latent growth modeling. It is well known that LMM can be formulated as structural equation models. However, one main difference between the implementations in SEM and LMM is that maximum likelihood (ML) estimation is usually used in SEM, whereas restricted (or residual) maximum likelihood (REML) estimation is the default method in most LMM packages. This article shows how REML estimation can be implemented in SEM. Two empirical examples on latent growth model and meta-analysis are used to illustrate the procedures implemented in OpenMx. Issues related to implementing REML in SEM are discussed.

Serum Protein Abundance in Children with Sickle Cell Disease at Baseline, during Acute Pain Crisis, and on Hydroxyurea - Compared to Children with Other Pediatric Diseases

Chapter 4 - Restricted maximum likelihood and inference of random effects in linear mixed models

SUMMARY The maximum likelihood estimator of the variance components in a linear model can be biased downwards. Restricted maximum likelihood (REML) corrects this problem by using the likelihood of a set of residual contrasts and is generally considered superior. However, this original restricted maximum likelihood definition does not directly extend beyond linear models. We propose a REML-type estimator for generalised linear mixed models by correcting the bias in the profile score function of the variance components. The proposed estimator has the same consistency properties as the maximum likelihood estimator if the number of parameters in the mean and variance components models remains fixed. However, the estimator of the variance components has a smaller finite sample bias. A simulation study with a logistic mixed model shows that the proposed estimator is effective in correcting the downward bias in the maximum likelihood estimator.

Linear mixed effects (LME) models are important statistical tools for analysis of clustered and correlated data. High breakdown estimators are currently the robust methods of choice for multivariate linear regression, but extensions of such estimators have been developed only for completely balanced LME models. In this work, we propose a generalized S-estimator for a general unbalanced LME model. Our GS-estimator reduces to the classic high breakdown S-estimator when the LME model reduces to a multivariate normal location and scale model or mul- tivariate regression model. The asymptotic properties are established, and we show that the estimator may be viewed as a redescending M-estimator. A small simu- lation study is conducted to compare performance of the GS-estimates, monotone M-estimates, and restricted maximum likelihood (REML) estimates under various contamination patterns. The proposed estimator is used for analysis of age-related changes in hemoglobin levels of sickle cell disease patients.

Approximate ML and REML estimation for regression models with spatial or time series AR(1) noise