Effect Size Estimation in Linear Mixed Models

The independent variables of linear mixed models are subject to measurement errors in practice. In this paper, we present a unified method for the estimation in linear mixed models with errors-in-variables, based upon the corrected score function of Nakamura (1990, Biometrika, 77, 127–137). Asymptotic normality properties of the estimators are obtained. The estimators are shown to be consistent and convergent at the order of n−1/2. The performance of the proposed method is studied via simulation and the analysis of a data set on hedonic housing prices.

In the past decade, mixed-effects modeling has received a great deal of attention in the applied and theoretical statistical literature. They are very flexible tools in analyzing repeated measures, panel data, cross-sectional data, and hierarchical data. However, the complex nature of these models has motivated researchers to study different aspects of this problem. One of which is to test the significance of random effects used to model unobserved heterogeneity in the population. The method of likelihood ratio test based on the normality assumption of the error term and random effects has been proposed. However, this assumption does not necessarily hold in practice. In this paper, we propose an optimal test based on the so-called uniform local asymptotic normality to detect the possible presence of random effects in linear mixed models. We show that the proposed test statistic is, consistent, locally asymptotically optimal even for a model that does not require the traditional assumption of normality and is comparable to the classical L.ratio-test when the standard assumptions are met. Finally, simulation studies and real data analysis are also conducted to empirically examine the performance of this procedure.

Today, Linear Mixed Models (LMMs) are fitted, mostly, by assuming that random effects and errors have Gaussian distributions, therefore using Maximum Likelihood (ML) or REML estimation. However, for many data sets, that double assumption is unlikely to hold, particularly for the random effects, a crucial component in which assessment of magnitude is key in such modeling. Alternative fitting methods not relying on that assumption (as ANOVA ones and Rao’s MINQUE) apply, quite often, only to the very constrained class of variance components models. In this paper, a new computationally feasible estimation methodology is designed, first for the widely used class of 2-level (or longitudinal) LMMs with only assumption (beyond the usual basic ones) that residual errors are uncorrelated and homoscedastic, with no distributional assumption imposed on the random effects. A major asset of this new approach is that it yields nonnegative variance estimates and covariance matrices estimates which are symmetric and, at least, positive semi-definite. Furthermore, it is shown that when the LMM is, indeed, Gaussian, this new methodology differs from ML just through a slight variation in the denominator of the residual variance estimate. The new methodology actually generalizes to LMMs a well known nonparametric fitting procedure for standard Linear Models. Finally, the methodology is also extended to ANOVA LMMs, generalizing an old method by Henderson for ML estimation in such models under normality.

Unified standard for fetal growth velocity: the Eunice Kennedy Shriver National Institute of Child Health and Human Development Fetal Growth Studies

Restricted Maximum Likelihood (REML) is the most recommended approach for fitting a Linear Mixed Model (LMM) nowadays. Yet, as ML, REML suffers the drawback that it performs such a fitting by assuming normality for both the random effects and the residual errors, a dubious assumption for many real data sets. Now, there have been several attempts at trying to justify the use of the REML likelihood equations outside of the Gaussian world, with varying degrees of success. Recently, a new fitting methodology, code named 3S, was presented for LMMs with only added assumption (to the basic ones) that the residual errors are uncorrelated and homoscedastic. Specifically, the 3S-A1 variant was designed and then shown, for Gaussian LMMs, to differ only slightly from ML estimation. In this article, using the same 3S framework, we develop another iterative nonparametric estimation methodology, code named 3S-A1.RE, for the kind of LMMs just mentioned. However, we show that if the LMM is, indeed, Gaussian with i.i.d. residual errors, then the set of estimating equations defining any 3S-A1.RE iterative procedure is equivalent to the set of REML equations, but while including the nonnegativity constraints on all variance estimates, as well as positive semi-definiteness on all covariance matrices. In numerical tests on some simulated and real world clustered and longitudinal data sets, our new methods proved to be highly competitive when compared to the traditional REML in the R statistical software.

This paper proposes a dimension reduction technique for estimation in linear mixed models. Specifically, we show that in a linear mixed model, the maximum-likelihood (ML) problem can be rewritten as a substantially simpler optimization problem which presents at least two main advantages: the number of variables in the simplified problem is lower and the search domain of the simplified problem is a compact set. Whereas the former advantage reduces the computational burden, the latter permits the use of stochastic optimization methods well qualified for closed bounded domains. The developed dimension reduction technique makes the computation of ML estimates, for fixed effects and variance components, feasible with large computational savings. Computational experience is reported here with the results evidencing an overall good performance of the proposed technique.

Variance parameter estimation in linear mixed models is a challenge for many classical nonlinear optimization algorithms due to the positive-definiteness constraint of the random effects covariance matrix. Many existing algorithms may get stuck on the boundary of the feasible space. In this paper, we address this problem by exploiting the intrinsic geometry of the parameter space, pulling iterates away from the boundary. In this novel approach, we formulate the problem of residual maximum likelihood estimation as an optimization problem on a Riemannian manifold. Based on the introduced formulation, we give geometric higher-order information on the problem via the Riemannian gradient and the Riemannian Hessian, making the algorithm more robust against singularities. We test our approach with Riemannian optimization algorithms numerically. Our approach yields a much better quality of the variance parameter estimates compared to existing approaches.

This article develops asymptotic theory for estimation of parameters in regression models for binomial response time series where serial dependence is present through a latent process. Use of generalized linear model estimating equations leads to asymptotically biased estimates of regression coefficients for binomial responses. An alternative is to use marginal likelihood, in which the variance of the latent process but not the serial dependence is accounted for. In practice, this is equivalent to using generalized linear mixed model estimation procedures treating the observations as independent with a random effect on the intercept term in the regression model. We prove that this method leads to consistent and asymptotically normal estimates even if there is an autocorrelated latent process. Simulations suggest that the use of marginal likelihood can lead to generalized linear model estimates result. This problem reduces rapidly with increasing number of binomial trials at each time point, but for binary data, the chance of it can remain over 45% even in very long time series. We provide a combination of theoretical and heuristic explanations for this phenomenon in terms of the properties of the regression component of the model, and these can be used to guide application of the method in practice.

It is quite appealing to extend existing theories in classical linear models to correlated responses where linear mixed-effects models are utilized and the dependency in the data is modeled by random effects. In the mixed modeling framework, missing values occur naturally due to dropouts or non-responses, which is frequently encountered when dealing with real data. Motivated by such problems, we aim to investigate the estimation and model selection performance in linear mixed models when missing data are present. Inspired by the property of the indicator function for missingness and its relation to missing rates, we propose an approach that records missingness in an indicator-based matrix and derive the likelihood-based estimators for all parameters involved in the linear mixed-effects models. Based on the proposed method for estimation, we explore the relationship between estimation and selection behavior over missing rates. Simulations and a real data application are conducted for illustrating the effectiveness of the proposed method in selecting the most appropriate model and in estimating parameters.

Linear mixed-effects models have increasingly replaced mixed-model analyses of variance for statistical inference in factorial psycholinguistic experiments. Although LMMs have many advantages over ANOVA, like ANOVAs, setting them up for data analysis also requires some care. One simple option, when numerically possible, is to fit the full variance-covariance structure of random effects (the maximal model; Barr et al. 2013), presumably to keep Type I error down to the nominal alpha in the presence of random effects. Although it is true that fitting a model with only random intercepts may lead to higher Type I error, fitting a maximal model also has a cost: it can lead to a significant loss of power. We demonstrate this with simulations and suggest that for typical psychological and psycholinguistic data, higher power is achieved without inflating Type I error rate if a model selection criterion is used to select a random effect structure that is supported by the data.

Three-step estimation in linear mixed models with skew- t distributions

Estimation of linear mixed models with a mixture of distribution for the random effects

ABSTRACTThis paper is concerned with the ridge estimation of fixed and random effects in the context of Henderson's mixed model equations in the linear mixed model. For this purpose, a penalized likelihood method is proposed. A linear combination of ridge estimator for fixed and random effects is compared to a linear combination of best linear unbiased estimator for fixed and random effects under the mean-square error (MSE) matrix criterion. Additionally, for choosing the biasing parameter, a method of MSE under the ridge estimator is given. A real data analysis is provided to illustrate the theoretical results and a simulation study is conducted to characterize the performance of ridge and best linear unbiased estimators approach in the linear mixed model.

Linear mixed models (LMMs) are modeled using both fixed effects and random effects for correlated data. The random intercepts (RI) and random intercepts and slopes (RIS) models are two exceptional cases from linear mixed models. Our primary focus is to propose an approach for simultaneous selection and estimation in linear mixed models. We design a penalized log-likelihood procedure referred to as the minimum approximated information criteria for LMMs (lmmMAIC), which is utilized to select the most appropriate model for generalizing the data. The proposed lmmMAIC procedure for variable selection and estimation can estimate the parameters while shrinking the unimportant fixed effects estimates to zero and is mainly motivated by both regularized methods and information criteria. This procedure enforces variable selection and sparse estimation simultaneously by adding a penalty term to the negative log-likelihood of linear mixed models. The method differs from existing regularized methods mainly due to the penalty parameter and the penalty function which is approximated L0 norm with unit dent. A simulation study is performed to demonstrate the effectiveness of the lmmMAIC method, and the simulation results show that the lmmMAIC method outperforms some other penalized methods in estimation and model selection by the comparison of the simulation results. The proposed method is also applied in Riboflavin data for the illustration of its behavior.

Small area estimation ‒ model-based approach in economic research