The influence of violations of assumptions on multilevel parameter estimates and their standard errors

Abstract

A crucial problem in the statistical analysis of hierarchically structured data is the dependence of the observations at the lower levels. Multilevel modeling programs account for this dependence and in recent years these programs have been widely accepted. One of the major assumptions of the tests of significance used in the multilevel programs is normality of the error distributions involved. Simulations were used to assess how important this assumption is for the accuracy of multilevel parameter estimates and their standard errors. Simulations varied the number of groups, the group size, and the intraclass correlation, with the second level residual errors following one of three non-normal distributions. In addition asymptotic maximum likelihood standard errors are compared to robust (Huber/White) standard errors. The results show that non-normal residuals at the second level of the model have little or no effect on the parameter estimates. For the fixed parameters, both the maximum likelihood-based standard errors and the robust standard errors are accurate. For the parameters in the random part of the model, the maximum likelihood-based standard errors at the lowest level are accurate, while the robust standard errors are often overcorrected. The standard errors of the variances of the level-two random effects are highly inaccurate, although the robust errors do perform better than the maximum likelihood errors. For good accuracy, robust standard errors need at least 100 groups. Thus, using robust standard errors as a diagnostic tool seems to be preferable to simply relying on them to solve the problem.