Comparison of Software Algorithms for Calculating REML Wald Type Confidence Limits for the Between-Group Variance Component in a Small Sample One-Way Random Effects Model Example

Abstract

Confidence limits for variance components in mixed effects models can be readily estimated by procedures such as PROC MIXED (SAS®), MIXED (SPSS®), LME (S-PLUS®), and XTMIXED (Stata®). For a small sample unbalanced one-way random effects model case study, the REML Wald type confidence limits for the between-group variance obtained from PROC MIXED differ substantially from those from MIXED, LME, and XTMIXED. Simulations indicate that the Satterthwaite approximation used by PROC MIXED results in highly inflated confidence interval lengths, while the interval lengths from the other procedures are usually similar but also biased upward. The coverage rates of the confidence limits from all four procedures appear above the nominal value. Extreme upper limits and problems with computing the confidence limits are also possible, especially with PROC MIXED. In general, one should avoid the naive use of statistical software packages when estimating REML confidence limits for variance components with small datasets, and simulations can assist in examining the validity of the results.