Estimation of Denominator Degrees of Freedom of F-Distributions for Assessing Wald Statistics for Fixed-Effect Factors in Unbalanced Mixed Models

Abstract

SUMMARY Tests for fixed-effect factors in unbalanced mixed models have previously used t-tests on a contrastby-contrast basis or Wald statistics without a universally accepted method of calculating the denominator degrees of freedom. This situation has arisen because the variances of different contrasts are differently weighted sums of the variance components with associated degrees of freedom that are not necessarily equal. A simultaneous F-test for differences between all levels of a fixed-effect factor can be derived by forming new contrasts, by rotation of the original contrasts, with variances that are close to being the same weighted sum of variance components. The associated degrees of freedom for these new contrasts are nearly equal. A small simulation study shows the appropriateness of a X2 approximation to the distribution of the weighted sums of variance components. Three simple examples are used to demonstrate the effects of rotation. The last of these examples is also used to compare the proposed simultaneous F-test with the distribution of the Wald statistic obtained by numerical simulation. The method of rotations is then applied to data on the range size of mountain hares (Lepus timidus) to assess the evidence for a two-way interaction between season and habitat. Analysis of data from balanced experiments is traditionally performed by an analysis of variance in which the total sum of squared deviations of observations about their mean is partitioned into sums of squares attributable to either the different treatment effects or the different random effects. Under standard distributional assumptions, each random effect sum of squares follows a multiple of a X2-distribution, this multiple being the product of the degrees of freedom (the number of independent error contrasts whose square has the required expectation) and a linear combination of the different variance components, the coefficients in this linear combination being fixed by the design. Under any null hypothesis concerning the absence of treatment effects, the corresponding treatment sum of squares also follows such a multiple of a X2-distribution. Furthermore, the linear combination of variance components in the multiple for the treatment sum of squares is identical to the linear combination for one of the random-effect sum of squares; hence, hypotheses about the treatment effects can be tested by dividing treatment mean squares by the appropriate error mean square to form a variance ratio with an F-distribution. The benefits of experimental design accrue, first, through the variance of all contrasts for any given treatment factor or interaction having the same expectation under the null hypothesis; second, through this variance being as small as possible; and, third, through there being as many error contrasts as possible whose variance is the same as that for the treatment factor or interaction. Analysis of data that lacks these properties of balance is much less straightforward, yet there are many areas in which, due to the nature of experimental material, the treatment effects cannot be applied in a balanced fashion, and so such unbalanced data are the norm rather than the