THE EVALUATION OF BIAS OF THE WEIGHTED RANDOM EFFECTS MODEL ESTIMATORS

Abstract

ABSTRACTEstimation of parameters of random effects models from samples collected via complex multistage designs is considered. One way to reduce estimation bias due to unequal probabilities of selection is to incorporate sampling weights. Many researchers have been proposed various weighting methods (Korn, & Graubard, 2003; Pfeffermann, Skinner, Holmes, Goldstein, & Rasbash, 1998) in estimating the parameters of hierarchical models, including random effects models as a special case. In this paper, the bias of the weighted analysis of variance (ANOVA) estimators of the variance components for a two‐level, one‐way random effects model is evaluated. For these estimators, analytic bias expressions are first developed, the expressions are then used to examine the impact of sample size, intraclass correlation coefficient (ICC), and the sampling design on the bias of the estimators. In addition, two‐stage sampling designs are considered, with a general probability design at the first stage (Level 2) and simple random sampling without replacement (SRS) at the second stage (Level 1). The study shows that firstorder weighted variance component estimators perform well when for moderate cluster sizes and ICC values. However, noticeable estimation bias can be found with this weighting method for small cluster sizes (less than 20), particularly when ICC is small (less than 0.2). In such scenarios, scaled first‐order weighted estimators can be an alternative. This paper is discussed in the context of National Assessment of Educational Progress (NAEP) 2003 4th Grade Reading National and State Assessment data, with Level 1 being the student level and Level 2 being the school level.