An Empirical Comparison of Five Linear Equating Methods for the NEAT Design

Abstract

In this study, a data base containing the responses of 40,000 candidates to 90 multiple-choice questions was used to mimic data sets for 50-item tests under the nonequivalent groups with anchor test (NEAT) design. Using these smaller data sets, we evaluated the performance of five linear equating methods for the NEAT design with five levels of group differences and five levels of form differences. The completeness of the data base, with all 40,000 candidates answering all 90 items, allowed us to use a linear equating relationship based on a single group design for the full data base as the criterion for evaluating equating methods with respect to bias, root mean squared difference (RMSD), and differences associated with slopes. All five methods worked well when groups were similar, regardless of form differences, and the Levine methods showed smaller levels of bias and RMSD values than the other methods across a range of group differences. Overall, the Levine methods were most effective in controlling for group differences and, in particular, were most accurate when group differences were moderate to large.