32 The Statistical Procedures Used in National Assessment of Educational Progress: Recent Developments and Future Directions

Abstract

The National Assessment of Educational Progress (NAEP) is an ongoing survey of the performance of the school students in the U.S. in a number of subject areas, including reading, writing, and mathematics. This chapter discusses in detail the statistical model, the current estimation technique, possible alternatives, and future directions of psychometric research in NAEP. The reporting method used in NAEP relies on a complex latent regression model and sampling weights with some of the assessments involving more than a hundred thousand examinees. Estimating the model parameters, therefore, is not straightforward for NAEP data, and neither is reporting of the results. For example, in order to generate maximum (marginal) likelihood estimates of the parameters of the latent regression model, multivariate (of up to five dimensions) integrals must be evaluated. Also, computing standard errors is not straightforward because of the complicated nature of the model and the complex sampling scheme. Still, the current estimation method used in NAEP performs respectably. However, the method has often been criticized recently. A number of researchers have suggested alternatives to the current statistical techniques in NAEP.