EXTENSION OF THE NAEP BGROUP PROGRAM TO HIGHER DIMENSIONS

Abstract

The reporting methods used in large scale assessments such as the National Assessment of Educational Progress (NAEP) rely on a latent regression model. The first component of the model consists of a p-scale IRT measurement model that defines the response probabilities on a set of cognitive items in p scales depending on a p-dimensional latent trait variable θ = (θ1, … θp). In the second component, the conditional distribution of this latent trait variable θ is modeled by a multivariate, multiple linear regression on a set of predictor variables, which are usually based on student, school and teacher variables in assessments such as NAEP. To fit the latent regression model using the maximum (marginal) likelihood estimation technique, multivariate integrals have to be evaluated. In the computer program MGROUP used by ETS for fitting the latent regression model to data from NAEP and other programs, the integration is currently done either by numerical quadrature for problems up to two dimensions or by an approximation of the integral. CGROUP, the current operational version of the MGROUP program used in NAEP and other assessments since 1993, is based on Laplace approximation, which may not provide fully satisfactory results, especially if the number of items per scale is small (see, e.g., Thomas, 1993a, or von Davier & Sinharay, 2004). There is scope for improvement in the technique used. This paper extends the NAEP BGROUP program to higher dimensions. Two real data analyses, one with a medium-sized data set and another with a large data set, show that the extension promises to be useful for fitting the NAEP model.