A Bayesian generalized multiple group IRT model with model-fit assessment tools

Abstract

The multiple group IRT model (MGM) proposed by Bock and Zimowski (1997) provides a useful framework for analyzing item response data from clustered respondents. In the MGM, the selected groups of respondents are of specific interest such that group-specific population distributions need to be defined. The main goal is to explore the potentials of an MCMC estimation procedure and Bayesian model-fit tools for the MGM. We develop a full Gibbs sampling algorithm (FGSA) for estimation as well as a Metropolis-Hastings within Gibss sampling algorithm (MHWGS) in order to use non-conjugate priors. The FGSA is compared with Bilog–MG, which uses marginal maximum likelihood (MML) and marginal maximum a posteriori (MMAP) methods. That is; Bilog–MG provides maximum likelihood (ML) and expected a posteriori (EAP) estimates for both item and population parameters, and maximum a posteriori (MAP) estimates for the latent traits. We conclude that, in general, the results from our approach are slightly better than Bilog–MG. Besides a simultaneous MCMC estimation procedure, model-fit assessment tools are developed. Furthermore, the prior sensitivity is investigated with respect to the parameters of the latent population distributions. It will be shown that the FGSA provides a wide set of model-fit tools. The proposed model assessment tools can evaluate important model assumptions of (1) the item response function (IRF) and (2) the latent trait distributions. The utility of the proposed estimation and model-fit assessment methods will be shown using data from a longitudinal data study concerning first to fourth graders of sampled Brazilian public schools.