Maximin Calibration Designs for the Nominal Response Model: an Empirical Evaluation

Abstract

The problem of finding optimal calibration designs for dichotomous item response theory(IRT) models has been extensively studied in the literature. In this study, this problem will be extended to polytomous IRT models. Focus is given to items described by the nominal response model (NRM). The optimization’s objective is to minimize the generalized variance of item parameter estimates. For nonlinear models, Fisher’s information depends on the unknown parameters to be estimated. By assuming knowledge of the latter, D-optimal designs can be found, however, with the inconvenience that these are only locally optimal. A modified maximin approach, based on the relative efficiency of calibration designs with respect to the D-optimal design, is introduced as a viable method to circumvent the problem of local optimality. The performance of maximin designs is compared to that of alternative calibration designs, those with normally and uniformly distributed latent trait values. Index terms: polytomous IRT models, item calibration, maximin design, relative efficiency.