Using Bayesian Nonparametric Item Response Function Estimation to Check Parametric Model Fit.

Abstract

Previous studies indicated that the assumption of logistic form of parametric item response functions (IRFs) is violated often enough to be worth checking. Using nonparametric item response theory (IRT) estimation methods with the posterior predictive model checking method can obtain significance probabilities of fit statistics in a Bayesian framework by accounting for the uncertainty of the parameter estimation and can indicate the location and magnitude of misfit for an item. The purpose of this study is to check the performance of the Bayesian nonparametric method to assess the IRF fit of parametric IRT models for mixed-format tests and compare it with the existing bootstrapping nonparametric method under various conditions. The simulation study results show that the Bayesian nonparametric method can detect misfit items with higher power and lower type I error rates when the sample size is large and with lower type I error rates compared with the bootstrapping method for the conditions with nonmonotonic items. In the real-data study, several dichotomous and polytomous misfit items were identified and the location and magnitude of misfit were indicated.