An Extension of Rudner-Based Consistency and Accuracy Indices for Multidimensional Item Response Theory

Abstract

Although the field of multidimensional item response theory (MIRT) has enjoyed tremendous growth over recent years, solutions to some problems remain to be studied. One case in point is the estimate of classification accuracy and consistency indices. There have been a few research studies focusing on these indices based on total scores under MIRT. The purposes of this study are to extend Rudner-based index for MIRT under complex decision rules and to compare it with the Guo-based index and the Lee-based index. The Rudner-based index assumes that an ability estimation error follows a multivariate normal distribution around each examinee’s ability estimate, and a simple Monte Carlo method is used to estimate accuracy and consistency indices. The simulation results showed that the Rudner-based index worked well under various conditions. Finally, conclusions are described along with thoughts for future research.