The Effect of Proportion Common Item's with Mixed Format Test on Multidimensional Item Response Theory Linking

Abstract

Numerous assessments contain a mixture of multiple choice and constructed response item types, which are found to measure more than one trait. Thus, there is a need for multidimensional dichotomous and polytomous item response theory modeling solutions, including multidimensional linking scores. Common items are most important for linking scores. Practitioners need empirical data to inform their selection of the number common items’ for linking scores. Previous research conducted in tests composed exclusively of multiple-choice items demonstrated that common-item sets should be representative of the overall test. The extent to which the number of common items in a mixed-format test differed measured different aspects of the construct. This research explored the effects of proportion common items’ with mixed format test on the multidimensional item response theory linking by Non-Orthogonal Procrustes Method. There were 3 conditions of proportion common items’ with mixed format test consisting of 20%, 25% and, 30% of total items, respectively. This research was based on data simulation using Monte Carlo Method and included 3,000 examinees. Data simulation consisted of three steps - generating true item parameters and response patterns for each grade level, calibrating multidimensional item response theory parameter model, and equating the procedure of linking. RMSE and BIAS were used as criteria to compare the quality of linking scores in each condition. Results showed that the proportion of common items affected the difference between the stability and the accuracy of linking method at .05 significant level. In other words, the difference in quality linking method depended on proportion of common items, especially the condition of 25% which approximated to multidimensional linking scores.