Comparison of Testlet Effect on Parameter Estimates Using Different Item Response Theory Models

Abstract

In this study, the testlet effect was calculated for each testlet in the PISA 2018 reading literacy test, and it was examined whether this effect caused a difference in item and ability parameters. The data set was analyzed with a two-parameter logistic item response theory model and a two-parameter logistic testlet model. The results show that variances of testlet effects range from .100 to .432. When the item and ability parameter estimation results of the models were compared, it was determined that the item and ability parameters estimated from the two approaches were highly correlated with each other. It can be said that the item slope and item intercept parameters estimated from different models remained unaffected. However, when the local dependency assumption is not met, it was observed that the standard error values of the two-parameter model for the ability parameter were underestimated. The implications for the analysis and evaluation of the tests based on testlet are discussed. In conclusion, in this study, it was concluded that the testlet effect caused a difference in parameter estimates, but the local dependence among the items was negligible because of the small testlet effects.