Use of Mixed Item Response Theory in Rating Scales

Seher Yalçın

doi:10.26822/iejee.2019349251

Abstract

This study aimed to compare the Graded Response Model (GRM) and the Mixed-Graded Response Model (MixGRM) in terms of model data-fit and parameters and demonstrate the application of MixGRM on real data. In this context, this study is basic research based on the International Computer and Information Literacy Study in 2013 conducted with eighth-grade participants from Turkey. The data from a total of 2,356 students were used in the study. In testing the models, data was obtained from an 11-item Likert scale that measured the students' interest and enjoyment in using Information and Communication Technologies (ICTs). When the GRM- and MixGRM-based model data-fit results were compared, the model with the best fit was the MixGRM with four latent classes. Students who reported to enjoy using ICT and who had the highest computer and information literacy (CIL) score were found to be in the first latent class, those with least enjoyment or dislike and those with the lowest CIL score were in the fourth latency class. The findings show that reducing the heterogeneity of Mixed-Item Response Theory models in the dataset is a preferable model for research situations and that Turkish students are not yet prepared for life in the digital age.

Highlights

Different models and theories are being developed to make decisions about the results of tests taken by individuals more valid and reliable
There are studies in the literature suggesting that item response theory (IRT)-based parameter predictions are more reliable because they create homogeneous latent classes (LCs) according to the response pattern in data in heterogeneous groups, meaning that the sample consists of latent subclasses (De Ayala & Santiago, 2017; Maij-de Meij, Kelderman, & Van der Flier, 2008; Yalçın, 2018)
The current study presents an example of an application through real data and compares the Mixed-Graded Response Model (MixGRM) and Graded Response Model (GRM), which are among the Mixed-Item Response Theory (MixIRT) models for Likert-type items, are frequently used in measuring the latent traits of individuals, such as personality, interest, and attitude

Summary

Introduction

Different models and theories are being developed to make decisions about the results of tests taken by individuals more valid and reliable. The classical test theory has certain limitations (Hambleton, Swaminathan & Rogers, 1991), such as being dependent on group, individuals’ being dependent on the item they receive, the quality of the item being dependent on the responding group, the difficulty of comparing the individuals who take the different tests, being test based and the need for parallel tests for the reliability prediction. Given these limitations, IRT models are more frequently preferred.

Objectives

Methods

Results