Comparison of Item Difficulty Estimates in a Basic Statistics Test using ltm and CTT Software Packages in R

Jonald L Pimentel,Marah Luriely

doi:10.14569/ijacsa.2020.0110346

Abstract

Two free computer software packages “ltm” and “CTT” in the R software environment were tested to demonstrate its usefulness in an item test analysis. The calibration of the item difficulty parameters given the binary responses of two hundred five examinees for the fifteen items multiple choice test were analyzed using the Classical Test Theory (CTT) and Item Response Theory (IRT) methodologies. The software latent trait model “ltm” employed the IRT framework while the software classical test theory functions “CTT” operated under CTT. The IRT Rasch model was used to model the responses of the examinees. The conditional maximum likelihood estimation method was used to estimate the item difficulty parameters for all the items. On the other hand, all the item difficulty indices using the “CTT” software were also calculated. Both the statistical analyses of this study were done in the R software. Results showed that among the fifteen items, the estimates of their item difficulty parameters differed mostly on their values between the two methods. In an IRT framework, items showed extreme difficulty or easy cases as compared to CTT. However, when the estimated values were categorized into intervals and labelled according to its verbal difficulty description, both methodologies showed some similarities in their item difficulties.

Highlights

In the field of education in test and measurement, it is important that any method that uses technology should be upgraded from time to time
The primary objective of this paper is to demonstrate the usefulness of the two computer software programs, the latent trait model “ltm” [1] and the classical test theory functions “CTT” [2] in the R software environment in the calibration of item difficulty parameter estimates/indices for a multiplechoice test
In the calibration of item parameters, the difficulty indices β of an examination test say in the case of a multiple-choice test in which the resulting data is a matrix of binary responses of the number of examinees who took the examination and the number of items being answered, Two methodologies are available at present in the literatures to handle such calibration

Summary

Introduction

In the field of education in test and measurement, it is important that any method that uses technology should be upgraded from time to time. In the calibration of item parameters, the difficulty indices β of an examination test say in the case of a multiple-choice test in which the resulting data is a matrix of binary responses of the number of examinees who took the examination and the number of items being answered, Two methodologies are available at present in the literatures to handle such calibration These methods are the Classical Test Theory (CTT) which is based on prediction of outcomes on a test that is, in particular an examinee’s observed score which is composed of a true score and an error score and the Item Response Theory (IRT) which is based on a response probabilistic modeling [4]. See [7], [8], [9] and [10] for more discussion about these item response theory modeling

Objectives

Methods

Results

Conclusion