Implementing Item Response Theory (IRT) Method in Quiz Assessment System

Abstract

A quiz is one of the methods to test students’ abilities. Estimation of ability scores can be calculated by using the Classical Test Theory (CTT) and Item Response Theory (IRT) approaches. IRT is more sensitive to item characteristics that can estimate students’ ability. This study implements IRT in a quiz assessment system, compares the results of the CTT and IRT, then analyses the functionality and usability of the built system. The system was tested by 50 respondents using 30 question items with 5 answer choices. The collected data is automatically assessed by the system using CTT and IRT calculations. The results showed differences in ranking between CTT and IRT methods. Using the IRT method, there were 14 respondents who experienced an increase in rank, 19 respondents experienced a decrease in rank, and the remaining 17 respondents did not experience a change in ranking. The functionality of built system was tested with the Blackbox Testing method and it has passed all the test functions. The usability of the system was also tested by involving 23 respondents using the System Usability Scale (SUS) questionnaire and the SUS score showed that the system can be accepted by users.