The robust estimation of examinee ability based on the four-parameter logistic model when guessing and carelessness responses exist.

Xiaozhu Jian,Deng Yuanping,Dai Buyun,Lianmeng Jiao

doi:10.1371/journal.pone.0250268

Abstract

The three-parameter Logistic model (3PLM) and the four-parameter Logistic model (4PLM) have been proposed to reduce biases in cases of response disturbances, including random guessing and carelessness. However, they could also influence the examinees who do not guess or make careless errors. This paper proposes a new approach to solve this problem, which is a robust estimation based on the 4PLM (4PLM-Robust), involving a critical-probability guessing parameter and a carelessness parameter. This approach is compared with the 2PLM-MLE(two-parameter Logistic model and a maximum likelihood estimator), the 3PLM-MLE, the 4PLM-MLE, the Biweight estimation and the Huber estimation in terms of bias using an example and three simulation studies. The results show that the 4PLM-Robust is an effective method for robust estimation, and its calculation is simpler than the Biweight estimation and the Huber estimation.

Highlights

One of the major concerns in item response theory (IRT) is the robust estimation of latent ability
Random guessing responses undoubtedly exist in practice tests, especially for low-ability and medium-ability subjects
Cheating, sharing answers on similar tests, and other reasons result in response disturbances similar to random guessing

Summary

Introduction

One of the major concerns in item response theory (IRT) is the robust estimation of latent ability. In Case II, the approach using the 4PLM-Robust to robustify ability estimation is perceived as c1⁄40 for all items without random guessing ð16Þ g 1⁄4 1 À p carelessness errors appear where the carelessness parameter p in the 4PLM-Robust is set at 0.00, 0.01, and 0.03. The absolute values of the biases of the 2PLM-MLE, the BIW estimation, the Huber estimation, and the 4PLM-Robust (p = 0.00) of these subjects are quite small, close to zero. When the critical points of p and p in the 4PLM-Robust are equal to those in the response-generated model, the absolute values of the biases of the corresponding 4PLM-Robust are the smallest and close to zero among the four forms of the 4PLM-Robust, which indicates that the 4PLM-Robust (p = critical point, p = critical point) is better than the other robust methods in the four combinations.

Discussion and conclusion

Findings

Method