Abstract
Many test-takers do not carefully answer every test question; instead they sometimes quickly answer without thoughtful consideration (rapid guessing, RG). Researchers have not modeled RG when assessing student learning with cognitive diagnostic models (CDMs) to personalize feedback on a set of fine-grained skills (or attributes). Therefore, this study proposes to enhance cognitive diagnosis by modeling RG via an advanced CDM with item response and response time. This study tests the parameter recovery of this new CDM with a series of simulations via Markov chain Monte Carlo methods in JAGS. Also, this study tests the degree to which the standard and proposed CDMs fit the student response data for the Programme for International Student Assessment (PISA) 2015 computer-based mathematics test. This new CDM outperformed the simpler CDM that ignored RG; the new CDM showed less bias and greater precision for both item and person estimates, and greater classification accuracy of test results. Meanwhile, the empirical study showed different levels of student RG across test items and confirmed the findings in the simulations.
Highlights
Cognitive diagnostic models (CDMs) assess whether test-takers have the skills needed to answer test questions, so that their test results can give them diagnostic feedback on their strengths and weaknesses in these attributes (Rupp et al, 2010)
We focus on two frequently-observed test-taking behaviors during actual tests: solution attempt and rapid guessing (RG; Wise and Kong, 2005)
This study proposes a new framework of cognitive diagnostic models (CDMs) to recognize different test-taking behaviors by using reaction time (RT) and item responses simultaneously
Summary
Cognitive diagnostic models (CDMs) assess whether test-takers have the skills needed to answer test questions (attributes), so that their test results can give them diagnostic feedback on their strengths and weaknesses in these attributes (Rupp et al, 2010). Some test-taking behaviors can distort current CDM results and thereby jeopardize the validity of their assessments. Researchers have proposed different approaches to account for test-taking behaviors when assessing test-taker performance and item characteristics. We focus on two frequently-observed test-taking behaviors during actual tests: solution attempt and rapid guessing (RG; Wise and Kong, 2005). Test-takers carefully try to find answers to test questions. RG refers to test-takers quickly answering test questions without thoughtful consideration (e.g., Wise and DeMars, 2006).
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
