The Effect of Item Preknowledge on Response Time: Analysis of Two Datasets Using the Multiple‐Group Lognormal Response Time Model with a Gating Mechanism

Abstract

AbstractResponse time information has recently attracted significant attention in the literature as it may provide meaningful information about item preknowledge. The methods that use response time information to identify examinees with potential item preknowledge make an implicit assumption that the examinees with item preknowledge differ in their response time patterns when compared to examinees without item preknowledge. This study uses a multiple‐group extension of van der Linden's Lognormal Response Time Model with a gated mechanism to quantify the differences in latent speed between examinees with and without item preknowledge when responding to compromised and uncompromised items. Our findings indicate that there may be a wide range for potential effect of item preknowledge on response time. The effect of item preknowledge on response times is likely to be contextual and depends on many factors such as characteristics of test‐takers, the nature of item preknowledge, and item characteristics. We recommend that researchers systematically quantify the effect sizes in their studies, and if possible, manipulate the effect size for a wide range while studying the effectiveness of already existing or new methods to detect examinees with item preknowledge through simulation.