Setting the Response Time Threshold Parameter to Differentiate Solution Behavior From Rapid-Guessing Behavior

Abstract

This study compared four methods for setting item response time thresholds to differentiate rapid-guessing behavior from solution behavior. Thresholds were either (a) common for all test items, (b) based on item surface features such as the amount of reading required, (c) based on visually inspecting response time frequency distributions, or (d) statistically estimated using a two-state mixture model. The thresholds were compared using the criteria proposed by Wise and Kong to establish the reliability and validity of response time effort scores, which were generated on the basis of the specified threshold values. The four methods yielded very similar results, indicating that response time effort is not very sensitive to the particular threshold identification method used. Recommendations are given regarding use of the various methods.