Use of Response Time for Measuring Cognitive Ability

Patrick Kyllonen,Jiyun Zu

doi:10.3390/jintelligence4040014

Abstract

The purpose of this paper is to review some of the key literature on response time as it has played a role in cognitive ability measurement, providing a historical perspective as well as covering current research. We discuss the speed-level distinction, dimensions of speed and level in cognitive abilities frameworks, speed–accuracy tradeoff, approaches to addressing speed–accuracy tradeoff, analysis methods, particularly item response theory-based, response time models from cognitive psychology (ex-Gaussian function, and the diffusion model), and other uses of response time in testing besides ability measurement. We discuss several new methods that can be used to provide greater insight into the speed and level aspects of cognitive ability and speed–accuracy tradeoff decisions. These include item-level time limits, the use of feedback (e.g., CUSUMs), explicit scoring rules that combine speed and accuracy information (e.g., count down timing), and cognitive psychology models. We also review some of the key psychometric advances in modeling speed and level, which combine speed and ability measurement, address speed–accuracy tradeoff, allow for distinctions between response times on items responded to correctly and incorrectly, and integrate psychometrics with information-processing modeling. We suggest that the application of these models and tools is likely to advance both the science and measurement of human abilities for theory and applications.

Highlights

Response time has always been a concern in ability measurement, sometimes implicitly as in time-limit tests, sometimes explicitly as in so-called speeded tests, and sometimes as the main outcome variable as in reaction time, or information-processing tests
The purpose of this paper is to review some of the key literature on response time as it has played a role in cognitive ability measurement
Several policies were considered for the GRE computer adaptive testing (CAT), including an 80% completion minimum, a proportional scoring method, and a penalty function in which items not reached are assumed to be guessed at randomly and a score is assigned that way [55]

Summary

Introduction

Response time has always been a concern in ability measurement, sometimes implicitly as in time-limit tests, sometimes explicitly as in so-called speeded tests, and sometimes as the main outcome variable as in reaction time, or information-processing tests. The measurement of processing speed per se has been a part of testing since its earliest days. The use of computerized testing has led to significant development of models of response time because item response time is often automatically collected. A major challenge in the measurement of response time is that there are many factors that influence it, and it is typically impossible to attribute response time uniquely to any particular factor. A respondent’s slow response might reflect either slow processing speed or carefulness

Objectives

Findings

Discussion

Conclusion