Using dwell time as an implicit measure of usefulness in different task types

Abstract

In this study, we attempted to use dwell time on content pages as a predictor of document usefulness, and evaluated the prediction performance in different types of tasks. A user study was conducted to address this research problem. A total of 32 participants conducted searches associated with 4 different tasks, varying along several task type dimensions. In the study, participants were asked to save documents that were useful for the tasks during the searches. Participants' information-seeking activities were recorded. Using the logged dataset, we used the recursive partitioning method to identify the best threshold (cutoff point) of dwell time to predict which pages would be useful, i.e., be saved by the users. We examined the differences in the threshold and compared the prediction performance for each type of task. Our results indicate that the threshold should be developed according to the type of task users are working on. While the prediction performance using dwell time as the only indicator of document usefulness is acceptable for Factual tasks, it is not good for Intellectual tasks. Our results have implications on IR system design.