The exploration of objective task difficulty and domain knowledge effects on users' query formulation

Abstract

In this paper, we explore the effects of objective task difficulty and domain knowledge on users' query formulation behaviors. The dataset from a user experiment was used in this research, which focused on the medical domain. The objective difficulty was measured by the precision level of search topics (as queries) in the search system, and searchers' domain knowledge was assessed by a self-reported rating on the familiarity with MeSH terms that were related to the search topics in the study. We compared expert searchers' and novice searchers' query similarity and query features between easy and difficult tasks. The results showed that there was no significant difference between domain experts and novices in query similarity, but there existed an opposite pattern of task difficulty effects on query similarity for searchers with different domain knowledge levels. Domain expert searchers had more diverse vocabulary in difficult tasks than in easy tasks, whereas novice searchers had to rely heavily on task descriptions to formulate their queries. Task difficulty had also influenced searchers' performance, especially the precision measure. Novice searchers' recall was relatively low in both easy and difficult tasks. The findings in this study help us further understand users' query formulation process, and have implication for the design of query suggestion functions in search systems.