Empirical studies of end-user information searching

Abstract

An empirical investigation of information retrieval (IR) using the MEDLINE1 database was carried out to study user behaviour, performance and to investigate the reasons for suboptimal searches. The experimental subjects were drawn from two groups of final year medical students who differed in their knowledge of the search system (i.e., novice and expert users). The subjects carried out four search tasks and their recall and precision performance was recorded. Data was captured on the search strategies used, duration, and logs of submitted queries. Differences were found between the groups for the performance measure of recall in only one of the four experimental tasks. Overall performance was poor. Analysis of strategies, timing data, and query logs showed that there were many different causes for search failure or success. Poor searchers either gave up too quickly, employed few search terms, used only simple queries, or used the wrong search terms. Good searchers persisted longer, used a larger, richer set of terms, constructed more complex queries, and were more diligent in evaluating the retrieved results. However, individual performances were not correlated with all of these factors. Poor performers frequently exhibited several factors of good searcher behaviour and failed for just one reason. Overall end-user searching behaviour is complex and it seems that just one factor can cause poor performance, whereas good performance can result from suboptimal strategies that compensate for some difficulties. The implications of the results for the design of IR interfaces are discussed.