Abstract
Result diversification aims to provide searchers with a broader view of a given topic while attempting to maximise the chances of retrieving relevant material. Diversifying results also aims to reduce search bias by increasing the coverage over different aspects of the topic. As such, searchers should learn more about the given topic in general. Despite diversification algorithms being introduced over two decades ago, little research has explicitly examined their impact on search behaviour and performance in the context of Interactive Information Retrieval (IIR). In this paper, we explore the impact of diversification when searchers undertake complex search tasks that require learning about different aspects of a topic (aspectual retrieval). We hypothesise that by diversifying search results, searchers will be exposed to a greater number of aspects. In turn, this will maximise their coverage of the topic (and thus reduce possible search bias). As a consequence, diversification should lead to performance benefits, regardless of the task, but how does diversification affect search behaviours and search satisfaction? Based on Information Foraging Theory (IFT), we infer two hypotheses regarding search behaviours due to diversification, namely that (i) it will lead to searchers examining fewer documents per query, and (ii) it will also mean searchers will issue more queries overall. To this end, we performed a within-subjects user study using the TREC AQUAINT collection with 51 participants, examining the differences in search performance and behaviour when using (i) a non-diversified system (BM25) versus (ii) a diversified system (BM25 + xQuAD) when the search task is either (a) ad-hoc or (b) aspectual. Our results show a number of notable findings in terms of search behaviour: participants on the diversified system issued more queries and examined fewer documents per query when performing the aspectual search task. Furthermore, we showed that when using the diversified system, participants were: more successful in marking relevant documents, and obtained a greater awareness of the topics (i.e. identified relevant documents containing more novel aspects). These findings show that search behaviour is influenced by diversification and task complexity. They also motivate further research into complex search tasks such as aspectual retrieval—and how diversity can play an important role in improving the search experience, by providing greater coverage of a topic and mitigating potential bias in search results.
Highlights
Interactive Information Retrieval (IIR) is a complex process (Ingwersen and Järvelin 2005) in which a searcher issues a variety of queries as a means to explore the topic space (Kelly et al 2015)
Syed and Collins-Thompson (Syed and Collins-Thompson 2017) hypothesised that diversifying the results presented to searchers would improve their learning efficiency and that this would be observed by a change in vocabulary expressed in their queries
Given the findings presented by Syed and Collins-Thompson (2017), we hypothesise that diversification will lead to a greater awareness of the topic, regardless of the task
Summary
Interactive Information Retrieval (IIR) is a complex (and often exploratory) process (Ingwersen and Järvelin 2005) in which a searcher issues a variety of queries as a means to explore the topic space (Kelly et al 2015) Often, such tasks are aspectual in nature, where an underlying goal is to find out about the different facets, dimensions or aspects of the topic. While aspectual retrieval has been heavily studied in the past (during the TREC Interactive Tracks Over 2001), there has been renewed interest in the search task as it represents a novel context to explore the idea of “search as learning” (Collins-Thompson et al 2017) In this context, the goal of the system is to help the searcher learn about a topic (Collins-Thompson et al 2017)—and in doing so, the number of aspects that the searcher finds indicates how much they learned during the process (Syed and CollinsThompson 2017). Other aspects (in this case, species of endangered animals) would need to be found, such as ‘Sumatran Rhinos in Malaysia’, ‘Crested Ibis in Japan’, ‘Black-Necked Crane in India’ etc
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
