Query exhaustivity, relevance feedback and search success in automatic and interactive query expansion

Abstract

This study explored how the expression of search facets and relevance feedback (RF) by users was related to search success in interactive and automatic query expansion in the course of the search process. Search success was measured both in the number of relevant documents retrieved, whether identified by users or not. Research design consisted of 26 users searching for four TREC topics in Okapi IR system, half of the searchers using interactive and half automatic query expansion based on RF. The search logs were recorded, and the users filled in questionnaires for each topic concerning various features of searching. The results showed that the exhaustivity of the query was the most significant predictor of search success. Interactive expansion led to better search success than automatic expansion if all retrieved relevant items were counted, but there was no difference between the methods if only those items recognised relevant by users were observed. The analysis showed that the difference was facilitated by the liberal relevance criterion used in TREC not favouring highly relevant documents in evaluation.