Improved retrieval effectiveness through impact transformation

Abstract

Users of web search engines are notoriously parsimonious in their use of search terms, and search effectiveness has tended to be relatively poor on the resulting short queries, especially when compared against the good performance attained by recent systems when working with long TREC-like queries. In this paper we examine the reasons why short queries are hard to deal with, and propose a modification to the vector-space paradigm, which we call impact transformation. Experiments with a collection of web data and web queries shows that impact transformation significantly boosts retrieval effectiveness. Moreover, the use of quantised values in the transformation allows extremely fast query processing.