How many relevances in information retrieval?

Abstract

The aim of an information retrieval system is to find relevant documents, thus relevance is a (if not ‘the’) central concept of information retrieval. Notwithstanding its importance, and the huge amount of research on this topic in the past, relevance is not yet a well understood concept, also because of inconsistently used terminology. In this paper, I try to clarify this issue, classifying the various kinds of relevance. I show that: (i) there are many kinds of relevance, not just one; (ii) these kinds can be classified in a formally defined four dimensional space, and (iii) such classification helps us to understand the nature of relevance and relevance judgement. Finally, the consequences of this classification on the design and evaluation of information retrieval systems are analysed.