Information Retrieval Evaluation Measures Defined on Some Axiomatic Models of Preferences

Abstract

Information retrieval (IR) evaluation measures are essential for capturing the relevance of documents to topics and determining the task performance efficiency of retrieval systems. The study of IR evaluation measures through their formal properties enables a better understanding of their suitability for a specific task. Some works have modeled the effectiveness of retrieval measures with axioms, heuristics, or desirable properties, leading to order relationships on the set where they are defined. Each of these ordering structures constitutes an axiomatic model of preferences (AMP), which can be considered as an “ideal” scenario of retrieval. Based on lattice theory and on the representational theory of measurement, this work formally explores numeric, metric, and scale properties of some effectiveness measures defined on AMPs. In some of these scenarios, retrieval measures are completely determined from the scores of a subset of document rankings: join-irreducible elements. All the possible metrics and pseudometrics, defined on these structures are expressed in terms of the join-irreducible elements. The deduced scale properties of the precision, recall, F -measure, RBP , DCG , and AP confirm some recent results in the IR field.