The likelihood property in general retrieval operations

Abstract

Probabilistic models of information retrieval rank objects (e.g. documents) in response to a query according to the probability of some matching criterion (e.g. relevance). These models rarely yield an actual probability and their scoring functions are interpreted to be purely ordinal within a given retrieval task. In this paper we show that some scoring functions possess a likelihood property, which means that the scoring function indicates the likelihood of matching when compared to other retrieval tasks. This is potentially more useful than pure ranking even though it cannot be interpreted as an actual probability. This property can be detected by using two modified effectiveness measures, entire precision and entire recall. Experimental evidence is offered to show the existence of this property both for traditional document retrieval and for the analysis of crime data where suspects of an unsolved crime are ranked according to the probability of culpability.