Semi-Active Learning to Rank Algorithms for Document Retrieval

Abstract

Recently, several search engine applications are using learning to rank technologies to train their ranking models whose performance is strongly affected by labelled examples' number in the training set. Since these labels might be costly to acquire as labelling is usually scarce and expensive to get, active learning and semi-supervised learning technologies aim to reduce manual labelling workload. In this paper, we propose two inductive learning to rank strategies of alternatives that combine active and semi-supervised learning to assign the relevance scores to an unlabeled set of document-query pairs, using selectively sampled and automatically labelled data. These propositions enable the exploitation of all collected data and the avoidance of some problems caused by employing only active or semi-supervised learning. We showed through different ranking measures that the algorithms proposed yielded into competitive results compared to some other semi-supervised and active ranking algorithms on collections from the standard benchmark Letor.