Robust Learning to Rank Based on Portfolio Theory and AMOSA Algorithm

Abstract

Effectiveness is the most important factor considered in the ranking models yielded by algorithms of learning to rank (LTR). Most of the related ranking models only focus on improving the average effectiveness but ignore robustness. When a ranking model ignores robustness, the effectiveness for many queries is possibly very poor although the average effectiveness for all queries is relatively high. Therefore, Wang et al. first consider robustness in their ranking models. However, the robustness formula defined by Wang et al. cannot characterize those queries whose effectiveness are hurt seriously in comparison with the baseline model. In order to overcome this shortcoming, we propose a novel formula of characterizing robustness based on portfolio theory, and construct a multiobjective optimization model of the robust LTR in which the formula is used. Based on this model, we propose an approach of risk-sensitive and robust LTR, named as R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Rank, which is based on the framework of archived multiobjective simulated annealing algorithm and the idea of preference ranking organization method for enrichment evaluation. The experimental results show that the ranking models produced by our proposed R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Rank approach are better in both effectiveness and robustness than those produced by three state-of-the-art LTR approaches.