Enumerative feature subset based ranking system for learning to rank in presence of implicit user feedback

Abstract

This paper proposed a new method for learning to rank documents using enumerative feature subsetting in the presence of the implicit user feedback of the various classes of users. The objective of this research was to provide an alternative method for learning ranking functions using important subsets of the LETOR (Learning to Rank) features based on the feedback of various classes of the users identified from active subsets of the features. This research, unlike other feature engineering approaches, do not force the learner to drop the inactive features permanently; instead, it allows to learn ranking function on currently active feature subsets while keeping inactive subsets of the features in the training process. The proposed model allows the search engine to dynamically utilize the implicit user feedback of the various classes of the users in learning ranking models repeatedly. The experiments performed on the LETOR MQ2008 dataset shows that the proposed model gives better NDCG (Normalized Discounted Cumulative Gain) scores for the subsets of users in the ensemble settings. Results also show that the variance of the predicted ranking can be controlled by controlling the hyper-parameters like the probability of the selection of a subset, feedback on the subsets and the weights of each subset used in training the low-level ranker. We have used cross-entropy pairwise learner RankNet and the maximum margin type svmRank as low-level rankers, but their objective functions are modified for the feature subsets. Results obtained by bagging and boosting based ensemble methods shows that the proposed method is flexible enough to model a family of the feedback weights on the individual models and can be used to provide personalized rank learning functions to the selected subsets of the users.