A multi-view semi-supervised approach for task-level web search success evaluation

Abstract

Web search success evaluation is an effective way to evaluate the performance of search engines from the perspective of search experience. Many research efforts have been made to evaluate web search success via modeling user search behavior by analyzing search logs. Most of these studies consider the web search success evaluation as a binary classification problem, and use supervised learning approaches to evaluate whether a search experience is successful or not, which often require a large number of labeled data to learn accurately. Since unlabeled data of user search behaviors are easily obtainable in search logs, semi-supervised approaches have been exploited to improve the performance of web search success evaluation via combining labeled data and unlabeled data. However, the existing semi-supervised web search success evaluation approach would suffer from the model assumption violation, i.e., when the assumption of the model is not correct, training the web search success evaluation model with large amounts of unlabeled data would hurt the evaluation performance. In order to address this problem, a Multi-view Semi-Supervised web Search Success Evaluation (M4SE) approach is proposed, which exploits a multi-view mechanism during the semi-supervised learning process of web search success evaluation. M4SE considers the transitions between any two actions as the action view, and treats the dwell time between contiguous actions as the time view. M4SE uses the strategy of different parameter configurations on action and time views to generate diverse classifiers. Experiments on different search log datasets show that the proposed approach achieves better performance than the state-of-the-art semi-supervised approach for evaluating web search success.