Improving Scalability of Personalized Recommendation Systems for Enterprise Knowledge Workers

Abstract

Enterprise knowledge workers have been overwhelmed by the growing rate of incoming data in recent years. In this paper, we present a recommendation system with the goal of helping knowledge workers in discovering useful new content. In particular, our system builds personalized user models based on file activities on enterprise network file servers. Our models use novel features that are derived from file metadata and user collaboration. Through extensive evaluation on real-world enterprise data, we demonstrate the effectiveness of our system with high precision and recall values. Unfortunately, our experiments reveal that per-user models are unable to handle heavy workloads. To address this limitation, we propose a novel optimization technique, active feature-based model selection, that predicts the user models that should be applied on each test file. Such a technique can reduce the classification time per file by as much as 23 times without sacrificing accuracy. We also show how this technique can be extended to improve the scalability exponentially at marginal cost of prediction accuracy, e.g., we can gain 169 times faster performance on an average across all shares by sacrificing 4% of F-score.