A scalable P2P recommender system based on distributed collaborative filtering

Abstract

Collaborative Filtering (CF) technique has been proved to be one of the most successful techniques in recommender systems in recent years. However, most existing CF based recommender systems worked in a centralized way and suffered from its shortage in scalability as their calculation complexity increased quickly both in time and space when the record in user database increases. In this article, we first propose a distributed CF algorithm called PipeCF together with two novel approaches: significance refinement and unanimous amplification, to further improve the scalability and prediction accuracy. We then show how to implement this algorithm on a Peer-to-Peer (P2P) structure through distributed hash table method, which is the most popular and efficient P2P routing algorithm, to construct a scalable distributed recommender system. The experimental data show that the distributed CF-based recommender system has much better scalability than traditional centralized ones with comparable prediction efficiency and accuracy. q 2004 Elsevier Ltd. All rights reserved.