The improvement and implementation of distributed item-based collaborative filtering algorithm on Hadoop

Abstract

In the background of big data era, after analyzing the traditional collaborative filtering algorithm, this paper proposes improved item-based collaborative filtering algorithm using “hotweight” as the weight, which is aimed at improving the accuracy of the algorithm and overcoming the defects such as rarefaction and cold-starting. We distribute the algorithm with MapReduce framework, and apply it to the distributed cluster platform Hadoop. This paper adopts real data set to run the algorithm and the experiment's result expresses that the improved algorithm can run efficiently on the large amounts of data with the better accuracy, and at the same time, can overcome the cold-starting drawback successfully.