Research on parallelisation of collaborative filtering recommendation algorithm based on Spark

Abstract

More and more people become conscious of the recommendation system to make good use of the data through their inherent advantages faced with the large amount of data on the Internet. The collaborative filtering recommendation algorithm cannot avoid the bottleneck of computing performance problems in the recommendation process. In this paper, we propose a parallel collaborative filtering recommendation algorithm RLPSO_KM_CF which is implemented based on Spark. Firstly, the RLPSO (reverse-learning and local-learning PSO) algorithm is used to find the optimal solution of particle swarm and output the optimised clustering centre. Then, the RLPSO_KM algorithm is used to cluster the user information. Finally, make effective recommendations to the target user by combining the traditional user-based collaborative filtering algorithm with the RLPSO_KM clustering algorithm. The experimental results show that the RLPSO_KM_CF algorithm has a significant improvement in the recommendation accuracy and has a higher speed-up and stability.