A heuristic similarity measure and clustering model to improve the collaborative filtering algorithm

Abstract

In this paper, we proposed a collaborative filtering recommendation algorithm based on heuristic similarity measure and clustering, in order to alleviate the problem of data sparsity in collaborative filtering algorithm. Firstly, a PSD (Proximity-Significance-Distinction) similarity measure based on rating matrix was proposed, and the score difference in the use of the sigmoid function was made more obvious by expansion of the range of independent variables. On this basis, a multi-factor collaborative filtering recommendation algorithm based on Particle Swarm Optimization (PSO) was proposed, and the PSO algorithm was used to obtain the optimal weight combination of the similarity influence factors, so that the similarity measurement became more accurate. Further, we implemented an improved K nearest neighbor recommendation based on clustering algorithm for generation of a better recommendation list. The method divided the clusters based on the PSD similarity measure proposed in this paper, and searched the nearest K neighbors in the cluster to which the target user belongs, so as to reduce the search time of the nearest neighbor, and obtain a more accurate neighbor set. Finally, a comparative experiment on Movie Lens dataset shows that the proposed algorithm has improved the quality and accuracy of recommendation, thus overcome the data sparseness problem to a certain extent.