A Comparative and Performance Analysis of Similarity Metrics in Recommender System Based on Hadoop Framework

Abstract

With exponential growth of information available on web, there is a need for a personal assistance to customers for finding the best item out of the large set of items. There is also need for finding the best item having highest popularity. The personal assistance which will assist the user is the Recommender System. It is a software tool that uses knowledge discovery techniques to produce personalized recommendations. The majority of the recommendations are based on the machine learning algorithm and techniques which uses a standard data set for producing predictions. With tremendous growth of customers and items in the recent years, there is a key challenge for the recommendation system to produce quality recommendations based on the similarity of the users. There also lies a great challenge for the recommendation system to produce quality output when data set is huge, reduce latency, group users having similar interest and perform recommendations in seconds for millions of customers and products. Thus there is a need for new recommendation technologies which will produce quality output to users having similar tastes in a best possible manner. In this paper we present a comparative analysis of various similarity metrics used in recommendation system for clustering the users based on hadoop framework. The experimental results show that spearman rank correlation performs the best among the other similarity metrics based on three evaluation metrics.