Implementation of multi-criteria collaborative filtering on cluster using Apache Spark

Abstract

Scalability is a problem commonly faced by a recommendation system that uses collaborative filtering methods. Multi-criteria collaborative filtering recommender system has the exact same problem. The performance of multi-criteria collaborative filtering is reduced when the amount of data processed by recommender system is increasing too high. This research aims to complement previous research which is to improve the scalability of multi-criteria collaborative filtering recommender system by applying scale-out approach or adding computer node to run the recommender system. The process of generating a recommendation on multi-criteria collaborative filtering recommender system will be done on multiple nodes of computer network inside a cluster using Apache Spark framework. To measure system scalability, the running time of multi-criteria collaborative filtering recommender system that are implemented as a recommender program on Apache Spark cluster will be compared in the form of speedup value. Based on test results, it is known that multi-criteria collaborative filtering on Apache Spark cluster has better running time than its sequential counterpart. Unfortunately, as the numbers of nodes inside cluster are increased, multi-criteria collaborative filtering recommender system on Apache Spark cluster does not gain ideal speedup.