An empirical study of alternating least squares collaborative filtering recommendation for Movielens on Apache Hadoop and Spark

Abstract

In recent years, both consumers and businesses have faced the problem of information explosion, and the recommendation system provides a possible solution. This study implements a movie recommendation system that provides recommendations to consumers in an effort to increase consumer spending while reducing the time between film selection. This study is a prototype of collaborative filtering recommendation system based on Alternating Least Squares (ALS) algorithm. The advantage of collaborative filtering is that it avoids possible violations of the Personal Data Protection Act and reduces the possibility of errors due to poor quality of personal data. Our research improves the ALS's limited scalability by using a platform that combines Spark with Hadoop Yarn and uses this combination to calculate movie recommendations and store data separately. Based on the results of this study, our proposed system architecture provides recommendations with satisfactory accuracy while maintaining acceptable computational time with limited resources.