A model to improve recommendation systems based on Spark platform

Abstract

With the popularity of using the internet, the evolution of technology, the rapid growth of data volume, and applying recommendation systems in diverse life domains, adopting recommendation systems became indispensable. Most of the traditional recommendation systems are built un-oriented of real-time data challenges (such as item cold-start, scalability, heterogeneous data, items change, and users' opinions change), resulting in ineffective, unfulfilling recommendations. Therefore, what is needed is a real-time-based recommendation model that can make the user find what s/he needs without even searching for it; it's like building a model aware of the users' nature and behaviour. This paper aims at building a recommendation model that fulfils the user requirements and overcomes the shortcomings of the traditional recommendation systems. The proposed recommendation model is built on Apache Spark, Apache Kafka and alternative least square (ALS) algorithm, and the model is divided into two parts: recommendation system and real-time analytics dashboard. An e-commerce website is chosen as a case study to apply our model and the results are presented.