A Distributed Phoenix++ Framework for Big Data Recommendation Systems

Abstract

Recommendation systems are important big data applications that are used in many business sectors of the global economy. While many users utilize Hadoop-like MapReduce systems to implement recommendation systems, we utilize the highperformance shared-memory MapReduce system Phoenix++ to design a faster recommendation engine. In this paper, we design a distributed out-ofcore recommendation algorithm to maximize the usage of main memory, and devise a framework that invokes Phoenix++ as a sub-module to achieve high performance. The design of the framework can be extended to support different types of big data applications. The experiments on Amazon Elastic Compute Cloud (Amazon EC2) demonstrate that our new recommendation system can be faster than its Hadoop counterpart by up to 225% without losing recommendation quality.