An Efficient Parallel Stochastic Gradient Descent for Matrix Factorization On GPUS

Abstract

Matrix factorization (MF) is an essential method used in recommender systems, database systems, word-embedding, Graph-mining, and others. Stochastic gradient descent (SGD) is a widely-used method of solving the MF problem because it has effective accuracy in dealing with large datasets and high computing speed. SGD is hard to be parallelized as it is a sequential algorithm, but there are also some effective parallel methods proposed by researches. In this research, we propose EMF-SGD, an effective GPU-based method of large-scale recommender systems. EMF-SGD accelerated the SGD algorithm by utilizing the GPU shared-memory and warp operations. Besides, we focus on maintaining the relationship between users and items in preprocessing data to gain higher accuracy. Finally, we parallelize the EMF-SGD on multi-GPUS and proved it gains 1.8-4.3x speed up and higher accuracy over the most state-of arts algorithm GPU-MF-SGD, based on the different amount of GPUS we used.