An Efficient Parallelization Model for Sparse Non-negative Matrix Factorization Using cuSPARSE Library on Multi-GPU Platform

Abstract

AbstractPositive or Non-negative Matrix Factorization (NMF) is an effective technique and has been widely used for Big Data representation. It aims to find two non-negative matrices W and H whose product provides an optimal approximation to the original input data matrix A, such that \(A\approx W*H\). Although, NMF plays an important role in several applications, such as machine learning, data analysis and biomedical applications. Due to the sparsity that is caused by missing information in many high-dimension scenes (e.g., social networks, recommender systems and DNA gene expressions), the NMF method cannot mine a more accurate representation from the explicit information. Therefore, the Sparse Non-negative Matrix Factorization (SNMF) can incorporate the intrinsic geometry of the data, which is combined with implicit information. Thus, SNMF can realize a more compact representation for the sparse data. In this paper, we study the Sparse Non-negative Matrix Factorization (SNMF). We use Multiplicative Update Algorithm (MUA) that computes the factorization by applying update on both matrices W and H. Accordingly, to address these issue, we propose a two models to implement a parallel version of SNMF on GPUs using NVIDIA CUDA framework. To optimize SNMF, we use cuSPARSE optimized library to compute the algebraic operations in MUA where sparse matrices A, W and H are stored in Compressed Sparse Row (CSR) format. At last, our contribution is validated through a series of experiments achieved on two input sets i.e. a set of randomly generated matrices and a set of benchmark matrices from real applications with different sizes and densities. We show that our algorithms allow performance improvements compared to baseline implementations. The speedup on multi-GPU platform can exceed \(11\times \) as well as the Ratio can exceed 91%.KeywordsSNMFUnsupervised learning algorithmMultiplicative update algorithmSparse matrixCompressed formatsOptimization