Accelerating Machine Learning on Sparse Datasets with a Distributed Memory Vector Architecture

Abstract

We propose methods to accelerate machine learning (ML) on sparse datasets with a distributed memory vector architecture. First, we propose a new communication method that reduces the amount of communication by exploiting the sparsity of the data. Second, we propose a new sparse matrix vector multiplication (SpMV) for a vector architecture, which often becomes the kernel operation of ML on sparse datasets. The method keeps the vector length long even in the case of a power law matrix that typically appears in ML workloads. We implemented middleware that incorporates these two methods so that programmers who implement ML algorithms can easily utilize them. The evaluation results show that these methods improve the performance of ML algorithms by up to 4 times. The resulting performance of a vector architecture machine was up to 8.1 times faster than a commodity PC cluster with the same number of cores.