Multi-kernel approach to Parallelization of EM Algorithm for GMM Training

Abstract

Most machine learning algorithms need to handle large datasets. This feature often leads to limitations on processing time and memory. The Expectation-Maximization (EM) is one of such algorithms, which is used to train one of the most commonly used parametric statistical models, the Gaussian Mixture Models (GMM). All steps of the algorithm are potentially parallelizable once they iterate over the entire data set. In this work, we propose a parallel implementation of EM for training GMM using CUDA cores. Experimentation scenario consists of five different datasets and four metrics. Results show a speedup of 12.7 if compared to sequential version. With coalesced access to CUDA global memory and shared memory usage, we have achieved up to 99.4% of actual occupancy, regardless the number of Gaussians considered.