GPU Acceleration of Interior Point Methods in Large Scale SVM Training

Abstract

The convex quadratic programming problem, involved in the large scale support vector machine (SVM) training phase, is computationally expensive. Interior Point Methods (IPM) have been used successfully to solve this problem. They have polynomial time complexity and maintain a constant predictable structure of the linear system that needs to solve each iteration in IPM. The main problem is its complexity both in workload and storage when it is used for real-life problems with millions of examples. This paper proposes an approach that significantly improves the performance of large scale SVM training on GPU-equipped cluster. It exploits the parallelism of IPM with Compute Unified Device Architecture (CUDA) on NVIDIA GTX480 GPUs. The dominant cost of several operations such as Cholesky Factorization (CF) motivates the implementation on GPU to yield further performance gains. The proposed solution allows efficient training on the large datasets, such as cover types, rcv1 and url. The speedup achieved with GPUs is about 3 over using only quad-core processors on our 5-node cluster. The equivalent speedup of a single node over LibSVM is about 90 times for the big dataset. It demonstrates that we can improve performance on clusters sufficiently by using GPUs in the large scale SVM training.