Asynchronous Bundle Method for Large-Scale Regularized Risk Minimization

Abstract

Bundle method for regularized risk minimization (BMRM) is a variant of Cutting Plane Method (CPM). It performs efficiently in solving a convex minimization problem, which is a core part in a plethora of machine learning applications. Nonetheless, while exposed to the challenge of large-scale learning, the synchronous parallel implementation of BMRM easily encounters the straggler problem due to the diversity among heterogeneous working nodes’ capability and unevenness in the inherent data distribution. In this paper, we propose a novel asynchronous distributed BMRM implementation, which employs an asynchronous computing window to fully explore the fast nodes’ computational capabilities while reserving the good convergence of the BMRM. Extensive experiments show that the asynchronous BMRM algorithm has significant improvement of performance over its synchronous counterpart, and owns the ability to solve large-scale problems efficiently.