Flexible Nonparametric Kernel Learning with Different Loss Functions

Abstract

AbstractSide information is highly useful in the learning of a nonparametric kernel matrix. However, this often leads to an expensive semidefinite program (SDP). In recent years, a number of dedicated solvers have been proposed. Though much better than off-the-shelf SDP solvers, they still cannot scale to large data sets. In this paper, we propose a novel solver based on the alternating direction method of multipliers (ADMM). The key idea is to use a low-rank decomposition of the kernel matrix Z = X ⊤ Y, with the constraint that X = Y. The resultant optimization problem, though non-convex, has favorable convergence properties and can be efficiently solved without requiring eigen-decomposition in each iteration. Experimental results on a number of real-world data sets demonstrate that the proposed method is as accurate as directly solving the SDP, but can be one to two orders of magnitude faster.KeywordsLoss FunctionSide InformationKernel MatrixPairwise ConstraintMultiple Kernel LearningThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.