Trust Region Levenberg-Marquardt Method for Linear SVM

Abstract

Support Vector Machine is an optimal margin based classification technique in Machine Learning. In this paper, we have proposed Trust Region Levenberg-Marquardt (TRLM) method as a novel problem solver for L2 regularized L2 loss (L2RL2) primal SVM classification problem. Levenberg-Marquardt (LM) method is an extension of Gauss-Newton method for solving least squares non-linear optimization problems and Trust-Region (TR) method is used to find the step size. In LM method, LM parameter $A$ is changed by an arbitrary factor but in TRLM instead of changing A, the trust region radius Δ is changed. The proposed solver for L2RL2 primal SVM, performs well with medium sized problems. Experimental results establish TRLM as a solver for linear SVM as it performs at par and better in selective cases than existing state of the art solvers in terms of test accuracy.