Joint sample and feature selection via sparse primal and dual LSSVM

Abstract

Least squares support vector machine (LSSVM) is a powerful pattern recognition method. However, the solution of LSSVM loses sample and feature sparseness, and therefore LSSVM cannot perform sample selection or feature selection directly. Recent studies focus on sparse LSSVMS by performing sample selection or feature selection separately. In this paper, it is the first time to combine both the sample and feature selection in a uniform sparse primal and dual LSSVM model, called SPDLSSVM. SPDLSSVM solves an L1-norm based sparse primal and dual optimization problem to obtain the final classifier with sample and feature selection simultaneously. An alternating direction method of multipliers (ADMM) is proposed to solve the optimization problem of SPDLSSVM, and its convergence has been proved. In addition, an L1-norm based sparse primal LSSVM (SPLSSVM) and an L1-norm based sparse dual LSSVM (SDLSSVM) are also presented, where SPLSSVM achieves feature selection and SDLSSVM achieves sample selection. Experimental evaluation shows that the proposed SPDLSSVM yields better generalization ability via sample and feature selection, and at the same time, with faster testing speed.