A Mixed Integer Linear Programming Support Vector Machine for Cost-Effective Group Feature Selection: Branch-Cut-and-Price Approach

Abstract

Recently, cost-based feature selection has received significant attention due to its great ability to achieve promising prediction accuracy at a minimum feature acquisition cost. To further improve its predictive and economic performances, this research proposes a cost-effective 1-norm support vector machine with group feature selection as GFS-CESVM1. Its robust counterpart model, GFS-RCESVM1, is also introduced to address the cost uncertainty of features and feature groups because cost variation commonly exists in real-world problems. The proposed models are formulated as Mixed Integer Linear Programming (MILP). To efficiently solve the proposed SVM MILP models, we develop a Branch-Cut-and-Price (BCP) algorithm that considers only a limited number of variables and/or constraints, which thereby leads to rapid convergence to an optimal solution. Various experimental results on benchmark and synthetic datasets demonstrate that GFS-CESVM1 can achieve competitive outcomes by considering not only individual feature evaluation but also group structural information among features. The GFS-RCESVM1 can identify the subset of features that is immune to cost uncertainty and therefore provide feasible and optimal solutions. Furthermore, our BCP algorithm can dominantly outperform the ordinary BB algorithm for finding better objective value and integrality gap within a short period of time.