Hybrid search of feature subsets

Abstract

Feature selection is a search problem for an “optimal” subset of features. The class separability is normally used as one of the basic feature selection criteria. Instead of maximizing the class separability as in the literature, this work adopts a criterion aiming to maintain the discriminating power of the data. After examining the pros and cons of two existing algorithms for feature selection, we propose a hybrid algorithm of probabilistic and complete search that can take advantage of both algorithms. It begins by running LVF (probabilistic search) to reduce the number of features; then it runs “Automatic Branch & Bound (ABB)” (complete search). By imposing a limit on the amount of time this algorithm can run, we obtain an approximation algorithm. The empirical study suggests that dividing the time equally between the two phases yields nearly the best performance, and that the hybrid search algorithm substantially outperforms earlier methods in general.