Efficient feature-subset selection with probabilistic distance criteria

Abstract

This paper considers the problem of finding the best feature subset by exhaustive search, using probabilistic distance measures as criteria. Recursive expressions are derived for efficiently computing the commonly used probabilistic distance measures as a change in the criteria both when a feature is added to and when a feature is deleted from the current feature subset. A combinatorial algorithm is presented for generating all possible r-feature combinations from a given set of S features in ( S r ) steps with a change of a single feature at each step. These expressions can also be used for both forward and backward sequential feature selection.