An approach of feature selection using graph-theoretic heuristic and hill climbing

Abstract

Search-based methods that use matrix- or vector-based representations of the dataset are commonly employed to solve the problem of feature selection. These methods are more generalized and easy to apply. Recently, a set of algorithms have started using graph-based representation of the dataset instead of the traditional representations. These methods require additional modelling as the dataset needs to be represented as a graph. However, graph-based methods help in visualizing inter-feature relationship based on which graph-theoretic principles can be applied to identify good-quality feature subsets. A combination of the graph-based representation with traditional search techniques has the potential to increase model performance as well as interpretability. As per literature study, there is hardly any method which combines these approaches. In this paper, we have proposed a feature selection algorithm, which represents the dataset as a graph and then uses maximal independent sets and minimal vertex covers to improve traditional hill climbing search. The proposed method produces statistically significant improvement over (i) hill climbing, (ii) standard search-based methods and (iii) pure graph-based methods.