Feature Selection for Multiclass Problems Based on Information Weights

Abstract

Before a pattern classifier can be properly designed, it is necessary to consider the feature extraction and data reduction problems. It is evident that the number of features needed to successfully perform a given recognition task depends on the discriminatory qualities of the chosen feature. We propose a new hybrid approach addressing feature selection based on information weights which allows feature categorization on the basis of specified classification task. The purpose is to efficiently achieve high degree of dimensionality reduction and enhance or maintain predictive accuracy with selected features. The novelty is to combine the competitiveness of the filter approach which makes it undependable from the nature of the pattern classifier and embed the algorithm within the pattern classifier structure in order to increase the accuracy of the learning phase as wrapper algorithms do. The algorithm is generalized for multiclass implementation.