Comparison between Supervised and Unsupervised Feature Selection Methods

Abstract

The reduction of the feature set by selecting relevant features for the classification process is an important step within the image processing chain, but sometimes too little attention is paid to it. Such a reduction has many advantages. It can remove irrelevant and redundant data, improve recognition performance, reduce storage capacity requirements, computational time of calculations and also the complexity of the model. Within this paper supervised and unsupervised feature selection methods are compared with respect to the achievable recognition accuracy. Supervised Methods include information of the given classes in the selection, whereas unsupervised ones can be used for tasks without known class labels. Feature clustering is an unsupervised method. For this type of feature reduction, mainly hierarchical methods, but also k-means are used. Instead of this two clustering methods, the Expectation Maximization (EM) algorithm was used in this paper. The aim is to investigate whether this type of clustering algorithm can provide a proper feature vector using feature clustering. There is no feature reduction technique that provides equally best results for all datasets and classifiers. However, for all datasets, it was possible to reduce the feature set to a specific number of useful features without losses and often even with improvements in recognition performance.