Ensemble learning via feature selection and multiple transformed subsets: Application to image classification

Abstract

In the machine learning field, especially in classification tasks, the model’s design and construction are very important. Constructing the model via a limited set of features may sometimes bound the classification performance and lead to non-optimal performances that some algorithms can provide. To this end, Ensemble learning methods were proposed in the literature. These methods’ main goal is to learn a set of models that provide features or predictions whose joint use could lead to a performance better than that obtained by the single model. In this paper, we propose three variants of a new efficient ensemble learning approach that was able to enhance the classification performance of a linear discriminant embedding method. As a case study we consider the efficient “Inter-class sparsity discriminative least square regression” method. We seek the estimation of an enhanced data representation. Instead of deploying multiple classifiers on top of the transformed features, we target the estimation of multiple extracted feature subsets obtained by multiple learned linear embeddings. These are associated with subsets of ranked original features. Multiple feature subsets were used for estimating the transformations. The derived extracted feature subsets were concatenated to form a single data representation vector that is used in the classification process. Many factors were studied and investigated in this paper including (Parameter combinations, number of models, different training percentages, feature selection methods combinations, etc.). Our proposed approach has been benchmarked on different image datasets of various sizes and types (faces, objects and scenes). The proposed scheme achieved competitive performance on four face image datasets (Extended Yale B, LFW-a, Gorgia and FEI) as well as on the COIL20 object dataset and the Outdoor Scene dataset. We measured the performance of our proposed schemes in comparison to (the single model ICS_DLSR, RDA_GD, RSLDA, PCE, LDE, LDA, SVM as well as the KNN algorithm) The conducted experiments showed that the proposed approach can enhance the classification performance in an efficient manner compared to the single-model based learning and was able to outperform its competing methods.