Incremental feature selection by sample selection and feature-based accelerator

Abstract

Incremental feature selection is an efficient paradigm that updates an optimal feature subset from added-in data without forgetting the previously learned knowledge. Most existing studies of rough set-based incremental feature selection require scanning all added-in samples and all possible candidate features when determining a best feature. However, such a classical search strategy has to perform some redundant calculations, which increase the computing and memory space resources. To avoid the redundant calculations, we propose a novel incremental feature selection method using sample selection and feature-based accelerator. First, a feature selection framework based on discernibility score is proposed as basis for our incremental method. Second, sample selection scheme is proposed to eliminate useless samples from added-in data. This scheme ensures that only useful samples are considered in the incremental process. Third, feature-based accelerator is designed to incrementally select a best feature and simultaneously remove redundant candidate features. It is theoretically guaranteed redundant features removed earlier remain redundant and will not be reexamined during the rest of the process. Finally, our incremental feature selection algorithm is designed by a two-stage procedure including sample selection scheme and feature-based accelerator. The results of experiments validate the time efficiency of the proposed incremental algorithm, especially on datasets with numerous instances or high dimensions.