An adaptive loss backward feature elimination method for class-imbalanced and mixed-type data in medical diagnosis

Abstract

Feature selection or elimination methods have been widely used to mine high-dimensional data from many fields such as biomedicine. However, classical feature selection methods usually focus on class-balanced and numerical data, and the classification performance is poor when processing class-imbalanced or mixed-type data. Moreover, existing studies hardly consider the co-occurrence of both class-imbalance and mixture of numerical and nominal features. In this paper, we propose an adaptive loss backward feature elimination method for class-imbalanced and mixed-type data (FEIM). In FEIM, a technique to convert and decompose nominal features is employed to process mixed-type data. Meanwhile, an adaptive loss function is proposed to make the classifier pay more attention to the minority class. In experiments, the effectiveness and stability of FEIM are synthetically evaluated against seven medical diagnostic datasets in terms of the measurements of F measurement and precision, and the results of feature selection are analyzed in detail. The experimental results show that, compared with three common methods (Recursive Feature Elimination SVM (RFE-SVM), Relief, and Holdout Algorithm for Backward Feature Elimination (HO-BFE)), FEIM can achieve a higher classification accuracy and is more stable when the parameters are changed. On the other hand, the feature selection results of FEIM are more meaningful by processing class-imbalanced and mixed-type data. Therefore, FEIM is a competitive and effective method to process feature elimination for class-imbalanced and mixed-type biomedical data for in medical diagnosis. The supporting information is available in the electronic supplementary material.