Explainable feature selection and ensemble classification via feature polarity

Abstract

Feature selection aims to choose the most relevant features from the dataset that can enhance the performance and efficiency of machine learning models. Although feature selection has been studied for many years, most existing methods focus on accuracy and efficiency while neglecting the interpretability of selected features. Therefore, inspired by the “Yin-Yang” philosophy, we introduce the concept of feature polarity for the first time and divide the features into positive and negative features. For example, by analyzing a patient's symptoms (features), we can obtain two sets of features to explain whether the patient has the flu. Positive features help us determine if the patient has the flu, while negative features can help us rule out the possibility of the flu. We introduce the PN (Positive and Negative) coefficient to measure the polarity of candidate features and develop a novel and explainable feature selection method based on feature polarity. Furthermore, we propose an ensemble classification framework that leverages both positive and negative features for each class to improve classification performance. Extensive experiments demonstrate the effectiveness of the PN coefficient compared to other information measurements. Moreover, our proposed classification framework performs excellently compared to some state-of-the-art feature selection methods.