Entropy-based Feature Selection Research Articles

Electrocardiogram morphological arrhythmia classification using fuzzy entropy-based feature selection and optimal classifier

Electrocardiogram (ECG) signal analysis has become significant in recent years as cardiac arrhythmia shares a major portion of all mortality worldwide. To detect these arrhythmias, computer-assisted algorithms play a pivotal role as beat-by-beat monitoring of holter ECG signals is required. In this paper, a morphological arrhythmia classification algorithm has been proposed to classify seven different ECG beats, namely Normal Beat (N), Left Bundle Branch Block Beat (L), Right Bundle Branch Block Beat (R), Atrial Premature Contraction Beat (A), Premature Ventricular Contraction Beat (V), Fusion of Normal and Ventricle Beat (F) and Pace Beat (P). A novel feature set of 25 attributes has been extracted from each ECG beat and ranked using the Fuzzy Entropy-based feature selection (FEBFS) technique. In addition, two distinct classifiers, support vector machine with radial basis function as the kernel (SVM-RBF) and weighted K-nearest neighbor (WKNN), are used to categorize ECG beats, and their performances are also evaluated after adjusting vital parameters. The performance of classifiers is compared for four different ECG beat segmentation approaches and further analyzed using three similarity measurement techniques and two fuzzy entropy methods while feature selection. The classifier results are also cross-validated using a 10-fold cross-validation scheme, and the MIT-BIH Arrhythmia Database has been used to validate the proposed work. After selecting 21 highly ranked features, WKNN achieves the best results with the nearest neighbor value K = 3 and cityblock distance metrics, with Average Sensitivity (Sen) = 94.89%, Positive Predictivity (Ppre) = 97.13%, Specificity (Spe) = 99.72%, F1 Score = 95.95%, and Overall Accuracy (Acc) = 99.15%. The novelty of this work relies on formulating a unique feature set, including proposed symbolic features, followed by the FEBFS technique making this algorithm efficient and reliable for morphological arrhythmia classification. The above results demonstrate that the proposed algorithm performs better than many existing state-of-the-art works.

Multiscale Fuzzy Entropy-Based Feature Selection

In practice, it is common that there will be the same decision results under different scale conditions. Therefore, knowledge representation based on a single scale feature framework is far from meeting the needs of practical applications. Based on this, multi-scale data has received extensive attention. Feature selection is an important application of fuzzy multigranularity data analysis model. The existing multi-scale fuzzy granulation-based feature selection methods remove redundant or irrelevant features by selecting the optimal scale. However, this will lose the information corresponding to the remaining scale fuzzy granules, which will affect the classification results or learning tasks. Inspired by this, multi-scale fuzzy entropy is defined to fuse the granule information at different scales, and applied to feature selection. First, the feature with maximum multi-scale fuzzy mutual information is first selected. Then, the most significant features are gradually selected by evaluation metric that simultaneously considers the redundancy, relevance and complementary. A multi-scale fuzzy entropy-based feature selection (MFEFS) algorithm by means of this evaluation index is further designed. Finally, the proposed method is compared with some state-of-the-art methods. The experimental results show that the proposed algorithm has higher reduction efficiency than the comparison algorithms.

A retrospective cohort analysis leveraging augmented intelligence to characterize long COVID in the electronic health record: A precision medicine framework.

Physical and psychological symptoms lasting months following an acute COVID-19 infection are now recognized as post-acute sequelae of COVID-19 (PASC). Accurate tools for identifying such patients could enhance screening capabilities for the recruitment for clinical trials, improve the reliability of disease estimates, and allow for more accurate downstream cohort analysis. In this retrospective cohort study, we analyzed the EHR of hospitalized COVID-19 patients across three healthcare systems to develop a pipeline for better identifying patients with persistent PASC symptoms (dyspnea, fatigue, or joint pain) after their SARS-CoV-2 infection. We implemented distributed representation learning powered by the Machine Learning for modeling Health Outcomes (MLHO) to identify novel EHR features that could suggest PASC symptoms outside of typical diagnosis codes. MLHO applies an entropy-based feature selection and boosting algorithms for representation mining. These improved definitions were then used for estimating PASC among hospitalized patients. 30,422 hospitalized patients were diagnosed with COVID-19 across three healthcare systems between March 13, 2020 and February 28, 2021. The mean age of the population was 62.3 years (SD, 21.0 years) and 15,124 (49.7%) were female. We implemented the distributed representation learning technique to augment PASC definitions. These definitions were found to have positive predictive values of 0.73, 0.74, and 0.91 for dyspnea, fatigue, and joint pain, respectively. We estimated that 25 percent (CI 95%: 6-48), 11 percent (CI 95%: 6-15), and 13 percent (CI 95%: 8-17) of hospitalized COVID-19 patients will have dyspnea, fatigue, and joint pain, respectively, 3 months or longer after a COVID-19 diagnosis. We present a validated framework for screening and identifying patients with PASC in the EHR and then use the tool to estimate its prevalence among hospitalized COVID-19 patients.

Selecting features by utilizing intuitionistic fuzzy Entropy method

Feature selection is the most significant pre-processing activity, which intends to reduce the data dimensionality for enhancing the machine learning process. The evaluation of feature selection must consider classification, performance, efficiency, stability, and many factors. Nowadays, uncertainty is commonly occurred in the feature selection process due to time limitations, imprecise information, and the subjectivity of human minds. Moreover, the theory of intuitionistic fuzzy set has been proven as an extremely valuable tool to tackle the uncertainty and ambiguity that arises in many practical situations. Thus, this study introduces a novel feature selection framework using intuitionistic fuzzy entropy. In this regard, new entropy for IFS is proposed first and then compared with some of the previously developed entropy measures. As entropy is a measure of uncertainty present in data (features), features with higher entropy values are filtered out, and the remaining features having lower entropy values have been used to classify the data. To verify the effectiveness of the proposed entropy-based feature selection, some experiments are done with ten standard benchmark datasets by employing a support vector machine, K-nearest neighbor, and Naïve Bias classifiers. The outcomes of the study validate that the proposed entropy-based filter feature selection is more feasible and impressive than existing filter-based feature selection methods.

An Efficient Automated Technique for Classification of Breast Cancer Using Deep Ensemble Model

Breast cancer is one of the leading cancers among women. It has the second-highest mortality rate in women after lung cancer. Timely detection, especially in the early stages, can help increase survival rates. However, manual diagnosis of breast cancer is a tedious and time-consuming process, and the accuracy of detection is reliant on the quality of the images and the radiologist’s experience. However, computer-aided medical diagnosis has recently shown promising results, leading to the need to develop an efficient system that can aid radiologists in diagnosing breast cancer in its early stages. The research presented in this paper is focused on the multi-class classification of breast cancer. The deep transfer learning approach has been utilized to train the deep learning models, and a pre-processing technique has been used to improve the quality of the ultrasound dataset. The proposed technique utilizes two deep learning models, MobileNetV2 and DenseNet201, for the composition of the deep ensemble model. Deep learning models are fine-tuned along with hyperparameter tuning to achieve better results. Subsequently, entropy-based feature selection is used. Breast cancer identification using the proposed classification approach was found to attain an accuracy of 97.04%, while the sensitivity and F1 score were 96.87% and 96.76%, respectively. The performance of the proposed model is very effective and outperforms other state-of-the-art techniques presented in the literature.

MonDiaL-CAD: Monkeypox diagnosis via selected hybrid CNNs unified with feature selection and ensemble learning.

Recently, monkeypox virus is slowly evolving and there are fears it will spread as COVID-19. Computer-aided diagnosis (CAD) based on deep learning approaches especially convolutional neural network (CNN) can assist in the rapid determination of reported incidents. The current CADs were mostly based on an individual CNN. Few CADs employed multiple CNNs but did not investigate which combination of CNNs has a greater impact on the performance. Furthermore, they relied on only spatial information of deep features to train their models. This study aims to construct a CAD tool named "Monkey-CAD" that can address the previous limitations and automatically diagnose monkeypox rapidly and accurately. Monkey-CAD extracts features from eight CNNs and then examines the best possible combination of deep features that influence classification. It employs discrete wavelet transform (DWT) to merge features which diminishes fused features' size and provides a time-frequency demonstration. These deep features' sizes are then further reduced via an entropy-based feature selection approach. These reduced fused features are finally used to deliver a better representation of the input features and feed three ensemble classifiers. Two freely accessible datasets called Monkeypox skin image (MSID) and Monkeypox skin lesion (MSLD) are employed in this study. Monkey-CAD could discriminate among cases with and without Monkeypox achieving an accuracy of 97.1% for MSID and 98.7% for MSLD datasets respectively. Such promising results demonstrate that the Monkey-CAD can be employed to assist health practitioners. They also verify that fusing deep features from selected CNNs can boost performance.

Estimation of human pose by tsallis entropy-based feature selection with ensemble machine learning model

Estimation of human pose by tsallis entropy-based feature selection with ensemble machine learning model

Modelling the non-linear development of Shenley Station blue cheese volatiles during ripening using untargeted volatile fingerprinting and self-organizing maps

Blue cheese flavour development derives from complex biochemical reactions that depend on numerous factors including milk source, culture/strain selection, processing, and ripening conditions. Understanding volatile compound development during blue cheese ripening will help reduce production costs and facilitate quality improvements. Volatile compounds contribute to the characteristic flavours of the cheeses but ripening time predictions based on chemical data have proven difficult. The present study employed untargeted fingerprinting combined with linear and non-linear chemometric approaches to identify key volatiles for the modelling of Shenley Station blue cheese ripening times. Self-organizing maps and entropy-based feature selection along with partial least squares regression and variable identification coefficients were used to parse the linear and non-linear development behaviours of volatiles. The blue cheese ripening times were accurately modelled by twenty-three discriminant volatiles. The present study demonstrated that volatile fingerprints can be used to effectively model blue cheese ripening times using a non-linear chemometric approach.

Effective Prediction of Software Defects using Random-tree Entropy based Feature Selection Framework

Effective Prediction of Software Defects using Random-tree Entropy based Feature Selection Framework

An automatic ECG arrhythmia diagnosis system using support vector machines optimised with GOA and entropy-based feature selection procedure

An automatic ECG arrhythmia diagnosis system using support vector machines optimised with GOA and entropy-based feature selection procedure

Terrorism And Fake News Detection

Fake news dissemination is a critical issue in today’s fast-changing network environment. The issues of online fake news have attained an increasing eminence in the diffusion of shaping news stories online. This paper deals with the categorical cyber terrorism threats on social media and preventive approach to minimize their issues. Misleading or unreliable information in form of videos, posts, articles, URLs are extensively disseminated through popular social media platforms such as Facebook, Twitter, etc. As a result, editors and journalists are in need of new tools that can help them to pace up the verification process for the content that has been originated from social media. existing classification models for fake news detection have not completely stopped the spread because of their inability to accurately classify news, thus leading to a high false alarm rate. This study proposed a model that can accurately identify and classify deceptive news articles content infused on social media by malicious users. The news content, social-context features and the respective classification of reported news was extracted from the PHEME dataset using entropy-based feature selection. The selected features were normalized using Min-Max Normalization techniques. The model was simulated and its performance was evaluated by benchmarking with an existing model using detection accuracy, sensitivity, and precision as metrics. The result of the evaluation showed a higher 17.25% detection accuracy, 15.78% sensitivity, but lesser 0.2% precision than the existing model, Thus, the proposed model detects more fake news instances accurately based on news content and social content perspectives. This indicates that the proposed classification model has a better detection rate, reduces the false alarm rate of news instances and thus detects fake news more accurately.

Saliency Detection in Weak Light Images via Optimal Feature Selection-Guided Seed Propagation

Salient object detection has a wide range of applications in computer vision tasks. Although tremendous progress has been made in recent decades, the weak light image still poses formidable challenges to current saliency models due to its low illumination and low signal-to-noise ratio properties. Traditional hand-crafted features inevitably encounter great difficulties in handling images with weak light backgrounds, while most of the high-level features are unfavorable to highlight visually salient objects in weak light images. In allusion to these problems, an optimal feature selection-guided saliency seed propagation model is proposed for salient object detection in weak light images. The main idea of this paper is to hierarchically refine the saliency map by learning the optimal saliency seeds in weak light images recursively. Particularly, multiscale superpixel segmentation and entropy-based optimal feature selection are first introduced to suppress the background interference. The initial saliency map is then obtained by the calculation of global contrast and spatial relationship. Moreover, local fitness and global fitness are used to optimize the prediction saliency map. Extensive experiments on six datasets show that our saliency model outperforms 20 state-of-the-art models in terms of popular evaluation criteria.

Incremental neighborhood entropy-based feature selection for mixed-type data under the variation of feature set

Incremental neighborhood entropy-based feature selection for mixed-type data under the variation of feature set

ECG based Decision Support System for Clinical Management using Machine Learning Techniques

Heart disease prediction system using ECG is to predict heart disease using ECG signals. Heart is the next major organ comparing to brain, which has more priority in human body. Heart disease diagnosis is a complex task which requires much experience and knowledge. The huge amount of data generated for prediction of heart disease is too complex and voluminous to be processed by traditional methods. By using traditional methods doctors took lot of time to diagnosis the disease. So, an entropy based feature selection technique is used with classification algorithms in order to reduce the search space. The proposed model was tested on the real time dataset of NRI Hospital medical data. Using this system it is easier to predict the disease. It will also helpful for the doctors to take quick decisions.

Segmentation and Classification of Stomach Abnormalities Using Deep Learning

An automated system is proposed for the detection and classification of GI abnormalities. The proposed method operates under two pipeline procedures: (a) segmentation of the bleeding infection region and (b) classification of GI abnormalities by deep learning. The first bleeding region is segmented using a hybrid approach. The threshold is applied to each channel extracted from the original RGB image. Later, all channels are merged through mutual information and pixel-based techniques. As a result, the image is segmented. Texture and deep learning features are extracted in the proposed classification task. The transfer learning (TL) approach is used for the extraction of deep features. The Local Binary Pattern (LBP) method is used for texture features. Later, an entropy-based feature selection approach is implemented to select the best features of both deep learning and texture vectors. The selected optimal features are combined with a serial-based technique and the resulting vector is fed to the Ensemble Learning Classifier. The experimental process is evaluated on the basis of two datasets: Private and KVASIR. The accuracy achieved is 99.8 per cent for the private data set and 86.4 percent for the KVASIR data set. It can be confirmed that the proposed method is effective in detecting and classifying GI abnormalities and exceeds other methods of comparison.

An automatic ECG arrhythmia diagnosis system using support vector machines optimised with GOA and entropy-based feature selection procedure

An automatic ECG arrhythmia diagnosis system using support vector machines optimised with GOA and entropy-based feature selection procedure

Characterization of blue cheese volatiles using fingerprinting, self-organizing maps, and entropy-based feature selection

Understanding which volatile compounds discriminate between products can be useful for quality, innovation or product authenticity purposes. As dataset size and dimensionality increase, linear chemometric techniques like partial least squares discriminant analysis and variable identification (PLS-DA-VID) may not identify the most discriminant compounds. This research compared the performance of self-organizing maps and entropy-based feature selection (SOM-EFS) and PLS-DA-VID to identify discriminant compounds in 17 blue cheese varieties. A total of 172 volatiles were detected using headspace solid phase microextraction, gas chromatography and mass spectrometry, including 1-nonene and 2,6-dimethylpyridine, which were newly identified in blue cheese. Despite SOM-EFS selecting only 14 volatiles compared to 78 for PLS-DA-VID, SOM-EFS proved more effectively discriminant and improved the median five-fold cross-validated prediction accuracy of the model to 0.94 compared to 0.82 for PLS-DA-VID. These findings introduce SOM-EFS as a powerful non-linear exploratory data analysis approach in the field of volatile analytical chemistry.

Information-theoretic feature selection with segmentation-based folded principal component analysis (PCA) for hyperspectral image classification

ABSTRACT Hyperspectral image (HSI) usually holds information of land cover classes as a set of many contiguous narrow spectral wavelength bands. For its efficient thematic mapping or classification, band (feature) reduction strategies through Feature Extraction (FE) and/or Feature Selection (FS) methods for finding the intrinsic bands’ information are typically applied. Principal Component Analysis (PCA) is a frequently employed unsupervised linear FE method whereas cumulative-variance accumulation is used for selecting top features from PCA data. However, PCA can fail to extract intrinsic structure of HSI due to global variance consideration and domination by visible and near infrared bands while cumulative-variance accumulation has no capability to exploit non-linear relationships among the transformed features produced by PCA-based FE methods. Consequently, we propose an information theoretic normalized Mutual Information (nMI)-based minimum Redundancy Maximum Relevance (mRMR) non-linear measure to select the intrinsic features from the transformed space of our previously proposed Segmented-Folded-PCA (Seg-Fol-PCA) and Spectrally Segmented-Folded-PCA (SSeg-Fol-PCA) FE methods. We extensively analyse the effectiveness of the proposed unsupervised FE and supervised FS combinations Seg-Fol-PCA-mRMR and SSeg-Fol-PCA-mRMR with that of PCA-based existing linear and non-linear state-of-the-art methods. In addition, cumulative variance-based top features pick-up strategy is considered with all FE methods and Renyi quadratic entropy-based FS is used with Kernel Entropy Component Analysis (Ker-ECA). The experimental results illustrate that SSeg-Fol-PCA-mRMR and Seg-Fol-PCA-mRMR obtain highest classification result e.g. 95.39% and 95.03% respectively for agricultural Indian Pines HSI, and 96.58% and 95.30% respectively for urban Washington DC Mall HSI while the classification accuracies using all original features of the HSIs are 70.28% and 91.90% respectively. Moreover, the proposed SSeg-Fol-PCA-mRMR and Seg-Fol-PCA-mRMR outperform all investigated combinations of FE and FS using the real HSI datasets.

Ensemble learning based on random super-reduct and resampling

Ensemble learning has been widely used for improving the performance of base classifiers. Diversity among base classifiers is considered as a key issue in ensemble learning. Recently, to promote the diversity of base classifiers, ensemble methods through multi-modal perturbation have been proposed. These methods simultaneously use two or more perturbation techniques when generating base classifiers. In this paper, from the perspective of multi-modal perturbation, we propose an ensemble approach (called ‘E $$\_$$ RSRR’) based on random super-reduct and resampling. To generate a set of accurate and diverse base classifiers, E $$\_$$ RSRR adopts a new multi-modal perturbation strategy. This strategy combines two perturbation techniques together, that is, resampling and random super-reduct. First, it perturbs the sample space via the resampling technique; Second, it perturbs the feature space via the random super-reduct technique, which is a combination of RSS (random subspace selection) technique and ADEFS (approximate decision entropy-based feature selection) method in rough sets. Experimental results show that E $$\_$$ RSRR can provide competitive solutions for ensemble learning.

Accelerating information entropy-based feature selection using rough set theory with classified nested equivalence classes

Feature selection effectively reduces the dimensionality of data. For feature selection, rough set theory offers a systematic theoretical framework based on consistency measures, of which information entropy is one of the most important significance measures of attributes. However, an information-entropy-based significance measure is computationally expensive and requires repeated calculations. Although many accelerating strategies have been proposed thus far, there remains a bottleneck when using an information-entropy-based feature selection algorithm to handle large-scale datasets with high dimensions. In this study, we introduce a classified nested equivalence class (CNEC)-based approach to calculate the information-entropy-based significance for feature selection using rough set theory. The proposed method extracts knowledge of the reducts of a decision table to reduce the universe and construct CNECs. By exploring the properties of different types of CNECs, we can not only accelerate both outer and inner significance calculation by discarding useless CNECs but also effectively decrease the number of inner significance calculations by using one type of CNECs. The use of CNECs is shown to significantly enhance three representative entropy-based feature selection algorithms that use rough set theory. The feature subset selected by the CNEC-based algorithms is the same as that selected by algorithms using the original definition of information entropies. Experiments conducted using 31 datasets from multiple sources, such as the UCI repository and KDD Cup competition, including large-scale and high-dimensional datasets, confirm the efficiency and effectiveness of the proposed method.