mRMR Feature Selection Method Research Articles

Detection of Thymoma Disease Using mRMR Feature Selection and Transformer Models

Background: Thymoma is a tumor that originates in the thymus gland, a part of the human body located behind the breastbone. It is a malignant disease that is rare in children but more common in adults and usually does not spread outside the thymus. The exact cause of thymic disease is not known, but it is thought to be more common in people infected with the EBV virus at an early age. Various surgical methods are used in clinical settings to treat thymoma. Expert opinion is very important in the diagnosis of the disease. Recently, next-generation technologies have become increasingly important in disease detection. Today’s early detection systems already use transformer models that are open to technological advances. Methods: What makes this study different is the use of transformer models instead of traditional deep learning models. The data used in this study were obtained from patients undergoing treatment at Fırat University, Department of Thoracic Surgery. The dataset consisted of two types of classes: thymoma disease images and non-thymoma disease images. The proposed approach consists of preprocessing, model training, feature extraction, feature set fusion between models, efficient feature selection, and classification. In the preprocessing step, unnecessary regions of the images were cropped, and the region of interest (ROI) technique was applied. Four types of transformer models (Deit3, Maxvit, Swin, and ViT) were used for model training. As a result of the training of the models, the feature sets obtained from the best three models were merged between the models (Deit3 and Swin, Deit3 and ViT, Deit3 and ViT, Swin and ViT, and Deit3 and Swin and ViT). The combined feature set of the model (Deit3 and ViT) that gave the best performance with fewer features was analyzed using the mRMR feature selection method. The SVM method was used in the classification process. Results: With the mRMR feature selection method, 100% overall accuracy was achieved with feature sets containing fewer features. The cross-validation technique was used to verify the overall accuracy of the proposed approach and 99.22% overall accuracy was achieved in the analysis with this technique. Conclusions: These findings emphasize the added value of the proposed approach in the detection of thymoma.

Antiprotozoal peptide prediction using machine learning with effective feature selection techniques

BackgroundProtozoal pathogens pose a considerable threat, leading to notable mortality rates and the ongoing challenge of developing resistance to drugs. This situation underscores the urgent need for alternative therapeutic approaches. Antimicrobial peptides stand out as promising candidates for drug development. However, there is a lack of published research focusing on predicting antimicrobial peptides specifically targeting protozoal pathogens. In this study, we introduce a successful machine learning-based framework designed to predict potential antiprotozoal peptides effective against protozoal pathogens. ObjectiveThe primary objective of this study is to classify and predict antiprotozoal peptides using diverse negative datasets. MethodsA comprehensive literature review was conducted to gather experimentally validated antiprotozoal peptides, forming the positive dataset for our study. To construct a robust machine learning classifier, multiple negative datasets were incorporated, including (i) non-antimicrobial, (ii) antiviral, (iii) antibacterial, (iv) antifungal, and (v) antimicrobial peptides excluding those targeting protozoal pathogens. Various compositional features of the peptides were extracted using the pfeature algorithm. Two feature selection methods, SVC-L1 and mRMR, were employed to identify highly relevant features crucial for distinguishing between the positive and negative datasets. Additionally, five popular classifiers i.e. Decision Tree, Random Forest, Support Vector Machine, Logistic Regression, and XGBoost were used to build efficient decision models. ResultsXGBoost was the most effective in classifying antiprotozoal peptides from each negative dataset based on the features selected by the mRMR feature selection method. The proposed machine learning framework efficiently differentiate the antiprotozoal peptides from (i) non-antimicrobial (ii) antiviral (iii) antibacterial (iv) antifungal and (v) antimicrobial with accuracy of 97.27 %, 93.64 %, 86.36 %, 90.91 %, and 89.09 % respectively on the validation dataset. ConclusionThe models are incorporated in a user-friendly web server (www.soodlab.com/appred) to predict the antiprotozoal activity of given peptides.

An Intelligent Healthcare System for Automated Diabetes Diagnosis and Prediction using Machine Learning

The progression of biotechnological and health science fields has instigated a substantial proliferation of data, encompassing high-throughput genetic information and extensive clinical data derived from expansive electronic health record repositories. In the domain of biosciences, the indispensability of machine learning and data mining methodologies has surged, attaining heightened significance for the purpose of converting extant information into usable knowledge. Diabetes mellitus is a metabolic condition that affects human health. Extensive research on diabetes (diagnosis, therapy, etc.) has generated vast volumes of data. Manual diagnosis of diabetes has always been a time-consuming task. Therefore, automatic detection and diagnosis of diabetes using artificial intelligence and machine learning is gaining prominence. In our work, we have devised a novel architecture using machine learning for the automatic diagnosis of diabetes. We implemented our model using many algorithms and found that random forest is the most optimized and accurate one for classification purposes. It produced the highest accuracy of 98.07%, precision, recall, F1-score of 98%, and logarithmic loss of 0.03 using the mRMR feature selection method and 0.2 test split. It produced a recall of 97%, the precision of 97%, an F1-score of 97%, an accuracy of 96.79%, and a log loss of 0.11 on 0.3 test split with mRMR feature selection. Also, the proposed model has performed better than most state-of-the-art models.

Minimising redundancy, maximising relevance: HRV feature selection for stress classification

Heart rate variability serves as a valuable indicator and biomarker for stress detection and monitoring. Feature selection, which aims to identify relevant features from a large set of variables, is a crucial preprocessing step towards this. However, this task becomes challenging due to high dimensionality and the presence of irrelevant and redundant attributes. The Minimum Redundancy and Maximum Relevance (mRMR) feature selection method addresses this challenge by selecting relevant features while controlling redundancy. This paper presents extensions and evaluated versions of the mRMR feature selection methods for stress detection using Heart Rate Variability (HRV) measures. The proposed feature selection methods extend the traditional mRMR by replacing the Pearson correlation redundancy with non-linear feature redundancy measures capable of capturing more complex relationships between variables. An extensive empirical evaluation is conducted on the proposed mRMR extensions, comparing them with four other baseline feature selection methods using three publicly available datasets. The experimental results demonstrate the effectiveness of incorporating the non-linear feature redundancy measure into the feature selection process.

Machine learning based readmission and mortality prediction in heart failure patients

This study intends to predict in-hospital and 6-month mortality, as well as 30-day and 90-day hospital readmission, using Machine Learning (ML) approach via conventional features. A total of 737 patients remained after applying the exclusion criteria to 1101 heart failure patients. Thirty-four conventional features were collected for each patient. First, the data were divided into train and test cohorts with a 70–30% ratio. Then train data were normalized using the Z-score method, and its mean and standard deviation were applied to the test data. Subsequently, Boruta, RFE, and MRMR feature selection methods were utilized to select more important features in the training set. In the next step, eight ML approaches were used for modeling. Next, hyperparameters were optimized using tenfold cross-validation and grid search in the train dataset. All model development steps (normalization, feature selection, and hyperparameter optimization) were performed on a train set without touching the hold-out test set. Then, bootstrapping was done 1000 times on the hold-out test data. Finally, the obtained results were evaluated using four metrics: area under the ROC curve (AUC), accuracy (ACC), specificity (SPE), and sensitivity (SEN). The RFE-LR (AUC: 0.91, ACC: 0.84, SPE: 0.84, SEN: 0.83) and Boruta-LR (AUC: 0.90, ACC: 0.85, SPE: 0.85, SEN: 0.83) models generated the best results in terms of in-hospital mortality. In terms of 30-day rehospitalization, Boruta-SVM (AUC: 0.73, ACC: 0.81, SPE: 0.85, SEN: 0.50) and MRMR-LR (AUC: 0.71, ACC: 0.68, SPE: 0.69, SEN: 0.63) models performed the best. The best model for 3-month rehospitalization was MRMR-KNN (AUC: 0.60, ACC: 0.63, SPE: 0.66, SEN: 0.53) and regarding 6-month mortality, the MRMR-LR (AUC: 0.61, ACC: 0.63, SPE: 0.44, SEN: 0.66) and MRMR-NB (AUC: 0.59, ACC: 0.61, SPE: 0.48, SEN: 0.63) models outperformed the others. Reliable models were developed in 30-day rehospitalization and in-hospital mortality using conventional features and ML techniques. Such models can effectively personalize treatment, decision-making, and wiser budget allocation. Obtained results in 3-month rehospitalization and 6-month mortality endpoints were not astonishing and further experiments with additional information are needed to fetch promising results in these endpoints.

IIL13Pred: improved prediction of IL-13 inducing peptides using popular machine learning classifiers

BackgroundInflammatory mediators play havoc in several diseases including the novel Coronavirus disease 2019 (COVID-19) and generally correlate with the severity of the disease. Interleukin-13 (IL-13), is a pleiotropic cytokine that is known to be associated with airway inflammation in asthma and reactive airway diseases, in neoplastic and autoimmune diseases. Interestingly, the recent association of IL-13 with COVID-19 severity has sparked interest in this cytokine. Therefore characterization of new molecules which can regulate IL-13 induction might lead to novel therapeutics.ResultsHere, we present an improved prediction of IL-13-inducing peptides. The positive and negative datasets were obtained from a recent study (IL13Pred) and the Pfeature algorithm was used to compute features for the peptides. As compared to the state-of-the-art which used the regularization based feature selection technique (linear support vector classifier with the L1 penalty), we used a multivariate feature selection technique (minimum redundancy maximum relevance) to obtain non-redundant and highly relevant features. In the proposed study (improved IL-13 prediction (iIL13Pred)), the use of the mRMR feature selection method is instrumental in choosing the most discriminatory features of IL-13-inducing peptides with improved performance. We investigated seven common machine learning classifiers including Decision Tree, Gaussian Naïve Bayes, k-Nearest Neighbour, Logistic Regression, Support Vector Machine, Random Forest, and extreme gradient boosting to efficiently classify IL-13-inducing peptides. We report improved AUC, and MCC scores of 0.83 and 0.33 on validation data as compared to the current method.ConclusionsExtensive benchmarking experiments suggest that the proposed method (iIL13Pred) could provide improved performance metrics in terms of sensitivity, specificity, accuracy, the area under the curve - receiver operating characteristics (AUCROC) and Matthews correlation coefficient (MCC) than the existing state-of-the-art approach (IL13Pred) on the validation dataset and an external dataset comprising of experimentally validated IL-13-inducing peptides. Additionally, the experiments were performed with an increased number of experimentally validated training datasets to obtain a more robust model. A user-friendly web server (www.soodlab.com/iil13pred) is also designed to facilitate rapid screening of IL-13-inducing peptides.

Identification of COVID-19-Specific Immune Markers Using a Machine Learning Method.

Notably, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has a tight relationship with the immune system. Human resistance to COVID-19 infection comprises two stages. The first stage is immune defense, while the second stage is extensive inflammation. This process is further divided into innate and adaptive immunity during the immune defense phase. These two stages involve various immune cells, including CD4+ T cells, CD8+ T cells, monocytes, dendritic cells, B cells, and natural killer cells. Various immune cells are involved and make up the complex and unique immune system response to COVID-19, providing characteristics that set it apart from other respiratory infectious diseases. In the present study, we identified cell markers for differentiating COVID-19 from common inflammatory responses, non-COVID-19 severe respiratory diseases, and healthy populations based on single-cell profiling of the gene expression of six immune cell types by using Boruta and mRMR feature selection methods. Some features such as IFI44L in B cells, S100A8 in monocytes, and NCR2 in natural killer cells are involved in the innate immune response of COVID-19. Other features such as ZFP36L2 in CD4+ T cells can regulate the inflammatory process of COVID-19. Subsequently, the IFS method was used to determine the best feature subsets and classifiers in the six immune cell types for two classification algorithms. Furthermore, we established the quantitative rules used to distinguish the disease status. The results of this study can provide theoretical support for a more in-depth investigation of COVID-19 pathogenesis and intervention strategies.

Classification of the weather images with the proposed hybrid model using deep learning, SVM classifier, and mRMR feature selection methods

As in many fields, the use of artificial intelligence methods in the classification of weather images will be very useful. In this study, a data set consisting of five classes such as cloudy, foggy, rainy, shine, and sunrise was used. A hybrid model has been developed to classify the images in the dataset. First of all, the features of the images in the dataset are obtained by using MobilenetV2, Densenet201, and Efficientnetb0 architectures, which are the most popular Convolutional Neural Network (CNN) architectures. These features are combined and optimized so that these optimized features are classified in the Support Vector Machine (SVM) classifier, one of the most popular classifier methods in machine learning. As a result, the developed hybrid model has outperformed the existing pre-trained architectures in the study. In addition, it has been proven that classification by concatenating the features obtained with CNN architectures is a successful method.

An Improved Power System Transient Stability Prediction Model Based on mRMR Feature Selection and WTA Ensemble Learning

Fast online transient stability assessment (TSA) is very important to maintain the stable operation of power systems. However, the existing transient stability assessment methods suffer the drawbacks of unsatisfactory prediction accuracy, difficult applicability, or a heavy computational burden. In light of this, an improved high accuracy power system transient stability prediction model is proposed, based on min-redundancy and max-relevance (mRMR) feature selection and winner take all (WTA) ensemble learning. Firstly, the contributions of four different series of raw sampled data from all of the three-time stages, namely the pre-fault, during-fault and post-fault, to transient stability are compared. The new feature of generator electromagnetic power is introduced and compared with three conventional types of input features, through a support vector machine (SVM) classifier. Furthermore, the two types of most contributive input features are obtained by the mRMR feature selection method. Finally, the prediction results of the electromagnetic power of generators and the voltage amplitude of buses are combined using the WTA ensemble learning method, and an improved transient stability prediction model with higher accuracy for unstable samples is obtained, whose overall prediction accuracy would not decrease either. The real-time data collected by wide area monitoring systems (WAMS) can be fed into this model for fast online transient stability prediction; the results can also provide a basis for the future emergency control decision-making of power systems.

MUSE: Minimum Uncertainty and Sample Elimination Based Binary Feature Selection

This paper presents a novel incremental feature selection method based on minimum uncertainty and feature sample elimination (referred as MUSE). Feature selection is an important step in machine learning. In an incremental feature selection approach, past approaches have attempted to increase class relevance while simultaneously minimizing redundancy with previously selected features. One example of such an approach is the feature selection method of minimum Redundancy Maximum Relevance (mRMR). The proposed approach differs from prior mRMR approach in how the redundancy of the current feature with previously selected features is reduced. In the proposed approach, the feature samples are divided into a pre-specified number of bins; this step is referred to as feature quantization. A novel uncertainty score for each feature is computed by summing the conditional entropies of the bins, and the feature with the lowest uncertainty score is selected. For each bin, its impurity is computed by taking the minimum of the probability of Class 1 and of Class 2. The feature samples corresponding to the bins with impurities below a threshold are discarded and are not used for selection of the subsequent features. The significance of the MUSE feature selection method is demonstrated using the two datasets: arrhythmia and hand digit recognition (Gisette), and datasets for seizure prediction from five dogs and two humans. It is shown that the proposed method outperforms the prior mRMR feature selection method for most cases. For the arrhythmia dataset, the proposed method achieves 30 percent higher sensitivity at the expense of 7 percent loss of specificity. For the Gisette dataset, the proposed method achieves 15 percent higher accuracy for Class 2, at the expense of 3 percent lower accuracy for Class 1. With respect to seizure prediction among 5 dogs and 2 humans, the proposed method achieves higher area-under-curve (AUC) for all subjects.

Classification of breast mass in mammography using anisotropic diffusion filter by selecting and aggregating morphological and textural features

In this paper, we describe a novel mammogram classification framework for classifying breast tissues as normal, benign or malignant. First, we establish a set of symbolic rules for mammograms pre-classification from the particularities of a tissue (the background, class and severity of abnormality) taking into account the different forms of breast according to their abnormality. A decision tree reflecting a preliminary attribution from the region of interest (ROI) of a mammogram represents this pre-classification. The images obtained by this technique are processed with an anisotropic diffusion filter to reduce the noise and preserve edges. Then, a feature matrix is generated using: on the one hand, textural features from both a Gray Level Co-occurrence Matrix (GLCM) and a Gray Level Run Lengths Matrix (GLRLM) applied to all the filtered regions of interest (ROI) of a mammogram; and on the other, after mass region is segmented out, the features extracted from the mass boundary and the margin region between the mass and background for classification. To derive the relevant features from the feature matrix, we resort to RELIEF and MRMR feature selection method separately. The key point of our proposal is the modeling of Back Propagation Neural Network (BPNN) by a network of queues for representing the textural and morphological properties of the masses. The relevant and ordered features are injected in BPNN classifier using queuing network. The standard database MIAS is used for validation of the proposed scheme. In any case, we observed that the opening of closed neural network improves the performance. However, the MRMR feature selection method outperforms RELIEF one the point of view of precision and area under curve of receiver operating characteristic. These measures are respectively 98.1% and0.9650 for normal vs. abnormal classification, whereas, they are respectively 95.2% and 0.9200 for benign vs. malignant classification.

Abstract 3040: Radiomics discriminates pseudo-progression from true progression in glioblastoma patients: A large-scale multi-institutional study

Abstract BACKGROUND: Treatment-related imaging changes are often difficult to distinguish from true tumor progression. Treatment-related changes or pseudoprogression (PsP) subsequently subside or stabilize without any further treatment, whereas progressive tumor requires a more aggressive approach in patient management. Pseudoprogression can mimic true progression radiographically and may potentially alter the physician's judgment about the recurrent disease. Hence, it can predispose a patient to overtreatment or be categorized as a non-responder and exclude him from the clinical trials. This study aims at assessing the potential of radiomics to discriminate PsP from progressive disease (PD) in glioblastoma (GBM) patients. METHODS: We retrospectively evaluated 304 GBM patients with new or increased enhancement on conventional MRI after treatment, of which it was uncertain for PsP versus PD. 149 patients had the histopathological evidence of PD and 27 of PsP. Remaining 128 patients were categorized into PD and PsP based on RANO criteria performed by a board-certified radiologist. Volumetrics using 3D slicer 4.3.1 and radiomics texture analysis were performed of the enhancing lesion(s) in question. RESULTS: Using the MRMR feature selection method, we identified 100 significant features that were used to build a SVM model. Five texture features (E, CS, SA, MP, CP) were found to be most predictive of pseudoprogression. On Leave One Out Cross-Validation (LOOCV), sensitivity, specificity and accuracy were 97%, 72%, and 90%, respectively. Using 70% of the patient data for training and 30% for validation, an AUC of 94% was achieved, with the sensitivity of 97% and specificity of 75%. CONCLUSION: 3D radiomic texture features of conventional MRI successfully discriminated pseudoprogression from true progression in a large cohort of GBM patients. Citation Format: Srishti Abrol, Aikaterini Kotrotsou, Ahmed Hassan, Nabil Elshafeey, Tagwa Idris, Naveen Manohar, Anand Agarwal, Islam Hassan, Kamel Salek, Nikdokht Farid, Carrie McDonald, Shiao-Pei Weathers, Naeim Bahrami, Samuel Bergamaschi, Ahmed Elakkad, Kristin Alfaro-Munoz, Fanny Moron, Jason Huse, Jeffrey Weinberg, Sherise Ferguson, Evangelos Kogias, Amy Heimberger, Raymond Sawaya, Ashok Kumar, John de Groot, Meng Law, Pascal Zinn, Rivka R. Colen. Radiomics discriminates pseudo-progression from true progression in glioblastoma patients: A large-scale multi-institutional study [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2018; 2018 Apr 14-18; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2018;78(13 Suppl):Abstract nr 3040.

Radiomic analysis of pseudo-progression compared to true progression in glioblastoma patients: A large-scale multi-institutional study.

2015 Background: Treatment-related imaging changes are often difficult to distinguish from true tumor progression. Treatment-related changes or pseudoprogression (PsP) subsequently subside or stabilize without any further treatment, whereas progressive tumor requires a more aggressive approach in patient management. Pseudoprogression can mimic true progression radiographically and may potentially alter the physician’s judgment about the residual disease. Hence, it can predispose a patient to overtreatment or be categorized as a non-responder and exclude him from the clinical trials. This study aims at assessing the potential of radiomics to discriminate PsP from progressive disease (PD) in glioblastoma (GBM) patients. Methods: We retrospectively evaluated 304 GBM patients with new or increased enhancement on conventional MRI after treatment, of which it was uncertain for PsP versus PD. 149 patients had the histopathological evidence of PD and 27 of PsP. Remaining 128 patients were categorized into PD and PsP based on RANO criteria performed by a board-certified radiologist. Volumetrics using 3D slicer 4.3.1 and radiomics texture analysis were performed of the enhancing lesion(s) in question. Results: Using the MRMR feature selection method, we identified 100 significant features that were used to build a SVM model. Five texture features (E, CS, SA, MP, CP) were found to be most predictive of pseudoprogression. On Leave One Out Cross-Validation (LOOCV), sensitivity, specificity and accuracy were 97%, 72%, and 90%, respectively. Conclusions: 3D radiomic texture features of conventional MRI successfully discriminated pseudoprogression from true progression in a large cohort of GBM patients.

Prediction of Allergenic Proteins by Means of the Concept of Chou's Pseudo Amino Acid Composition and a Machine Learning Approach

Because of the importance of proteins in inducing allergenic reactions, the ability of predicting their potential allergenicity has become an important issue. Bioinformatics presents valuable tools for analyzing allergens and these complementary approaches can help traditional techniques to study allergens. This work proposes a computational method for predicting the allergenic proteins. The prediction was performed using pseudo-amino acid composition (PseAAC) and Support Vector Machines (SVMs). The predictor efficiency was evaluated by fivefold cross-validation. The overall prediction accuracies and Matthew's correlation coefficient (MCC) obtained by this method were 91.19% and 0.82, respectively. Furthermore, the minimum Redundancy and Maximum Relevance (mRMR) feature selection method was utilized for measuring the effect and power of each feature. Interestingly, in our study all six characters (hydrophobicity, hydrophilicity, side chain mass, pK1, pK2 and pI) are present among the 10 higher ranked features obtained from the mRMR feature selection method.

Prediction of Allergenic Proteins by Means of the Concept of Chou’s Pseudo Amino Acid Composition and a Machine Learning Approach

Because of the importance of proteins in inducing allergenic reactions, the ability of predicting their potential allergenicity has become an important issue. Bioinformatics presents valuable tools for analyzing allergens and these complementary approaches can help traditional techniques to study allergens. This work proposes a computational method for predicting the allergenic proteins. The prediction was performed using pseudo-amino acid composition (PseAAC) and Support Vector Machines (SVMs). The predictor efficiency was evaluated by fivefold cross-validation. The overall prediction accuracies and Matthew’s correlation coefficient (MCC) obtained by this method were 91.19% and 0.82, respectively. Furthermore, the minimum Redundancy and Maximum Relevance (mRMR) feature selection method was utilized for measuring the effect and power of each feature. Interestingly, in our study all six characters (hydrophobicity, hydrophilicity, side chain mass, pK1, pK2 and pI) are present among the 10 higher ranked features obtained from the mRMR feature selection method. Keywords: Allergenic proteins, Bioinformatics, Chou’s Pseudo amino acid composition, Support Vector Machines

Prediction of GABA A receptor proteins using the concept of Chou's pseudo-amino acid composition and support vector machine

The amino acid gamma-aminobutyric-acid receptors (GABA ARs) belong to the ligand-gated ion channels (LGICs) superfamily. GABA ARs are highly diverse in the central nervous system. These channels play a key role in regulating behavior. As a result, the prediction of GABA ARs from the amino acid sequence would be helpful for research on these receptors. We have developed a method to predict these proteins using the features obtained from Chou's pseudo-amino acid composition concept and support vector machine as a powerful machine learning approach. The predictor efficiency was assessed by five-fold cross-validation. This method achieved an overall accuracy and Matthew's correlation coefficient (MCC) of 94.12% and 0.88, respectively. Furthermore, to evaluate the effect and power of each feature, the minimum Redundancy and Maximum Relevance (mRMR) feature selection method was implemented. An interesting finding in this study is the presence of all six characters (hydrophobicity, hydrophilicity, side chain mass, pK1, pK2 and pI) or combination of the characters among the 5 higher ranked features (pk2 and pI, hydrophobicity and mass, pk1, hydrophilicity and mass) obtained from the mRMR feature selection method. The results show a biologically justifiable ranked attributes of pk2 and pI; hydrophobicity, hydrophilicity and mass; mass and pk1; pk2 and mass. Based on our results, using the concept of Chou's pseudo-amino acid composition and support vector machine is an effective approach for the prediction of GABA ARs.