mRMR Feature Selection Research Articles

Predictive Model for O3 in Shanghai Based on the KZ Filtering Technique and LSTM

In this study, a Kolmogorov-Zurbenko （KZ） filter was proposed to decompose the original ozone （O3） sequence to improve the accuracy of ozone long-term series prediction and select relevant meteorological features. Furthermore, the enhanced maximal minimal redundancy （mRMR） feature selection technique was combined with the support vector regression （SVR） approach to select the most illuminating meteorological features. Subsequently, from May to August 2023, during high ozone concentration periods, a long short-term memory network （LSTM） was utilized to assess and predict high ozone concentration periods at the monitoring stations of Jingan （urban area）, Pudong-Chuansha （suburban area）, and Dianshan Lake （suburban area） in Shanghai. The results showed that pressure, temperature, humidity, boundary layer height, and wind direction were the best combinations of O3 baseline and short-term components, as chosen by feature screening. The R2 values for Jingan Station, Pudong-Chuansha Station, and Dianshan Lake Station were 0.86, 0.83, and 0.85, respectively. The RMSE values were 18.26, 18.74, and 20.02 μg·m-3, respectively. These findings suggest that decomposing the original O3 sequence improved the prediction accuracy of ozone concentrations. Additionally, as indicated by the R2 and RMSE values found for every monitoring station, feature screening preserved the model's predictive performance.

Optimum feature selection for classification of PD signals produced by multiple insulation defects in electric motors

Partial discharges (PD) are initiated in electrical equipment during various points of the equipment’s lifecycle. The intensity of PD defects rises continuously with time, which can lead to insulation degradation and reduced operational life of the electrical equipment. The optimum feature selection of PD signals captured, from different insulation defects, can enhance the classification accuracy of PD defects and facilitate better visualization of PD parameters for electric motor (EM) insulation monitoring and diagnostics. This paper presents a hybrid approach, based on Maximize Relevancy and Minimize Redundancy (mRMR) and random forest (RF), for the optimum feature selection and classification of PD signals in EMs containing multiple defects. For this purpose, four PD defects are developed in the EMs insulation under laboratory conditions, and 800 PD signals are acquired using a conventional IEC-60,270 experimental platform. The severity of these defects is determined and investigated based on PD characteristic parameters. Several features of both PD sweep signals and conventional PD pulses are extracted. Consequently, the mRMR feature selection technique is implemented to select the significant features of the detected PD signals. To establish the plausibility of this technique, several other feature selection algorithms, including RefliefF, Gini Index (GI), and Information Gain (IG), are introduced for the same datasets. The performance of all these feature selection algorithms is validated using three commonly used classification techniques such as RF, support vector machines (SVM), and k-nearest neighbors (k-NN). In summary, the results show that the combination of mRMR and RF proves to be the most effective feature selection algorithm for the classification of insulation defects in EMs, achieving an accuracy of 99.875%. This accuracy is significantly better than other feature selection and classification techniques and indicates its potential for application to other power system components.

Detection of Thymoma Disease Using mRMR Feature Selection and Transformer Models

Background: Thymoma is a tumor that originates in the thymus gland, a part of the human body located behind the breastbone. It is a malignant disease that is rare in children but more common in adults and usually does not spread outside the thymus. The exact cause of thymic disease is not known, but it is thought to be more common in people infected with the EBV virus at an early age. Various surgical methods are used in clinical settings to treat thymoma. Expert opinion is very important in the diagnosis of the disease. Recently, next-generation technologies have become increasingly important in disease detection. Today’s early detection systems already use transformer models that are open to technological advances. Methods: What makes this study different is the use of transformer models instead of traditional deep learning models. The data used in this study were obtained from patients undergoing treatment at Fırat University, Department of Thoracic Surgery. The dataset consisted of two types of classes: thymoma disease images and non-thymoma disease images. The proposed approach consists of preprocessing, model training, feature extraction, feature set fusion between models, efficient feature selection, and classification. In the preprocessing step, unnecessary regions of the images were cropped, and the region of interest (ROI) technique was applied. Four types of transformer models (Deit3, Maxvit, Swin, and ViT) were used for model training. As a result of the training of the models, the feature sets obtained from the best three models were merged between the models (Deit3 and Swin, Deit3 and ViT, Deit3 and ViT, Swin and ViT, and Deit3 and Swin and ViT). The combined feature set of the model (Deit3 and ViT) that gave the best performance with fewer features was analyzed using the mRMR feature selection method. The SVM method was used in the classification process. Results: With the mRMR feature selection method, 100% overall accuracy was achieved with feature sets containing fewer features. The cross-validation technique was used to verify the overall accuracy of the proposed approach and 99.22% overall accuracy was achieved in the analysis with this technique. Conclusions: These findings emphasize the added value of the proposed approach in the detection of thymoma.

A machine learning model based on CHAT-23 for early screening of autism in Chinese children.

Autism spectrum disorder (ASD) is a neurodevelopmental condition that significantly impacts the mental, emotional, and social development of children. Early screening for ASD typically involves the use of a series of questionnaires. With answers to these questionnaires, healthcare professionals can identify whether a child is at risk for developing ASD and refer them for further evaluation and diagnosis. CHAT-23 is an effective and widely used screening test in China for the early screening of ASD, which contains 23 different kinds of questions. We have collected clinical data from Wuxi, China. All the questions of CHAT-23 are regarded as different kinds of features for building machine learning models. We introduce machine learning methods into ASD screening, using the Max-Relevance and Min-Redundancy (mRMR) feature selection method to analyze the most important questions among all 23 from the collected CHAT-23 questionnaires. Seven mainstream supervised machine learning models were built and experiments were conducted. Among the seven supervised machine learning models evaluated, the best-performing model achieved a sensitivity of 0.909 and a specificity of 0.922 when the number of features was reduced to 9. This demonstrates the model's ability to accurately identify children for ASD with high precision, even with a more concise set of features. Our study focuses on the health of Chinese children, introducing machine learning methods to provide more accurate and effective early screening tests for autism. This approach not only enhances the early detection of ASD but also helps in refining the CHAT-23 questionnaire by identifying the most relevant questions for the diagnosis process.

Kruskal Wallis and mRMR Feature Selection based Online Signature Verification System using Multiple SVM and KNN

Signature verification is a very important research area. Signature has been widely accepted as a person authentication method for centuries. It is mostly used in financial transactions, document authentication and agreements. It is more susceptible to being forged than any other biometrics. Online signature verification is used in real-time applications like e-commerce, online resource access, online financial transactions, physical access into a restricted area and many more. In order to achieve high efficiency in online signature verification systems, feature extraction and feature selection play a significant role. A suitable signature verification system is needed to prevent forgery and accept the genuine signer. We have extracted 30 global features from all 40 signers for verification. Here, k fold cross-validation technique is used to enhance the model's performance on unseen data. User-specific feature selection and ranking are done using Kruskal Wallis and Minimum Redundancy Maximum Relevance (mRMR) algorithm to hunt which performs better in our case. Kruskal-Wallis method tests if two or more classes have an equal median and gives the value of P based on which discriminative features are selected, whereas the mRMR algorithm ranks the whole feature set according to its importance. It evaluates the relevance of a feature and penalizes redundancy. Finally, multiple SVM and KNN classifiers are trained and tested with various selected features using Kruskal Wallis and mRMR to determine which combination performs best for the online signature verification system. Our model is trained, validated and tested on the SVC 2004 Task 1 database, which consists of skill forgery signatures. Here, one to one verification is done using each user's genuine and skill forgery signatures, which is the hardest to detect. Best average testing accuracy achieved in our case is 90.25% using Weighted KNN and Kruskal Wallis selected 15 features.

A robust framework for Alzheimer’s disease detection and staging: incorporating multi-feature integration, MRMR feature selection, and Random Forest classification

A robust framework for Alzheimer’s disease detection and staging: incorporating multi-feature integration, MRMR feature selection, and Random Forest classification

Evaluating Feature Selection Algorithms for Machine Learning-Based Musical Instrument Identification in Monophonic Recordings

Musical instrument identification (MII) research has been studied as a subfield of the Music Information Retrieval (MIR) field. Conventional MII models are developed based on hierarchical models representing musical instrument families. However, for MII models to be used in the field of music production, they should be developed based on the arrangement-based functions of instruments in musical styles rather than these hierarchical models. This study investigates how the performance of machine learning based classification algorithms for Guitar, Bass guitar and Drum classes changes with different feature selection algorithms, considering a popular music production scenario. To determine the effect of feature statistics on model performance, Minimum Redundancy Maximum Relevance (mRMR), Chi-sqaure (Chi2), ReliefF, Analysis of Variance (ANOVA) and Kruskal Wallis feature selection algorithms were used. In the end, the neural network algorithm with wide hyperparameters (WNN) achieved the best classification accuracy (91.4%) when using the first 20 statistics suggested by the mRMR and ReliefF feature selection algorithms.

Leukemia Diagnosis using Machine Learning Classifiers based on MRMR Feature Selection

Leukemia Diagnosis using Machine Learning Classifiers based on MRMR Feature Selection

Evaluation of Machine Learning Classification Models for False-Positive Reduction in Prostate Cancer Detection Using MRI Data.

In this work, several machine learning (ML) algorithms, both classical ML and modern deep learning, were investigated for their ability to improve the performance of a pipeline for the segmentation and classification of prostate lesions using MRI data. The algorithms were used to perform a binary classification of benign and malignant tissue visible in MRI sequences. The model choices include support vector machines (SVMs), random decision forests (RDFs), and multi-layer perceptrons (MLPs), along with radiomic features that are reduced by applying PCA or mRMR feature selection. Modern CNN-based architectures, such as ConvNeXt, ConvNet, and ResNet, were also evaluated in various setups, including transfer learning. To optimize the performance, different approaches were compared and applied to whole images, as well as gland, peripheral zone (PZ), and lesion segmentations. The contribution of this study is an investigation of several ML approaches regarding their performance in prostate cancer (PCa) diagnosis algorithms. This work delivers insights into the applicability of different approaches for this context based on an exhaustive examination. The outcome is a recommendation or preference for which machine learning model or family of models is best suited to optimize an existing pipeline when the model is applied as an upstream filter.

Antiprotozoal peptide prediction using machine learning with effective feature selection techniques

BackgroundProtozoal pathogens pose a considerable threat, leading to notable mortality rates and the ongoing challenge of developing resistance to drugs. This situation underscores the urgent need for alternative therapeutic approaches. Antimicrobial peptides stand out as promising candidates for drug development. However, there is a lack of published research focusing on predicting antimicrobial peptides specifically targeting protozoal pathogens. In this study, we introduce a successful machine learning-based framework designed to predict potential antiprotozoal peptides effective against protozoal pathogens. ObjectiveThe primary objective of this study is to classify and predict antiprotozoal peptides using diverse negative datasets. MethodsA comprehensive literature review was conducted to gather experimentally validated antiprotozoal peptides, forming the positive dataset for our study. To construct a robust machine learning classifier, multiple negative datasets were incorporated, including (i) non-antimicrobial, (ii) antiviral, (iii) antibacterial, (iv) antifungal, and (v) antimicrobial peptides excluding those targeting protozoal pathogens. Various compositional features of the peptides were extracted using the pfeature algorithm. Two feature selection methods, SVC-L1 and mRMR, were employed to identify highly relevant features crucial for distinguishing between the positive and negative datasets. Additionally, five popular classifiers i.e. Decision Tree, Random Forest, Support Vector Machine, Logistic Regression, and XGBoost were used to build efficient decision models. ResultsXGBoost was the most effective in classifying antiprotozoal peptides from each negative dataset based on the features selected by the mRMR feature selection method. The proposed machine learning framework efficiently differentiate the antiprotozoal peptides from (i) non-antimicrobial (ii) antiviral (iii) antibacterial (iv) antifungal and (v) antimicrobial with accuracy of 97.27 %, 93.64 %, 86.36 %, 90.91 %, and 89.09 % respectively on the validation dataset. ConclusionThe models are incorporated in a user-friendly web server (www.soodlab.com/appred) to predict the antiprotozoal activity of given peptides.

Enhancing Breast Cancer Survival Prognosis Through Omic and Non-Omic Data Integration

Background and ObjectiveCancer, the second leading cause of death globally, claimed 685,000 lives among 2.3 million women affected by breast cancer in 2020. Cancer prognosis plays a pivotal role in tailoring treatments and assessing efficacy, emphasizing the need for a comprehensive understanding. The goal is to develop predictive model capable of accurately predicting patient outcomes and guiding personalized treatment strategies, thereby advancing precision medicine in breast cancer care. MethodsThis project addresses limitations in current cancer prognosis models by integrating omics and non-omics data. While existing models often neglect crucial omics data like DNA methylation and miRNA, the method utilizes the TCGA dataset to incorporate these data types along with others. Employing mRMR feature selection and CNN models for each type of data for feature extraction, features are stacked and a Random Forest classifier is employed for final prognosis. ResultThe proposed method is applied to the dataset to predict whether the patient is a long-time or a short-time survivor. This strategy showcases excellent performance, with an AUC value of 0.873, precision at 0.881, and sensitivity reaching 0.943. With an accuracy rate of 0.861, signaling an improvement of 11.96% compared to prior studies. ConclusionIn conclusion, integrating diverse data with advanced machine learning holds promise for improving breast cancer prognosis. Addressing model limitations and leveraging comprehensive datasets can enhance accuracy, paving the way for better patient care. Further refinement offers potential for significant advancements in cancer prognosis and treatment strategies.

Gaussian process regression coupled with mRMR to predict adulterant concentration in cocaine

Street cocaine is often mixed with various substances that intensify its harmful effects. This paper proposes a framework to identify attenuated total reflection Fourier transform infrared spectroscopy (ATR-FTIR) intervals that best predict the concentration of adulterants in cocaine samples. Wavelengths are ranked according to their relevance through ReliefF and mRMR feature selection approaches, and an iterative process removes less relevant wavelengths based on the ranking suggested by each approach. Gaussian Process (GP) regression models are constructed after each wavelength removal and the prediction performance is evaluated using RMSE. The subset balancing a low RMSE value and a small percentage of retained wavelengths is chosen. The proposed framework was validated using a dataset consisting of 345 samples of cocaine with different amounts of levamisole, caffeine, phenacetin, and lidocaine. Averaged over the four adulterants, the GP regression coupled with the mRMR retained 1.07 % of the 662 original wavelengths, outperforming PLS and SVR regarding prediction performance.

Serum Metabolites Relate to Mucosal and Transmural Inflammation in Pediatric Crohn Disease.

We aimed to identify serum metabolites associated with mucosal and transmural inflammation in pediatric Crohn disease (pCD). Fifty-six pCD patients were included through a pre-planned sub-study of the multicenter, prospective, ImageKids cohort, designed to develop the Pediatric Inflammatory Crohn's MRE Index (PICMI). Children were included throughout their disease course when undergoing ileocolonoscopy and magnetic resonance enterography (MRE) and followed for 18 months when MRE was repeated. Serum metabolites were identified using liquid chromatography/mass spectroscopy. Outcomes included: PICMI, the simple endoscopic score (SES), faecal calprotectin (FCP), and C-reactive protein (CRP), to assess transmural, mucosal, and systemic inflammation, respectively. Random forest models were built by outcome. Maximum relevance minimum redundancy (mRMR) feature selection with a j-fold cross validation scheme identified the best subset of features and hyperparameter settings. Tryptophan and glutarylcarnitine were the top common mRMR metabolites linked to pCD inflammation. Random forest models established that amino acids and amines were among the most influential metabolites for predicting transmural and mucosal inflammation. Predictive models performed well, each with an area under the curve (AUC) > 70%. In addition, serum metabolites linked with pCD inflammation mainly related to perturbations in citrate cycle (TCA cycle), aminoacyl-tRNA biosynthesis, tryptophan metabolism, butanoate metabolism, and tyrosine metabolism. We extend on recent studies, observing differences in serum metabolite between healthy controls and Crohn disease patients, and suggest various associations of serum metabolites with transmural and mucosal inflammation. These metabolites could improve the understanding of pCD pathogenesis and assess disease severity.

Hybrid mRMR and multi-objective particle swarm feature selection methods and application to metabolomics of traditional Chinese medicine

Hybrid mRMR and multi-objective particle swarm feature selection methods and application to metabolomics of traditional Chinese medicine

An open-access simultaneous electrocardiogram and phonocardiogram database

Objective. The EPHNOGRAM project aimed to develop a low-cost, low-power device for simultaneous electrocardiogram (ECG) and phonocardiogram (PCG) recording, with additional channels for environmental audio to enhance PCG through active noise cancellation. The objective was to study multimodal electro-mechanical activities of the heart, offering insights into the differences and synergies between these modalities during various cardiac activity levels. Approach. We developed and tested several hardware prototypes of a simultaneous ECG-PCG acquisition device. Using this technology, we collected simultaneous ECG and PCG data from 24 healthy adults during different physical activities, including resting, walking, running, and stationary biking, in an indoor fitness center. The data were annotated using a robust software that we developed for detecting ECG R-peaks and PCG S1 and S2 components, and overseen by a human expert. We also developed machine learning models using ECG-based, PCG-based, and joint ECG-PCG features, like R–R and S1–S2 intervals, to classify physical activities and analyze electro-mechanical dynamics. Main results. The results show a significant coupling between ECG and PCG components, especially during high-intensity exercise. Notable micro-variations in S2-based heart rate show differences in the heart’s electrical and mechanical functions. The Lomb-Scargle periodogram and approximate entropy analyses confirm the higher volatility of S2-based heart rate compared to ECG-based heart rate. Correlation analysis shows stronger coupling between R–R and R-S1 intervals during high-intensity activities. Hybrid ECG-PCG features, like the R-S2 interval, were identified as more informative for physical activity classification through mRMR feature selection and SHAP value analysis. Significance. The EPHNOGRAM database, is available on PhysioNet. The database enhances our understanding of cardiac function, enabling future studies on the heart’s mechanical and electrical interrelationships. The results of this study can contribute to improved cardiac condition diagnoses. Additionally, the designed hardware has the potential for integration into wearable devices and the development of multimodal stress test technologies.

Fed-mRMR: A lossless federated feature selection method

Feature selection has become a mandatory task in data mining, due to the overwhelming amount of features in Big Data problems. To handle this high-dimensional data and avoid the well-known curse of dimensionality, we need to pre-select an optimal subset of features to reduce redundant computations. Federated learning is a machine learning technique based on training an algorithm over many decentralized edge devices holding local rather than global data on a centralized server. Application of this technique is extending to fields such as self-driving cars, medicine and health, and Industry 4.0, where data privacy is compulsory. Feature selection through federated learning is a complicated task since suboptimal features calculated by feature selection methods may be different in heterogeneous datasets from different nodes. In this paper, we propose a lossless federated version of the classic minimum redundancy maximum relevance (mRMR) feature selection algorithm, called federated mRMR (fed-mRMR), which, without losing any effectiveness of the original mRMR method, is applicable to federated learning approaches and capable of dealing with data that are not independent and identically distributed (non-IID data).Implementation can be found at: https://github.com/jorgehermo9/fed-mrmr

Predictive modeling for breast cancer based on machine learning algorithms and features selection methods

Breast cancer is one of the leading causes of death among women worldwide. However, early prediction of breast cancer plays a crucial role. Therefore, strong needs exist for automatic accurate early prediction of breast cancer. In this paper, machine learning (ML) classifiers combined with features selection methods are used to build an intelligent tool for breast cancer prediction. The Wisconsin diagnostic breast cancer (WDBC) dataset is used to train and test the model. Classification algorithms, including support vector machine (SVM), light gradient boosting machine (LightGBM), random forest (RF), logistic regression (LR), k-nearest neighbors (k-NN), and naïve Bayes, were employed. Performance measures for each of them were obtained, namely: accuracy, precision, recall, F-score, Kappa, Matthews correlation coefficient (MCC), and time. The results indicate that without feature selection, LightGBM achieves the highest accuracy at 95%. With minimum redundancy maximum relevance (mRMR) feature selection (15 features), LightGBM outperforms other classifiers, achieving an accuracy of 98%. For Pearson correlation coefficient feature selection (15 features), LightGBM also excels with a 95% accuracy rate. Lasso feature selection (5 features) produces varied results across classifiers, with logistic regression achieving the highest accuracy at 96%. These findings underscore the importance of feature selection in refining model performance and in improving detection for breast cancer.

Analysis of selected deep features with CNN-SVM-based for bread wheat seed classification

The main ingredient of flour is processed wheat. Wheat is an agricultural product that is harvested once a year. It may be necessary to choose the variety of wheat for growing wheat and efficient harvesting. The variety of wheat is important for its economic value, taste, and crop yield. Although there are many varieties of wheat, they are very similar in colour, size, and shape, and it requires expertise to distinguish them by eye. This is very time consuming and can lead to human error. Using computer vision and artificial intelligence, such problems can be solved more quickly and objectively. In this study, an attempt was made to classify five bread wheat varieties belonging to different cultivars using Convolutional Neural Network (CNN) models. Three approaches have been proposed for classification. First, pre-trained CNN models (ResNet18, ResNet50, and ResNet101) were trained for bread wheat cultivars. Second, the features extracted from the fc1000 layer of the pre-trained CNN models ResNet18, ResNet50, and ResNet101 were classified using a support vector machine (SVM) classifier with different kernel features from machine learning techniques for classification with different variants. Finally, SVM methods were used in the second stage to classify the features obtained from the fc1000 layer of the pre-trained CNN models with an optimal set of features that can represent all features using the minimum redundancy maximum relevance (mRMR) feature selection algorithm.The accuracies obtained in the first, second, and last phases are as follows. In the first phase, the most successful method in classifying wheat grains was the ResNet18 model with 97.57%. In the second phase, the ResNet18 + ResNet50 + ResNet101 + Quadratic SVM model was the most successful model in classification using the features obtained from the ResNet CNN models with 94.08%.The accuracy for classification with the 1000 most effective features selected by the feature selection algorithm was 94.51%. Although the classification with features is slightly lower than deep learning, the classification time is much shorter and is 93%. This result confirms the great effectiveness of CNN models for wheat grain classification.

Precise Face Recognition through GWO-Cuckoo Optimized Neural Networks and MRMR Feature Selection from Compressed Hybrid Domain Fusion

Precise Face Recognition through GWO-Cuckoo Optimized Neural Networks and MRMR Feature Selection from Compressed Hybrid Domain Fusion

An Intelligent Healthcare System for Automated Diabetes Diagnosis and Prediction using Machine Learning

The progression of biotechnological and health science fields has instigated a substantial proliferation of data, encompassing high-throughput genetic information and extensive clinical data derived from expansive electronic health record repositories. In the domain of biosciences, the indispensability of machine learning and data mining methodologies has surged, attaining heightened significance for the purpose of converting extant information into usable knowledge. Diabetes mellitus is a metabolic condition that affects human health. Extensive research on diabetes (diagnosis, therapy, etc.) has generated vast volumes of data. Manual diagnosis of diabetes has always been a time-consuming task. Therefore, automatic detection and diagnosis of diabetes using artificial intelligence and machine learning is gaining prominence. In our work, we have devised a novel architecture using machine learning for the automatic diagnosis of diabetes. We implemented our model using many algorithms and found that random forest is the most optimized and accurate one for classification purposes. It produced the highest accuracy of 98.07%, precision, recall, F1-score of 98%, and logarithmic loss of 0.03 using the mRMR feature selection method and 0.2 test split. It produced a recall of 97%, the precision of 97%, an F1-score of 97%, an accuracy of 96.79%, and a log loss of 0.11 on 0.3 test split with mRMR feature selection. Also, the proposed model has performed better than most state-of-the-art models.