Feature Ranking Methods Research Articles

Machine learning-based prognostic model for 30-day mortality prediction in Sepsis-3

BackgroundSepsis poses a critical threat to hospitalized patients, particularly those in the Intensive Care Unit (ICU). Rapid identification of Sepsis is crucial for improving survival rates. Machine learning techniques offer advantages over traditional methods for predicting outcomes. This study aimed to develop a prognostic model using a Stacking-based Meta-Classifier to predict 30-day mortality risks in Sepsis-3 patients from the MIMIC-III database.MethodsA cohort of 4,240 Sepsis-3 patients was analyzed, with 783 experiencing 30-day mortality and 3,457 surviving. Fifteen biomarkers were selected using feature ranking methods, including Extreme Gradient Boosting (XGBoost), Random Forest, and Extra Tree, and the Logistic Regression (LR) model was used to assess their individual predictability with a fivefold cross-validation approach for the validation of the prediction. The dataset was balanced using the SMOTE-TOMEK LINK technique, and a stacking-based meta-classifier was used for 30-day mortality prediction. The SHapley Additive explanations analysis was performed to explain the model’s prediction.ResultsUsing the LR classifier, the model achieved an area under the curve or AUC score of 0.99. A nomogram provided clinical insights into the biomarkers' significance. The stacked meta-learner, LR classifier exhibited the best performance with 95.52% accuracy, 95.79% precision, 95.52% recall, 93.65% specificity, and a 95.60% F1-score.ConclusionsIn conjunction with the nomogram, the proposed stacking classifier model effectively predicted 30-day mortality in Sepsis patients. This approach holds promise for early intervention and improved outcomes in treating Sepsis cases.

PHIGrader: Evaluating the effectiveness of Manifest file components in Android malware detection using Multi Criteria Decision Making techniques

The popularity of the Android operating system has itself become a reason for privacy concerns. To deal with such malware threats, researchers have proposed various detection approaches using static and dynamic features. Static analysis approaches are the most convenient for practical detection. However, several patterns of feature usage were found to be similar in the normal and malware datasets. Such high similarity in both datasets’ feature patterns motivates us to rank and select only the distinguishing set of features. Hence, in this study, we present a novel Android malware detection system, termed as PHIGrader for ranking and evaluating the efficiency of the three most commonly used static features, namely permissions, intents, and hardware components, when used for Android malware detection. To meet our goals, we individually rank the three feature types using frequency-based Multi-Criteria Decision Making (MCDM) techniques, namely TOPSIS and EDAS. Then, the system applies a novel detection algorithm to the rankings involving machine learning and deep learning classifiers to present the best set of features and feature type with higher detection accuracy as an output. The experimental results highlight that our proposed approach can effectively detect Android malware with 99.10% detection accuracy, achieved with the top 46 intents when ranked using TOPSIS, which is better than permissions, hardware components, or even the case where other popular MCDM techniques are used. Furthermore, our experiments demonstrate that the proposed system with frequency-based MCDM rankings is better than other statistical tests such as mutual information, Pearson correlation coefficient, and t-test. In addition, our proposed model outperforms various popularly used feature ranking methods such as Chi-square, Principal Component Analysis (PCA), Entropy-based Category Coverage Difference (ECCD), and other state-of-the-art Android malware detection techniques in terms of detection accuracy.

Enhancing Feature Selection in MCI Diagnosis using FDG-PET Images: Leveraging Multiple Simple Autoencoder Architectures

Alzheimer’s Disease (AD) is the most common type of neurodegenerative brain disease in elderly people. Early diagnosis of AD is crucial for providing suitable care. Positron Emission Tomography (PET) images and machine learning can be used to support this purpose. In this paper, we present a method for ranking the effectiveness of brain regions of interest (ROIs) to distinguish between stable mild cognitive impairment (sMCI) from progressive mild cognitive impairment (pMCI) in brain PET images based on AutoEncoder (AE). Experiments on the ADNI dataset show that our proposed method significantly improves classifier performance when compared to other popular feature ranking methods such as Fisher score, t-score, and LASSO. Our results suggest that instead of focusing on designing a complex AE structure, we can also use simple-but-multiple AEs for feature ranking. The proposed method could be easily applied to any image dataset where a feature selection is needed.

Machine Learning and Statistical Methods for Predicting Survival of Patients with Heart Failure

Heart failure is a branch of heart disease, which causes millions of people to die worldwide. This research is an investigation of survival prediction of patients with heart failure. The type of this prediction is a binary classification problem. The machine learning models used in this paper are logistic regression, decision trees, random forests, support vector machines, and artificial neural networks. The methods for evaluating the performance of prediction models are k-fold and stratified k-fold cross-validation. The results of 2 cross-validation indicate that logistic regression has the best performance. In addition, according to the feature ranking method in the literature, it can be observed that the prediction of heart failure mainly relies on serum creatinine, ejection fraction, and follow-up time. The conclusion is that the logistic regression, which only involves features: serum creatinine, ejection fraction, and follow-up time, is well suited for predicting the survival of patients with heart failure.

Cell recognition based on features extracted by AFM and parameter optimization classifiers.

Intelligent technology can assist in the diagnosis and treatment of disease, which would pave the way towards precision medicine in the coming decade. As a key focus of medical research, the diagnosis and prognosis of cancer play an important role in the future survival of patients. In this work, a diagnostic method based on nano-resolution imaging was proposed to meet the demand for precise detection methods in medicine and scientific research. The cell images scanned by AFM were recognized by cell feature engineering and machine learning classifiers. A feature ranking method based on the importance of features to responses was used to screen features closely related to categorization and optimization of feature combinations, which helps to understand the feature differences between cell types at the micro level. The results showed that the Bayesian optimized back propagation neural network has accuracy rates of 90.37% and 92.68% on two cell datasets (HL-7702 & SMMC-7721 and GES-1 & SGC-7901), respectively. This provides an automatic analysis method for identifying cancer cells or abnormal cells, which can help to reduce the burden of medical or scientific research, decrease misjudgment and promote precise medical care for the whole society.

Impact of dimensionality reduction techniques on student performance prediction using machine learning

This study addresses the crucial issue of predicting student performance in educational data mining (EDM) by proposing an Adaptive Dimensionality Reduction Algorithm (ADRA). ADRA efficiently reduces the dimensionality of student data, encompassing various academic, demographic, behavioral, social, and health-related features. It achieves this by iteratively selecting the most relevant features based on a combined normalized mean rank of five feature ranking methods. This reduction in dimensionality enhances the performance of predictive models and provides valuable insights into the key factors influencing student performance. The study evaluates ADRA using four different student performance datasets and six machine learning algorithms, comparing it to three existing dimensionality reduction methods. The results show that ADRA achieves an average dimensionality reduction factor of 6.2 while maintaing comprable accuracy with other mehtods.

Automatic selection of spoken language biomarkers for dementia detection

This paper analyzes diverse features extracted from spoken language to select the most discriminative ones for dementia detection. We present a two-step feature selection (FS) approach: Step 1 utilizes filter methods to pre-screen features, and Step 2 uses a novel feature ranking (FR) method, referred to as dual dropout ranking (DDR), to rank the screened features and select spoken language biomarkers. The proposed DDR is based on a dual-net architecture that separates FS and dementia detection into two neural networks (namely, the operator and selector). The operator is trained on features obtained from the selector to reduce classification or regression loss. The selector is optimized to predict the operator’s performance based on automatic regularization. Results show that the approach significantly reduces feature dimensionality while identifying small feature subsets that achieve comparable or superior performance compared with the full, default feature set. The Python codes are available at https://github.com/kexquan/dual-dropout-ranking.

Feature Selection with Weighted Ensemble Ranking for Improved Classification Performance on the CSE-CIC-IDS2018 Dataset

Feature selection is a crucial step in machine learning, aiming to identify the most relevant features in high-dimensional data in order to reduce the computational complexity of model development and improve generalization performance. Ensemble feature-ranking methods combine the results of several feature-selection techniques to identify a subset of the most relevant features for a given task. In many cases, they produce a more comprehensive ranking of features than the individual methods used alone. This paper presents a novel approach to ensemble feature ranking, which uses a weighted average of the individual ranking scores calculated using these individual methods. The optimal weights are determined using a Taguchi-type design of experiments. The proposed methodology significantly improves classification performance on the CSE-CIC-IDS2018 dataset, particularly for attack types where traditional average-based feature-ranking score combinations result in low classification metrics.

A novel framework for strength prediction of geopolymer mortar: Renovative precursor effect

Concrete is the most used building material today as it has many advantages due to its structure. Geopolymer composites could potentially replace concrete in the future due to the demands of the construction industry. The studies related to the geopolymers have challenges due to the lack of a linear relationship between compressive strength (CS) and flexural strength (FS) caused by factors such as a binder, mixing parameters, limited available data, and time-consuming trial and error methods. Novel prediction models need to be developed as the reliability of the prediction model currently being used requires improvement. This study attempts to develop an optimum prediction technique for compressive strength and flexural strength of geopolymer by using Machine Learning (ML) algorithms to meet the mentioned research need. For this purpose, a new database was created with mechanical properties (156 (CS) and (FS) obtained from geopolymer mortar samples with a comprehensive laboratory process using materials such as obsidian (OB), Ground granulated blast furnace slag (GGBS), and metakaolin (MT) as binders. Five machine-learning algorithms were applied to the dataset. Feature statistics and two different feature ranking methods were also performed to regulate each ML algorithm. Among the algorithms, the best R square performance results were given by Random Forest as a value of 0.983. In the sensitivity analysis that examined the effects of inputs on outputs, obsidian was shown to be the most important input.

Assessment of Alzheimer-related pathologies of dementia using machine learning feature selection

Although a variety of brain lesions may contribute to the pathological assessment of dementia, the relationship of these lesions to dementia, how they interact and how to quantify them remains uncertain. Systematically assessing neuropathological measures by their degree of association with dementia may lead to better diagnostic systems and treatment targets. This study aims to apply machine learning approaches to feature selection in order to identify critical features of Alzheimer-related pathologies associated with dementia. We applied machine learning techniques for feature ranking and classification to objectively compare neuropathological features and their relationship to dementia status during life using a cohort (n=186) from the Cognitive Function and Ageing Study (CFAS). We first tested Alzheimer’s Disease and tau markers and then other neuropathologies associated with dementia. Seven feature ranking methods using different information criteria consistently ranked 22 out of the 34 neuropathology features for importance to dementia classification. Although highly correlated, Braak neurofibrillary tangle stage, beta-amyloid and cerebral amyloid angiopathy features were ranked the highest. The best-performing dementia classifier using the top eight neuropathological features achieved 79% sensitivity, 69% specificity and 75% precision. However, when assessing all seven classifiers and the 22 ranked features, a substantial proportion (40.4%) of dementia cases was consistently misclassified. These results highlight the benefits of using machine learning to identify critical indices of plaque, tangle and cerebral amyloid angiopathy burdens that may be useful for classifying dementia.

Two phase feature-ranking for new soil dataset for Coxiella burnetii persistence and classification using machine learning models

Coxiella burnetii (Cb) is a hardy, stealth bacterial pathogen lethal for humans and animals. Its tremendous resistance to the environment, ease of propagation, and incredibly low infectious dosage make it an attractive organism for biowarfare. Current research on the classification of Coxiella and features influencing its presence in the soil is generally confined to statistical techniques. Machine learning other than traditional approaches can help us better predict epidemiological modeling for this soil-based pathogen of public significance. We developed a two-phase feature-ranking technique for the pathogen on a new soil feature dataset. The feature ranking applies methods such as ReliefF (RLF), OneR (ONR), and correlation (CR) for the first phase and a combination of techniques utilizing weighted scores to determine the final soil attribute ranks in the second phase. Different classification methods such as Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), Logistic Regression (LR), and Multi-Layer Perceptron (MLP) have been utilized for the classification of soil attribute dataset for Coxiella positive and negative soils. The feature-ranking methods established that potassium, chromium, cadmium, nitrogen, organic matter, and soluble salts are the most significant attributes. At the same time, manganese, clay, phosphorous, copper, and lead are the least contributing soil features for the prevalence of the bacteria. However, potassium is the most influential feature, and manganese is the least significant soil feature. The attribute ranking using RLF generates the most promising results among the ranking methods by generating an accuracy of 80.85% for MLP, 79.79% for LR, and 79.8% for LDA. Overall, SVM and MLP are the best-performing classifiers, where SVM yields an accuracy of 82.98% and 81.91% for attribute ranking by CR and RLF; and MLP generates an accuracy of 76.60% for ONR. Thus, machine models can help us better understand the environment, assisting in the prevalence of bacteria and decreasing the chances of false classification. Subsequently, this can assist in controlling epidemics and alleviating the devastating effect on the socio-economics of society.

Fusion of Feature Ranking Methods for an Effective Intrusion Detection System

Fusion of Feature Ranking Methods for an Effective Intrusion Detection System

Identification of Methylation Signatures and Rules for Sarcoma Subtypes by Machine Learning Methods.

Sarcoma, the second common type of solid tumor in children and adolescents, has a wide variety of subtypes that are often not properly diagnosed at an early stage, leading to late metastases and causing serious loss of life and property to patients and families. It exhibits a high degree of heterogeneity at the cellular, molecular, and epigenetic levels, where DNA methylation has been proposed to play a role in the diagnosis of sarcoma subtypes. Thus, this study is aimed at finding potential biomarkers at the DNA methylation level to distinguish different sarcoma subtypes. A machine learning process was designed to analyse sarcoma samples, each of which was represented by lots of methylation sites. Irrelevant sites were removed using the Boruta method, and remaining sites related to the target variables were kept for further analyses. Afterward, three feature ranking methods (LASSO, LightGBM, and MCFS) were adopted to rank these features, and six classification models were constructed by combining incremental feature selection and two classification algorithms (decision tree and random forest). Among these models, the performance of RF model was higher than that of DT model under all three ranking conditions. The specific expression of genes obtained from the annotation of highly correlated methylation site features, such as PRKAR1B, INPP5A, and GLI3, was proven to be associated with sarcoma by publications. Moreover, the quantitative rules obtained by decision tree algorithm helped us to understand the essential differences between various sarcoma types and classify sarcoma subtypes, providing a new means of clinical identification and determining new therapeutic targets.

Predicting Customer Churn on OTT Platforms

No industry can thrive without customers and with customers comes the chances ofcustomer churn. Since customer churn have direct-impact on the revenue, all theindustries are focusing in understanding the factors influencing churn and aredeveloping methods to predict the customer churn effectively. Today, never as before,customers have wide variety of options to choose between any service or product. Inaddition, nowadays customers enjoy multiple subscriptions of service providers acrosssectors. In this study we aim to identify: i) Factors influencing customer churn on OTTplatform, and ii) Predict customer churn on OTT platform. The data for this study iscollected from 317 respondents, using questionnaire method, who have multiple OTTplatform subscription. The questionnaire data contains 19 items which includesdemographic features, usage of OTG platform, and user contentment factors about OTTservice. We have identified factors influencing customer churn in Over-The-Top (OTT)platform by combining Recursive Feature Elimination (RFE), Linear Regression, andRidge Regression feature ranking methods. We have used Hierarchical LogisticRegression, to understand impact of two newly introduced factors namely 'MultipleSubscription' and 'Switching Frequency' on the overall performance of the customerchurn prediction. Finally, customer churn prediction is done using Decision Tree,Random Forest, AdaBoost, and Gradient boosting techniques. We found that randomforest method gives better prediction results.

Design flood estimation using extreme Gradient Boosting-based on Bayesian optimization

• Proposed XGB-based ML model for RFFA and flood estimation. • CAMELs data for 671 catchments from the USA were used to test the efficacy of the approach. • Multiple Linear Regression methods and Artificial Neural Networks were used to compare the output of the eXtreme Gradient Boost-based ML model. • The importance of each catchment feature is characterized using Gini Impurity, Permutation, and Dropout Loss Feature Ranking methods. Regional Flood Frequency Analysis (RFFA) is one of the widely used approaches for estimating design floods in the ungauged basins. We developed an eXtreme Gradient Boost (XGB) machine learning model for RFFA and flood estimation. Our approach relies on developing a regression model between flood quantiles and the commonly available catchment descriptors. We used CAMELs data for 671 catchments from the USA to test the approach's efficacy. The results were compared with the traditional Multiple Linear Regression methods and Artificial Neural Networks. Results revealed that the XGB-based approach estimated design flood with the highest accuracy during training and validation with minor mean absolute error, root mean square error values, and percentage bias ranging from −10 to + 10. The importance of each catchment feature is visualized by three different approaches Gini Impurity, Permutation, and Dropout Loss Feature Ranking. We observed that the most dominating variables are rainfall intensity, slope, snow fraction, soil porosity, and temperature. It is observed that the importance of these variables is a function of the hydroclimatic regions and varies with space. In contrast, mean annual areal potential evapotranspiration, mean annual rainfall, fraction forest area, and soil conductivity have low significance in estimating design flood for an ungauged catchment. Indeed, the proposed XGB-based approach has broader applicability and replicability.

Functional and embedding feature analysis for pan-cancer classification.

With the increasing number of people suffering from cancer, this illness has become a major health problem worldwide. Exploring the biological functions and signaling pathways of carcinogenesis is essential for cancer detection and research. In this study, a mutation dataset for eleven cancer types was first obtained from a web-based resource called cBioPortal for Cancer Genomics, followed by extracting 21,049 features from three aspects: relationship to GO and KEGG (enrichment features), mutated genes learned by word2vec (text features), and protein-protein interaction network analyzed by node2vec (network features). Irrelevant features were then excluded using the Boruta feature filtering method, and the retained relevant features were ranked by four feature selection methods (least absolute shrinkage and selection operator, minimum redundancy maximum relevance, Monte Carlo feature selection and light gradient boosting machine) to generate four feature-ranked lists. Incremental feature selection was used to determine the optimal number of features based on these feature lists to build the optimal classifiers and derive interpretable classification rules. The results of four feature-ranking methods were integrated to identify key functional pathways, such as olfactory transduction (hsa04740) and colorectal cancer (hsa05210), and the roles of these functional pathways in cancers were discussed in reference to literature. Overall, this machine learning-based study revealed the altered biological functions of cancers and provided a reference for the mechanisms of different cancers.

Improvement of Patient Classification Using Feature Selection Applied to Bidirectional Axial Transmission.

Osteoporosis is still a worldwide problem, particularly due to associated fragility fractures. Patients at risk of fracture are currently detected using the X-Ray gold standard dual-energy X-ray absorptiometry (DXA), based on a calibrated 2-D image. Different alternatives, such as 3-D X-rays, magnetic resonance imaging (MRI) or ultrasound, have been proposed, the latter having advantages of being portable and sensitive to mechanical and geometrical properties. Bidirectional axial transmission (BDAT) has been used to classify between patients with or without nontraumatic fractures using "classical" ultrasonic parameters, such as velocities, as well as cortical thickness and porosity, obtained from an inverse problems. Recently, complementary parameters acquired with structural and textural analysis of guided wave spectrum images (GWSIs) have been introduced. These parameters are not limited by solution ambiguities, as for inverse problem. The aim of the study is to improve the patient classification using a feature selection strategy for all available ultrasound features completed by clinical parameters. To this end, three classical feature ranking methods were considered: analysis of variance (ANOVA), recursive feature elimination (RFE), and extreme gradient boosting importance feature (XGBI). In order to evaluate the performance of the feature selection techniques, three classical classification methods were used: logistic regression (LR), support vector machine (SVM), and extreme gradient boosting (XGB). The database was obtained from a previous clinical study [Minonzio et al., 2019]. Results indicate that the best accuracy of 71 [66-76]% was achieved by using RFE and SVM with 22 (out of 43) ultrasonic and clinical features. This value outperformed the accuracy of 68 [64-73]% reached with 2 (out of 6) DXA and clinical features. These values open promising perspectives toward improved and generalizable classification of patients at risk of fracture.

A Model for Classifying Rice Varieties Grown in Turkey Using Image-Based Morphological Features and Machine Learning

Abstract: Rice is one of the important cereals which feeds more than half of the world's population. It is used frequently in a variety of flavourful recipes. In this work, a dataset of Cammeo and Osmancik species found in Turkey has been selected for the study. It have 3810 samples containing seven morphological features. The feature ranking methods like Fisher score, FSV, infFS, Laplacian, ReliefF, MCFS and MUTinfFS are selected and applied on the above datasets for the purpose of finding key features for proper feature selection. After selecting key feature values, feature vector have been prepared. Then, Support vector machine technique was applied for classification based on the results obtained from feature ranking techniques.

Performance evaluation of machine learning models for distributed denial of service attack detection using improved feature selection and hyper‐parameter optimization techniques

SummaryThis article gives the framework of extensive experimentation of various machine learning models to detect distributed denial of service attacks (DDoS). We use six‐tier feature ranking methods that use statistical techniques as well as machine learning based classifiers to obtain the significant features. The measurable statistical based feature selection involves Chi‐Square (Chi2), information gain (IG), merged Chi‐Square (Chi2)‐IG ranking and machine learning classifiers involve ensemble classifiers, that is, decision tree, random forest and eXtreme gradient boosting (XGBoost). Different supervised machine learning models (logistic regression, decision tree classifier, linear support vector machine, k‐nearest neighbors, Gaussian Naive Bayes, random forest classifier, XGBoost) are trained on a feature‐engineered datasets. To further our research, we use neural networks (ANN and CNN) using both feature‐selected and auto‐feature selection training setup. We check the validation and adaptability of these models with the optimal tuning of various parameters using GridSearchCV and the effectiveness of random sampling in overcoming the class imbalance problem. Based on various feature selection methods, the models are evaluated for their best performance. The experimental results show that our trained machine learning models and neural networks outperformed the ones in the state of art. The performance analysis is done based on confusion matrix scores, that is, accuracy, false alarm rate, sensitivity, specificity, false‐positive rate, F1 score, area under curve analysis and loss functions on well‐known KDD Cup 99 and UNSW‐NB15 datasets. This study is significant for furthering the research in DDoS detection with machine learning and deep neural networks.

Using PRISMA Hyperspectral Satellite Imagery and GIS Approaches for Soil Fertility Mapping (FertiMap) in Northern Morocco

Quickly and correctly mapping soil nutrients significantly impact accurate fertilization, food security, soil productivity, and sustainable agricultural development. We evaluated the potential of the new PRISMA hyperspectral sensor for mapping soil organic matter (SOM), available soil phosphorus (P2O5), and potassium (K2O) content over a cultivated area in Khouribga, northern Morocco. These soil nutrients were estimated using (i) the random forest (RF) algorithm based on feature selection methods, including feature subset evaluation and feature ranking methods belonging to three categories (i.e., filter, wrapper, and embedded techniques), and (ii) 107 soil samples taken from the study area. The results show that the RF-embedded method produced better predictive accuracy compared with the filter and wrapper methods. The model for SOM showed moderate accuracy (Rval2 = 0.5, RMSEP = 0.43%, and RPIQ = 2.02), whereas that for soil P2O5 and K2O exhibited low efficiency (Rval2 = 0.26 and 0.36, RMSEP = 51.07 and 182.31 ppm, RPIQ = 0.65 and 1.16, respectively). The interpolation of RF-residuals by ordinary kriging (OK) methods reached the highest predictive results for SOM (Rval2 = 0.69, RMSEP = 0.34%, and RPIQ = 2.56), soil P2O5 (Rval2 = 0.44, RMSEP = 44.10 ppm, and RPIQ = 0.75), and soil K2O (Rval2 = 0.51, RMSEP = 159.29 ppm, and RPIQ = 1.34), representing the best fitting ability between the hyperspectral data and soil nutrients. The result maps provide a spatially continuous surface mapping of the soil landscape, conforming to the pedological substratum. Finally, the hyperspectral remote sensing imagery can provide a new way for modeling and mapping soil fertility, as well as the ability to diagnose nutrient deficiencies.