Feature Selection Techniques Research Articles

Alzheimer’s Prediction Methods with Harris Hawks Optimization (HHO) and Deep Learning-Based Approach Using an MLP-LSTM Hybrid Network

Background/Objective: Alzheimer’s disease is a progressive brain syndrome causing cognitive decline and, ultimately, death. Early diagnosis is essential for timely medical intervention, with MRI medical imaging serving as a primary diagnostic tool. Machine learning (ML) and deep learning (DL) methods are increasingly utilized to analyze these images, but accurately distinguishing between healthy and diseased states remains a challenge. This study aims to address these limitations by developing an integrated approach combining swarm intelligence with ML and DL techniques for Alzheimer’s disease classification. Method: This proposal methodology involves sourcing Alzheimer’s disease-related MRI images and extracting features using convolutional neural networks (CNNs) and the Gray Level Co-occurrence Matrix (GLCM). The Harris Hawks Optimization (HHO) algorithm is applied to select the most significant features. The selected features are used to train a multi-layer perceptron (MLP) neural network and further processed using a long short-term (LSTM) memory network in order to classify tumors as malignant or benign. The Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset is utilized for assessment. Results: The proposed method achieved a classification accuracy of 97.59%, sensitivity of 97.41%, and precision of 97.25%, outperforming other models, including VGG16, GLCM, and ResNet-50, in diagnosing Alzheimer’s disease. Conclusions: The results demonstrate the efficacy of the proposed approach in enhancing Alzheimer’s disease diagnosis through improved feature extraction and selection techniques. These findings highlight the potential for advanced ML and DL integration to improve diagnostic tools in medical imaging applications.

Research on Predictive Maintenance Methods for Current Transformers with Iron Core Structures

The reliable operation of power systems is heavily dependent on effective maintenance strategies for critical equipment. Current maintenance methods are typically categorized into corrective, preventive, and predictive approaches. While corrective maintenance often results in significant downtime and preventive maintenance can be inefficient, predictive maintenance emerges as a promising technique for accurately forecasting faults. In this study, we investigated the diagnosis and prediction of fault states, specifically single-phase short circuit (1HCF) and double-phase short circuit (2HCF) faults, using monitoring data from current transformers in 110 kV substations. We proposed a predictive maintenance method for current transformers based on core-type structures, which integrates wavelet transform to extract multi-level frequency domain features, employs feature selection techniques (including the Spearman correlation coefficient and mutual information) to identify key predictive features, and utilizes Random Forest classifiers for fault state prediction. Experimental results demonstrate an overall prediction accuracy of 94%.

An AI-based approach to predict delivery outcome based on measurable factors of pregnant mothers.

The desire for safer delivery mode that preserves the lives of both mother and child with minimal or no complications before, during and after childbirth is the wish for every expectant mother and their families. However, the choice for any particular delivery mode is supposedly influenced by a number of factors that leads to the ultimate decision of choice. Some of the factors identified include maternal birth history, maternal and child health conditions prevailing before and during labor onset. Predictive modeling has been used extensively to determine important contributory factors or artifacts influencing delivery choice in related research studies. However, missing among a myriad of features used in various research studies for this determination is maternal history of spontaneous, threatened and inevitable abortion(s). How its inclusion impacts delivery outcome has not been covered in extensive research work. This research work therefore takes measurable maternal features that include real time information on administered partographs to predict delivery outcome. This is achieved by adopting effective feature selection technique to estimate variable relationships with the target variable. Three supervised learning techniques are used and evaluated for performance. Prediction accuracy score of area under the curve obtained show Gradient Boosting classifier achieved 91% accuracy, Logistic Regression 93% and Random Forest 91%. Balanced accuracy score obtained for these techniques were; Gradient Boosting 82.73%, Logistic Regression 84.62% and Random Forest 83.02%. Correlation statistic for variable independence among input variables showed that delivery outcome type as an output is associated with fetal gestational age and the progress of maternal cervix dilatation during labor onset.

Forecasting the River Ice Break-Up Date in the Upper Reaches of the Heilongjiang River Based on Machine Learning

Ice-jam floods (IJFs) are a significant hydrological phenomenon in the upper reaches of the Heilongjiang River, posing substantial threats to public safety and property. This study employed various feature selection techniques, including the Pearson correlation coefficient (PCC), Grey Relational Analysis (GRA), mutual information (MI), and stepwise regression (SR), to identify key predictors of river ice break-up dates. Based on this, we constructed various machine learning models, including Extreme Gradient Boosting (XGBoost), Backpropagation Neural Network (BPNN), Random Forest (RF), and Support Vector Regression (SVR). The results indicate that the ice reserves in the Oupu to Heihe section have the most significant impact on the ice break-up date in the Heihe section. Additionally, the accumulated temperature during the break-up period and average temperature before river ice break-up are identified as features closely related to the river’s opening in all four feature selection methods. The choice of feature selection method notably impacts the performance of the machine learning models in predicting the river ice break-up dates. Among the models tested, XGBoost with PCC-based feature selection achieved the highest accuracy (RMSE = 2.074, MAE = 1.571, R2 = 0.784, NSE = 0.756, TSS = 0.950). This study provides a more accurate and effective method for predicting river ice break-up dates, offering a scientific basis for preventing and managing IJF disasters.

A novel early stage drip irrigation system cost estimation model based on management and environmental variables

One of the most significant, intricate, and little-discussed aspects of pressurized irrigation is cost estimation. This study attempts to model the early-stage cost of the drip irrigation system using a database of 515 projects divided into four sections the cost of the pumping station and central control system (TCP), the cost of on-farm equipment (TCF), the cost of installation and operation on-farm and pumping station (TCI), and the total cost (TCT). First, 39 environmental and management features affecting the cost of the listed sectors were extracted for each of the 515 projects previously mentioned. A database (a matrix of 515 × 43) was created, and the costs of all projects were updated for the baseline year of 2022. Then, several feature selection algorithms, such as WCC, LCA, GA, PSO, ACO, ICA, LA, HTS, FOA, DSOS, and CUK, were employed to choose the most significant features that had the biggest influence on the system cost. The selection of features was carried out for all features (a total of 39 features) as well as for easily available features (those features that existed before the irrigation system’s design phase, 18 features). Then, different machine learning models such as Multivariate Linear Regression, Support Vector Regression, Artificial Neural Networks, Gene Expression Programming, Genetic Algorithms, Deep Learning, and Decision Trees, were used to estimate the costs of each of the of the aforementioned sections. Support vector machine (SVM) and optimization algorithms (Wrapper) were found to be the best learner and feature selection techniques, respectively, out of all the available feature selection algorithms. The two LCA and FOA algorithms produced the best estimation, according to the evaluation criteria results. Their RMSE for all features was 0.0020 and 0.0018, respectively, and their R2 was 0.94 and 0.94. For readily available features, these criteria were 0.0006 and 0.95 for both algorithms. In the part of the overall feature, the early-stage cost modeling with selected features revealed that the SVM model (with RBF Kernel) is the best model among the four cost sections discussed. Its evaluation criteria in the training stage are R2 = 0.923, RMSE = 0.008, and VE = 0.082; in the testing stage, they are R2 = 0.893, RMSE = 0.009, and VE = 0.102. The ANN model (MLP) was found to be the best model for a subset of features in the easily available feature part, with R2 = 0.912, RMSE = 0.008, and VE = 0.083 in the training stage and R2 = 0.882, RMSE = 0.009, and VE = 0.103 in the testing stage. The findings of this study can be utilized to highly accurately estimate the cost of local irrigation systems based on the recognized environmental and management parameters and by employing particular models.

Combined hybrid feature selection and classification for heart disease prediction in the cloud-based IoT health care system using Machine Learning

Health care Management System (HMS) is a key to successful management of any health care industry. Health care management system has so many research dimensions such as identifying disease and diagnostic, drug discovery manufacturing, Bioinformatics’ problem, personalized treatments, Patient image analysis and so on. Heart Disease Prediction (HDP) is a process of identifying heart disease in advance and recognizes patient health condition by applying techniques on patient heart related symptoms. Now a day’s the problem of identifying heart diseases are solved by machine learning techniques. In this paper we are constructed heart disease prediction method using combined feature selection and classification machine learning techniques. According to the existing study the one of the main difficult in heart disease prediction system is that the available data in open sources are not properly recorded the necessary characteristics and also there is some lagging in finding the useful features from the available features. The process of removing inappropriate features from an available feature set while preserving sufficient classification accuracy is known as feature selection. A methodology is proposed in this paper that consists of two phases: Phase one employs two broad categories of feature selection techniques to identify the efficient feature sets and it is given to the input of our second phase such as classification. In this work we will concentrated on filter based method for feature selection such as Chi-square, Fast Correlation Based Filter (FCBF), Gini Index (GI), RelifeF, and wrapper based method for feature selection such as Backward Feature Elimination (BFE), Exhaustive Feature Selection (EFS), Forward Feature Selection (FFS), and Recursive Feature Elimination (RFE). The UCI heart disease data set is used to evaluate the output in this study. Finally, the proposed system's performance is validated by various experiments setups.

Robust machine learning based Intrusion detection system using simple statistical techniques in feature selection

There are serious security issues with the quick growth of IoT devices, which are increasingly essential to Industry 4.0. These gadgets frequently function in challenging environments with little energy and processing power, leaving them open to cyberattacks and making it more difficult to implement intrusion detection systems (IDS) that work. In order to address this issue, this study presents a unique feature selection algorithm based on basic statistical methods and a lightweight intrusion detection system. This methodology improves performance and cuts training time by 27–63% for a variety of classifiers. By utilizing the most discriminative features, the suggested methods lower the computational overhead and improve the detection accuracy. The IDS achieved over 99.9% accuracy, precision, recall, and F1-Score on the dataset IoTID20, with consistent performance on the NSLKDD dataset.

Predicting personality traits from Arabic text: an investigation of textual and demographic features with feature selection analysis

Automatic personality recognition (APR) utilizes machine learning to predict personality traits from various data sources. This study aims to predict the big five personality traits from modern standard Arabic (MSA) texts, using both textual and demographic features. The “MSAPersonality” dataset is employed to conduct a comprehensive analysis of features and feature selection methods to evaluate their impact on APR model performance. We compared feature selection algorithms from the filter, wrapper, and embedded-based categories through a systematic experimental design that consisted of feature engineering, feature selection, and regression. This study showed that each trait was more accurately predicted using a distinct set of features. However, age and study level were the most common features among the five traits. Moreover, although there were no statistically significant differences in performance between the feature selection techniques, embedded-based methods offered the best compromise between performance, time, and interpretability. These findings contribute to the understanding of APR in general and among Arabic speakers.

MLAPW: A framework to assess the impact of feature selection and sampling techniques on anti-pattern prediction using WSDL metrics

MLAPW: A framework to assess the impact of feature selection and sampling techniques on anti-pattern prediction using WSDL metrics

Implementing cloud computing in drug discovery and telemedicine for quantitative structure-activity relationship analysis

This work aims to use cutting-edge machine learning methods to improve quantitative structure-activity relationship (QSAR) analysis, which is used in drug development and telemedicine. The major goal is to examine the performance of several predictive modeling approaches, including random forest, deep learning-based QSAR models, and support vector machines (SVM). It investigates the potential of feature selection techniques developed in chemoinformatics for enhancing model accuracy. The innovative aspect is using cloud computing resources to strengthen computational skills, allowing for managing massive amounts of chemical information. This strategy produces accurate and generalizable QSAR models. By using the cloud's scalability and constant availability, remote healthcare apps have a workable answer. The goal is to show how these methods may improve telemedicine and the drug development process. Utilizing cloud computing equips researchers with a flexible set of tools for precise and timely QSAR analysis, speeding up the discovery of bioactive chemicals for therapeutic use. This new method fits well with the dynamic nature of pharmaceutical study and has the potential to transform the way drugs are developed and delivered to patients via telemedicine.

Multi-Dimensional Features Extraction for Voice Pathology Detection Based on Deep Learning Methods.

Voice pathology detection is a rapidly evolving field of scientific research focused on the identification and diagnosis of voice disorders. Early detection and diagnosis of these disorders is critical, as it increases the likelihood of effective treatment and reduces the burden on medical professionals. The objective of this scientific paper is to develop a comprehensive model that utilizes various deep learning techniques to improve the detection of voice pathology. To achieve this, the paper employs several techniques to extract a set of sensitive features from the original voice signal by analyzing the time-frequency characteristics of the signal. In this regard, as a means of extracting these features, a state-of-the-art approach combining Gammatonegram features with Scalogram Teager_Kaiser Energy Operator (TKEO) features is proposed, and the proposed feature extraction scheme is named Combine Gammatonegram with (TKEO) Scalogram (CGT Scalogram). In this study, ResNet deep learning is used to recognize healthy voices from pathological voices. To evaluate the performance of the proposed model, it is trained and tested using the Saarbrucken voice database. In the end, the proposed system yielded impressive results with an accuracy of 96%, a precision of 96.3%, and a recall of 96.1% for binary classification and an accuracy of 94.4%, a precision of 94.5%, and a recall of 94% for multi-class. The results of the experiments demonstrate the effectiveness of the feature selection technique in maximizing the prediction accuracy in both binary and multi-class classifications.

Student Performance Prediction with Decision Tree Ensembles and Feature Selection Techniques

Student Performance Prediction with Decision Tree Ensembles and Feature Selection Techniques

A Survey on Online Payment Fraud Detection Techniques using Machine Learning Algorithms

Online transaction fraud detection has become a critical challenge with the rise of digital payment systems. This paper surveys various machine learning techniques employed in fraud detection, including Support Vector Machines (SVM-QUBO), Logistic Regression, K-Nearest Neighbors (KNN), Naive Bayes, Decision Trees, and Random Forest. The performance of each method is evaluated in terms of accuracy, precision, recall, and computational efficiency. The study also explores how different algorithms handle high-dimensional, imbalanced datasets and the impact of feature selection techniques. The results provide insights into the strengths and limitations of these algorithms, offering a comprehensive comparison for fraud detection in digital transactions

Review of filtering based feature selection for Botnet detection in the Internet of Things

Botnets are a major security threat in the Internet of Things (IoT), posing significant risks to user privacy, network availability, and the integrity of IoT devices. With the increasing availability of large datasets that contain hundreds or even thousands of variables, selecting the right set of features can be a challenging task. Feature selection is a critical step in developing effective machine learning-based botnet detection systems, as it enables the selection of a subset of features that are most relevant for detection. This paper provides a comprehensive review of filtering based feature selection techniques for botnet detection in IoT. It examines a range of filtering based techniques and evaluates their effectiveness in addressing the challenges and limitations of botnet detection in IoT. It aims to identify the gaps in the literature and areas for future research, and discuss the broader implications of findings for the field of IoT botnet detection. This review provides valuable insights and guidance for researchers and practitioners working on botnet detection in IoT, and highlights the importance of effective feature selection in developing robust and reliable detection systems.

An Analytical Benchmark of Feature Selection Techniques for Industrial Fault Classification Leveraging Time-Domain Features

An Analytical Benchmark of Feature Selection Techniques for Industrial Fault Classification Leveraging Time-Domain Features

A Robust Framework for Detecting Brute-Force Attacks through Deep Learning Techniques

A considerable concern arises with the precise identification of brute-force threats within a networked environment. It emphasizes the need for new methods, as existing ones often lead to many false alarms, as well as delays in real-time threat detection. To tackle these issues, this study proposes a novel intrusion detection framework that utilizes deep learning models for more accurate and efficient detection of brute-force attacks. The framework’s structure includes data collection and preprocessing components performed at the outset of the study using the CSE-CICIDS2018 dataset. The design architecture includes data collection and preprocessing steps. Feature extraction and selection techniques are employed to optimize data for model training. Further, after building the model, various attributes are extracted from the data from feature selection to be used in the training. Then, the construction of multiple architectures of deep learning algorithms, which include Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Long Short-Term Memory (LSTM) models. Evaluation results show CNN and LSTM achieved the highest accuracy of 99.995 Parsant and 99.99 Parsant respectively. It showcases its ability to detect complex attack patterns in network traffic. It indicates that the CNN network got the best optimum results with a test time of 9.94 seconds. This establishes CNN as an effective method, achieving high accuracy quickly. In comparison, we have surpassed the accuracy of current methods while addressing their weaknesses. The findings are consistent with the effectiveness of CNN in brute-force attack detection frameworks as a more accurate and faster alternative, increasing the capability of detecting intrusions on a network in real-time.

Predicting COPD Readmission: An Intelligent Clinical Decision Support System

Background: COPD is a chronic disease characterized by frequent exacerbations that require hospitalization, significantly increasing the care burden. In recent years, the use of artificial intelligence-based tools to improve the management of patients with COPD has progressed, but the prediction of readmission has been less explored. In fact, in the state of the art, no models specifically designed to make medium-term readmission predictions (2–3 months after admission) have been found. This work presents a new intelligent clinical decision support system to predict the risk of hospital readmission in 90 days in patients with COPD after an episode of acute exacerbation. Methods: The system is structured in two levels: the first one consists of three machine learning algorithms —Random Forest, Naïve Bayes, and Multilayer Perceptron—that operate concurrently to predict the risk of readmission; the second level, an expert system based on a fuzzy inference engine that combines the generated risks, determining the final prediction. The employed database includes more than five hundred patients with demographic, clinical, and social variables. Prior to building the model, the initial dataset was divided into training and test subsets. In order to reduce the high dimensionality of the problem, filter-based feature selection techniques were employed, followed by recursive feature selection supported by the use of the Random Forest algorithm, guaranteeing the usability of the system and its potential integration into the clinical environment. After training the models in the first level, the knowledge base of the expert system was determined on the training data subset using the Wang–Mendel automatic rule generation algorithm. Results: Preliminary results obtained on the test set are promising, with an AUC of approximately 0.8. At the selected cutoff point, a sensitivity of 0.67 and a specificity of 0.75 were achieved. Conclusions: This highlights the system’s future potential for the early identification of patients at risk of readmission. For future implementation in clinical practice, an extensive clinical validation process will be required, along with the expansion of the database, which will likely contribute to improving the system’s robustness and generalization capacity.

Leveraging Explainable AI for Dementia Classification: A Machine Learning Approach

In last two decades the growing life expectancy has spiralled the prevalence of neurogenerative disorder such as Dementia among elderly population posing a significant challenge to mental health globally. The dementia is generally characterized by progressive cognitive decline which it affects memory, thinking, behaviour, and the ability to perform everyday activities. Early diagnosis and intervention are crucial for improving patient outcomes and managing the societal burden of dementia. The primary aim of this study is to deploy various machine learning models such as Logistic Regression, K-Nearest Neighbour (KNN), Support Vector Machine (SVM), Decision Tree, Random Forest (RF). Extreme Gradient Boosting (XGBoost) for the classification of Dementia. The suitable features required for the classification is determined through efficient filter-based and wrapper-based feature selection techniques. The experimental evaluation of machine learning models is performed using standard metrics such as accuracy, precision, recall and F1 score. Furthermore, Explainable AI (XAI) techniques such as SHAP and LIME are employed to interpret the black-box nature of these models, offering transparency and insights into the contribution of individual features to the model predictions. The thorough evaluation of machine learning models exhibited that Random Forest outperforms other models with an accuracy of 96%, with CDR identified as a key predictor through XAI analysis.

A comparison between penalized logistic regressions and classification tree approaches

. The penalized models have been established to be necessary for classifying the emerging complex, high-dimensional datasets. One of these penalized models is the “modified elastic-net” (M-Enet). The special cases of M-Enet (ridge, bridge, Lasso, and elastic-net) can be produced based on its penalty function. By using the fact that the Fisher-Scoring (FS) and Newton-Raphson (NR) algorithms use different approaches to solve numerical problems but are the same in the canonical link, in order to ease implementation, we developed an FS algorithm for the computation of the parameter estimates of the M-Enet. A simulation study of the microbiome as complex, high-dimensional data shows that each special case of the M-Enet penalty function has superiority in the sense of model accuracy, sensitivity, and feature selection technique. A comparison study is performed between the classification of binary responses using M-Enet as a parametric method and the classification tree via “GUIDE” as a non parametric method. The statistical analysis of the numerical results reveals that the special cases of M-Enet models outperform other M-Enet models and GUIDE in terms of average prediction accuracy and sensitivity, with p-values of 0. 0362 and 0. 0214, respectively. Consequently, these models exhibit superior classification performance compared to other classifier models.

Advancing lung cancer diagnosis: Combining 3D auto-encoders and attention mechanisms for CT scan analysis

Objective The goal of this study is to assess the effectiveness of a hybrid deep learning model that combines 3D Auto-encoders with attention mechanisms to detect lung cancer early from CT scan images. The study aims to improve diagnostic accuracy, sensitivity, and specificity by focusing on key features in the scans. Materials and methods A hybrid model was developed that combines feature extraction using 3D Auto-encoder networks with attention mechanisms. First, the 3D Auto-encoder model was tested without attention, using feature selection techniques such as RFE, LASSO, and ANOVA. This was followed by evaluation using several classifiers: SVM, RF, GBM, MLP, LightGBM, XGBoost, Stacking, and Voting. The model's performance was evaluated based on accuracy, sensitivity, F1-Score, and AUC-ROC. After that, attention mechanisms were added to help the model focus on important areas in the CT scans, and the performance was re-assessed. Results The 3D Auto-encoder model without attention achieved an accuracy of 93% and sensitivity of 89%. When attention mechanisms were added, the performance improved across all metrics. For example, the accuracy of SVM increased to 94%, sensitivity to 91%, and AUC-ROC to 0.96. Random Forest (RF) also showed improvements, with accuracy rising to 94% and AUC-ROC to 0.93. The final model with attention improved the overall accuracy to 93.4%, sensitivity to 90.2%, and AUC-ROC to 94.1%. These results highlight the important role of attention in identifying the most relevant features for accurate classification. Conclusions The proposed hybrid deep learning model, especially with the addition of attention mechanisms, significantly improves the early detection of lung cancer. By focusing on key features in the CT scans, the attention mechanism helps reduce false negatives and boosts overall diagnostic accuracy. This approach has great potential for use in clinical applications, particularly in the early-stage detection of lung cancer.