Feature Selection Algorithm Research Articles

An automated ensemble approach using Harris Hawk optimization for visually evoked EEG signal classification.

Steady-state visually evoked potential is one of the active explorations in the brain-computer interface research. Electroencephalogram based brain computer interface studies have been widely applied to perceive solutions for real-world problems in the healthcare domain. The classification of externally bestowed visual stimuli of different frequencies on a human was experimented to identify the need of paralytic people. Although many classifiers are at the fingertip of machine learning technology, recent research has proven that ensemble learning is more efficacious than individual classifiers. Despite its efficiency, ensemble learning technology exhibits certain drawbacks like taking more time on selecting the optimal classifier subset. This research article utilizes the Harris Hawk Optimization algorithm to select the best classifier subset from the given set of classifiers. The objective of the research is to develop an efficient multi-classifier model for electroencephalogram signal classification. The proposed model utilizes the Boruta Feature Selection algorithm to select the prominent features for classification. Thus selected prominent features are fed into the multi-classifier subset which has been generated by the Harris Hawk Optimization algorithm. The results of the multi-classifier ensemble model are aggregated using Stacking, Bagging, Boosting, and Voting. The proposed model is evaluated against the acquired dataset and produces a promising accuracy of 96.1%, 98.7%, 91.91%, and 99.01% with the ensemble techniques respectively. The proposed model is also validated with other performance metrics such as sensitivity, specificity, and F1-Score. The experimental results show that the proposed model proves its supremacy in segregating the multi-class classification problem with high accuracy.

Adaptive chaotic dynamic learning-based gazelle optimization algorithm for feature selection problems

Feature Selection (FS) is considered a crucial procedure for eliminating unnecessary features from datasets. FS is considered a challenging problem that is difficult to solve efficiently due to its combinatorial nature. As the problem size increases, so does the computation time. Recently, researchers have been focusing on metaheuristic FS algorithms. Therefore, this paper introduces an adaptive chaotic dynamic gazelle optimization algorithm (ACD-GOA) with enhanced elite Strategy for FS problems. The ACD-GOA is a novel enhanced version of the recently published GOA, incorporating multiple improved strategies to enhance its search capabilities and convergence speed. The initialization phase and iteration initialization adopt the dynamic opposition learning strategy to avoid premature convergence. Additionally, several enhancement strategies are employed to improve the efficiency of the standard GOA. Adaptive inertia weight and sigmoid function are used to enhance search efficiency. Furthermore, enhanced elite and exchanging information strategies are implemented to maintain population diversity and avoid local solutions, respectively. Experimental evaluations are conducted using various functions, including the twelve CEC2022 standard functions and fourteen FS benchmark datasets. The performance of ACD-GOA is compared with several other metaheuristic algorithms. Statistical tests such as Friedman and Wilcoxon signed-rank tests are used to analyze the experimental data. The experimental results highlight the proposed algorithm’s exceptional capacity to overcome problems associated with local minima and expedite the convergence process. The suggested algorithm has been extensively compared to state-of-the-art algorithms, demonstrating significant breakthroughs. The reported accuracy of the proposed algorithm ranges from 0.78 to 1.00 with K-NN classifier.

Feature Selection and Classification with the Annealing Optimization Deep Learning for the Multi-Modal Image Processing

This paper investigates and compares various feature selection algorithms within the context of image processing across multiple datasets. The study evaluates Seahorse Annealing Optimization for Feature Selection (SAO-FS), Genetic Algorithms (GA), CNN + Feature Fusion Network, and Lasso Regression on distinct image datasets—medical images, satellite images, MRI scans, and microscopy images. Performance metrics including accuracy, precision, recall, and computational time are analyzed to assess the efficacy of each algorithm in optimizing feature subsets for classification tasks. SAO-FS demonstrates superior performance in medical image classification with an accuracy of 92.5%, showcasing its ability to achieve high precision and recall rates critical for medical diagnostics. GA proves effective for satellite imagery with an accuracy of 87.3%, while the CNN + Feature Fusion Network excels in MRI scans with 89.8% accuracy. Lasso Regression, though slightly less accurate at 85.6%, efficiently selects features for microscopy images within a shorter computational time. These findings highlight the strengths and trade-offs of each algorithm across different image processing domains, providing insights for selecting appropriate feature selection methods tailored to specific imaging applications.

Feature selection for multi-labeled data based on label enhancement technique and mutual information

In multi-label data, the importance of each label within the logical label vector varies for each sample, and there exist inherent correlations among the labels. However, the logical label vector fails to capture these nuances. Consequently, relying solely on this vector for feature selection in multi-label data results in underutilization of supervisory information. To address this issue, this paper introduces a novel label enhancement algorithm. This algorithm leverages neighborhood information derived from features to transform the logical label vector into a label distribution that effectively reflects label differences and correlations. Subsequently, we propose a feature selection algorithm tailored for multi-label data, which incorporates both the transformed label distribution and mutual information. This algorithm not only accounts for the mutual information between features and label distributions but also captures the mutual information among features themselves. Finally, we evaluate our proposed feature selection algorithm against five state-of-the-art multi-label feature selection algorithms on ten publicly available datasets. The experimental results reveal that our algorithm outperforms its competitors in six distinct evaluation metrics, achieving an average performance improvement of approximately 9%. This substantial enhancement underscores the efficacy of our algorithm in handling complex multi-label data.

Machine learning methods for unveiling the potential of antioxidant short peptides in goat milk-derived proteins during in vitro gastrointestinal digestion

Milk serves as an important dietary source of bioactive peptides, offering notable benefits to individuals. Among the antioxidant short peptides (di- and tripeptides) generated from gastrointestinal digestion are characterized by enhanced bioavailability and bioaccessibility, while assessing them individually presents a labor-intensive and expensive challenge. Based on 4 distinct types of AA descriptors (physicochemical, 3-dimensional structural, quantum, and topological attributes) and genetic algorithms for feature selection, 1 and 4 machine learning-predicted models separately for di- and tripeptides with 2,2-azinobis(3-ethylbenzothiazoline-6-sulfonic acid) diammonium salt radical scavenging capacity exhibited excellent fitting and prediction ability with random forest regression as machine learning algorithm. Intriguingly, the electronic properties of N-terminal AA were considered as only factor affecting the antioxidant capacity of dipeptides containing both tyrosine and tryptophan. Four peptides from the potential di- and tripeptides exhibited highly predicted values by the constructed predicted models. Subsequently, a total of 45 dipeptides and 52 tripeptides were screened by a customized workflow in goat milk during in vitro simulated digestion. In addition to 5 known antioxidant dipeptides, 9 peptides were quantified during digestion, exhibiting concentrations ranging from 0.04 to 1.78 mg L-1. Particularly noteworthy was the promising in vivo functionality of antioxidant dipeptides with N-terminal tyrosine, supported by in silico assays. Overall, this investigation explored crucial molecular properties influencing antioxidant short peptides and high-throughput screening potential peptides with antioxidant activity from goat milk aided by machine learning, thereby facilitating the discovery of novel functional peptides from milk-derived proteins and paving the way for understanding their metabolites during digestion.

A Novel Feature Selection Strategy Based on the Harris Hawks Optimization Algorithm for the Diagnosis of Cervical Cancer

Cervical cancer is the fourth most commonly diagnosed cancer and one of the leading causes of cancer-related deaths among females worldwide. Early diagnosis can greatly increase the cure rate for cervical cancer. However, due to the need for substantial medical resources, it is difficult to implement in some areas. With the development of machine learning, utilizing machine learning to automatically diagnose cervical cancer has currently become one of the main research directions in the field. Such an approach typically involves a large number of features. However, a portion of these features is redundant or irrelevant. The task of eliminating redundant or irrelevant features from the entire feature set is known as feature selection (FS). Feature selection methods can roughly be divided into three types, including filter-based methods, wrapper-based methods, and embedded-based methods. Among them, wrapper-based methods are currently the most commonly used approach, and many researchers have demonstrated that these methods can reduce the number of features while improving the accuracy of diagnosis. However, this method still has some issues. Wrapper-based methods typically use heuristic algorithms for FS, which can result in significant computational time. On the other hand, heuristic algorithms are often sensitive to parameters, leading to instability in performance. To overcome this challenge, a novel wrapper-based method named the Binary Harris Hawks Optimization (BHHO) algorithm is proposed in this paper. Compared to other wrapper-based methods, the BHHO has fewer hyper-parameters, which contributes to better stability. Furthermore, we have introduced a rank-based selection mechanism into the algorithm, which endows BHHO with enhanced optimization capabilities and greater generalizability. To comprehensively evaluate the performance of the proposed BHHO, we conducted a series of experiments. The experimental results show that the proposed BHHO demonstrates better accuracy and stability compared to other common wrapper-based FS methods on the cervical cancer dataset. Additionally, even on other disease datasets, the proposed algorithm still provides competitive results, proving its generalizability.

FuDNN-FOSMO: Early detection of chronic kidney disease using FuDNN with fractional order sequence optimization algorithm classifier

Early diagnosis and prediction of chronic kidney disease (CKD) progress within a given duration are critical to ensure personalized treatment, which could improve patients’ quality of life and prolong survival time. In this paper, we employ experiential analysis of ML techniques for classifying the kidney patient dataset as CKD or NOTCKD. Feature selection is a crucial step in ML, as it helps extract a subset of important features from the dataset. This process offers several benefits, including improved prediction accuracy, reduced model complexity, and enhanced interpretability. In this study, we utilized the Tabu Search with Boruta Algorithm feature selection technique, which leverages random shadow features and an ML model. Boruta compares the importance of each feature to that of the shadow features iteratively, categorizing features as confirmed, tentative, or rejected based on their significance. Ultimately, Boruta provides a subset of the most significant features from the dataset. Feature selection is performed using the Tabu Search with Boruta algorithm, and the model is developed using ML algorithms. The proposed model was validated on the UCI CKD dataset, achieving outstanding performance with 99.75 % accuracy. This study introduces a novel Deep learning network based on PointRCNN (FuDNN) Architecture for CKD early detection and prediction. More processing attributes of characteristics chosen to indicate a kidney issue are extracted by PointRCNN. Accuracy, sensitivity, specificity, AUC score and F1 score are the performance metrics for the suggested CKD classification approach. Additional experimental findings demonstrate that the suggested method produces a better categorization of CKD than the present state-of-the-art method.

A novel adaptive neighborhood rough sets based on sparrow search algorithm and feature selection

Neighborhood rough sets-based methods have been widely used for feature selection. However, the existing methods have some problems in neighborhood construction, such as the application of the same neighborhood radius for all samples. Thus, this paper proposed a novel adaptive neighborhood rough set model based on Sparrow Search Algorithm (SSA) to tackle the above problems, and applied the model to feature selection. First, we reconsidered the problems mentioned above from the viewpoint of optimization where the neighborhood radius of the target sample is considered as the solution to the optimization, and the maximum percentage of the label of the neighborhood formed is considered as the target to the optimization. Second, SSA is introduced to design the adaptive neighborhood construction to tackle the optimization problem where all candidate neighborhood radii of the target sample are considered as sparrows, the maximum and minimum distances between the target sample and other samples are considered as the search range, and the maximum label rate defined in this paper is considered as the search target. Then, a novel adaptive neighborhood rough set model is proposed by using the adaptive neighborhood construction. Third, we proposed a feature selection algorithm based on the adaptive neighborhood rough set model. Finally, the experimental results on seventeen datasets demonstrate the effectiveness of our algorithm. The running time of the proposed algorithm is at least one time less than classical algorithms under the condition that the classification performance is better, the accuracy increases 3% and the balanced accuracy increases 4%.

Fusing 1H NMR and Raman experimental data for the improvement of wine recognition models

The present study proposes the development of new wine recognition models based on Artificial Intelligence (AI) applied to the mid-level data fusion of 1H NMR and Raman data. In this regard, a supervised machine learning method, namely Support Vector Machines (SVMs), was applied for classifying wine samples with respect to the cultivar, vintage, and geographical origin. Because the association between the two data sources generated an input space with a high dimensionality, a feature selection algorithm was employed to identify the most relevant discriminant markers for each wine classification criterion, before SVM modeling. The proposed data processing strategy allowed the classification of the wine sample set with accuracies up to 100% in both cross-validation and on an independent test set and highlighted the efficiency of 1H NMR and Raman data fusion as opposed to the use of a single-source data for differentiating wine concerning the cultivar and vintage.

Feature selection for label distribution learning based on the statistical distribution of data and fuzzy mutual information

Label distribution learning (LDL) is an emerging framework in machine learning. Fuzzy mutual information is mutual information under a fuzzy environment and plays an important role in handling uncertainty. This paper explores feature selection for LDL data based on the statistical distribution of data and fuzzy mutual information. The similarity between the feature values in the feature space is first defined by means of the statistical distribution of the data, and a threshold is introduced to control the similarity. Then, the fuzzy similarity relation for each feature subset is established via the similarity. This method utilizes adjustable fuzzy similarity radii to establish a fuzzy similarity relation and improve the classification ability of the data. The decision relation in the label space is then presented, and the decision class of each sample is constructed. Subsequently, two feature selection algorithms based on fuzzy mutual information are designed to remove the irrelevant features by employing a strategy that considers the correlation between the features and labels as well as the redundancy between the features in the LDL data. Finally, the experimental results show that the designed algorithms can effectively measure the uncertainty of LDL data and outperform four state-of-the-art feature selection algorithms. Specifically, our algorithms, LDFM and LDFMR, demonstrate their superiority by achieving overall average ranking improvements of 63.64% and 58.52%, respectively, across six evaluation metrics compared to the other four algorithms.

Bearing fault diagnosis using multiple feature selection algorithms with SVM

Bearing fault diagnosis using multiple feature selection algorithms with SVM

Using Optuna and Genetic Algorithm together facilitate the classification of plant leaf diseases

The identification and severity assessment of plant leaf diseases is crucial to food security and sustainable agriculture. This study shows an innovative way to improve plant leaf disease detection. The recommended method uses Optuna for parameter optimization and the Genetic Algorithm for feature selection to improve plant leaf disease identification. We do this to improve diagnosis accuracy. This method improves classification accuracy and is called ECPLDD-OGA. Modern hyper parameter optimization framework Optuna is employed. This allows classification model parameters to be fine-tuned. A systematic feature selection method is the Genetic Algorithm. It finds the most useful characteristics in the input dataset. By applying the algorithm on the data. By facilitation, the iterative process helps create a simplified and meaningful subset of features. Contrary to parameter tinkering and feature selection, empirical data suggests that utilizing Optuna and the Genetic Algorithm simultaneously improves disease identification. The updated model recognizes sick plants more accurately and generalizes better. Optimization enabled both gains. The usage of this technology can improve agricultural operations and reduce crop losses by increasing productivity. The present ECPLDD-OGA technique helps integrate hyper parameter tweaking and feature selection into machine learning-based agricultural applications.

A Study on Comparative Analysis of Feature Selection Algorithms for Students Grades Prediction

A Study on Comparative Analysis of Feature Selection Algorithms for Students Grades Prediction

Pavement raveling inspection using a new image texture-based feature set and artificial intelligence

Enhancing pavement distress inspection is crucial within a pavement management system. Recognizing the advantages of automated pavement condition inspection, considerable efforts have been invested in developing more efficient systems for this purpose. Raveling, a common asphalt pavement distress, significantly impacts both passenger comfort and pavement durability. Given its connection to pavement texture and the limited effectiveness of earlier inspection systems, improving automatic raveling inspection has become a challenging endeavor. This study proposes an image-based system for raveling detection, employing a new image processing-based feature set. This approach generates enhanced inputs for artificial intelligence models, thereby improving the efficiency of raveling inspection while optimizing resource allocation. To accomplish this, an image dataset is created by combining newly collected images with an existing collection. Novel combinations of features are then engineered to extract meaningful texture-based image features. Subsequently, three feature selection algorithms are employed to identify the most pivotal features. Machine Learning (ML) models are then trained for effective raveling detection based on the proposed feature set, and their performance is evaluated across various performance indices. The achieved performance metrics, reaching nearly 96 % on the test datasets, underscore the efficacy of the proposed method in automating the inspection of texture-based pavement distresses.

A multi-scale semantic feature fusion method for remote sensing crop classification

The efficient and accurate acquisition of crop spatial planting structure information is of great significance to ensure national food security. Current crop classification studies have still not achieved optimal results in accurate crop classification identification due to the complex heterogeneous details of clear images and insufficient analysis and utilization of information. This study takes the Yuncheng-Linfen Basin in Shanxi, China as the study area and uses the ReliefF-RFE feature selection algorithm to achieve data dimensionality reduction. Contextual semantic feature aggregation, spatial channel attention mechanism, and hierarchical refinement strategy are used to fuse the multi-scale feature information in the extracted remote sensing images from the top down and bottom up. This study shows that 1) the ReliefF-RFE algorithm can effectively reduce the dimensionality of 224-dimensional multisource features and generate a 31-dimensional optimal feature space subset with strong interpretation to improve the computational efficiency of the model; 2) the multi-scale feature fusion model constructed in this study is better than UNet, ResNet, DeepLabv3+, HRNet and Swin Transformer in classification recognition of winter wheat and corn, with an overall accuracy of 95.76% and 93.44%; 3) A multiple feature fusion strategy designed through ablation experiments was evaluated to evaluate the reasonableness of each module and validate the irreplaceability and superiority of the model for improving crop accuracy in complex and heterogeneous terrain; 4) the multi-scale feature fusion model converts deep abstract features into local detail information by cross-scale contextual aggregation of deep and shallow multi-scale features, it obtains good extraction effect on fragmented small plots, and the reduction of local fine details of features is significantly better than other models. The multi-scale feature fusion classification model constructed in this study fully considers the interpretability of the input features and fuses the semantic features at different scales to eliminate the “semantic gap”, which shows that the model has strong stability and robustness and provides a reference for fast and accurate mapping of Sentinel-2 images of crops in large areas.

Feature selection considering feature relevance, redundancy and interactivity for neighbourhood decision systems

Feature selection is an effective method to simplify data analysis and obtain key features, which improves the accuracy and generalization ability of classifiers. Neighbourhood rough set is a typical granular computing model that enables data analysis at different granularity by setting different neighbourhood radii. In recent years, feature selection based on neighbourhood rough sets has received widespread attention from researchers. However, these models still have the following shortcomings. On the one hand, they rarely simultaneously consider the relevance, the redundancy and interactivity among features, making it difficult to accurately depict classification information in the data. On the other hand, there is still a lack of effective uncertainty measures to evaluate the classification ability of feature subsets. For these reasons, this article proposes a feature selection model that considers the relevance, redundancy, and interactivity of features. Firstly, the relevance, the redundancy and interactivity between features, are defined based on the neighbourhood, respectively. Secondly, a new feature evaluation function (MMM function) is proposed based on the principles of maximum relevance, minimum redundancy, and maximum interactivity. Finally, from the perspective of maintaining classification ability, a heuristic feature selection algorithm (FS_CRRI) was designed. To evaluate the performance of the proposed algorithm, it was compared with some representative feature selection algorithms on twelve common datasets. The experimental results on three different classifiers indicate that the FS_CRRI algorithm can effectively reduce data and achieve higher classification performance.

Improved prediction of anti-angiogenic peptides based on machine learning models and comprehensive features from peptide sequences

Angiogenesis is a key process for the proliferation and metastatic spread of cancer cells. Anti-angiogenic peptides (AAPs), with the capability of inhibiting angiogenesis, are promising candidates in cancer treatment. We propose AAPL, a sequence-based predictor to identify AAPs with machine learning models of improved prediction accuracy. Each peptide sequence was transformed to a vector of 4335 numeric values according to 58 different feature types, followed by a heuristic algorithm for feature selection. Next, the hyperparameters of six machine learning models were optimized with respect to the feature subset. We considered two datasets, one with entire peptide sequences and the other with 15 amino acids from peptide N-termini. AAPL achieved Matthew’s correlation coefficients of 0.671 and 0.756 for independent tests based on the two datasets, respectively, outperforming existing predictors by a range of 5.3% to 24.6%. Further analyses show that AAPL yields higher prediction accuracy for peptides with more hydrophobic residues, and fewer hydrophilic and charged residues. The source code of AAPL is available at https://github.com/yunzheng2002/Anti-angiogenic.

Enhancing natural disaster analysis and waste classification: a novel VGG-FL approach.

The study of natural disasters is a crucial field that involves analyzing the occurrence, impact, and aftermath of various natural hazards that can cause significant harm to communities and the environment. Efficient waste management and environmental protection require proper classification of waste. Analyzing natural disasters and categorizing waste can be a time-consuming task, and conventional methods often struggle with it. However, a new approach called Visual Geometry Group with Federated Learning (VGG-FL) has been introduced to address these challenges. This methodology uses the golden search optimization (GSO) algorithm for feature selection and leverages VGG with federated learning for feature extraction and classification. To test the effectiveness of this method, a disaster image dataset was used to train the VGG-FL model. The results showed that the VGG-FL model attained exceptional accuracy in discerning and categorizing various disaster scenarios. The waste classification dataset simultaneously trains the VGG-FL model to categorize waste based on its characteristics and potential hazards. To measure the model's performance, several evaluation metrics such as accuracy, specificity, precision, F1-score, and recall are utilized to assess the effectiveness of the proposed VGG-FL method. These results are then compared with existing methodologies. The VGG-FL method performs exceptionally well, achieving 98.52% accuracy, 97.48% precision, 97.83% recall, 97.58% F1-score, and 97.12% specificity. These experimental findings demonstrate the efficacy of the VGG-FL method in analyzing natural disasters and classifying waste materials.

Dung Beetle Optimizer Algorithm and Machine Learning-Based Genome Analysis of Lactococcus lactis: Predicting Electronic Sensory Properties of Fermented Milk.

In the global food industry, fermented dairy products are valued for their unique flavors and nutrients. Lactococcus lactis is crucial in developing these flavors during fermentation. Meeting diverse consumer flavor preferences requires the careful selection of fermentation agents. Traditional assessment methods are slow, costly, and subjective. Although electronic-nose and -tongue technologies provide objective assessments, they are mostly limited to laboratory environments. Therefore, this study developed a model to predict the electronic sensory characteristics of fermented milk. This model is based on the genomic data of Lactococcus lactis, using the DBO (Dung Beetle Optimizer) optimization algorithm combined with 10 different machine learning methods. The research results show that the combination of the DBO optimization algorithm and multi-round feature selection with a ridge regression model significantly improved the performance of the model. In the 10-fold cross-validation, the R2 values of all the electronic sensory phenotypes exceeded 0.895, indicating an excellent performance. In addition, a deep analysis of the electronic sensory data revealed an important phenomenon: the correlation between the electronic sensory phenotypes is positively related to the number of features jointly selected. Generally, a higher correlation among the electronic sensory phenotypes corresponds to a greater number of features being jointly selected. Specifically, phenotypes with high correlations exhibit from 2 to 60 times more jointly selected features than those with low correlations. This suggests that our feature selection strategy effectively identifies the key features impacting multiple phenotypes, likely originating from their regulation by similar biological pathways or metabolic processes. Overall, this study proposes a more efficient and cost-effective method for predicting the electronic sensory characteristics of milk fermented by Lactococcus lactis. It helps to screen and optimize fermenting agents with desirable flavor characteristics, thereby driving innovation and development in the dairy industry and enhancing the product quality and market competitiveness.

Development of an intrusion detection system using mayfly feature selection and artificial neural network algorithms

Protecting the privacy and confidentiality of information and devices in computer networks requires reliable methods of intrusion detection. However, effective intrusion detection is made more difficult by the enormous dimensions of data available in computer networks. To boost intrusion detection classification performance in computer networks, this study developed a feature selection mode for the classification task. The proposed model utilized the Mayfly feature selection algorithm and ANN as the classifiers. The model was also tested without a mayfly algorithm. The model's efficacy was determined through a comparison of its accuracy, specificity, precision, sensitivity, and F1 score. The experimental outcomes revealed that the proposed model is more efficient than existing models based on the performance evaluation and the CIC-IDS 2017 dataset employed in this research. Accuracy scores of 99.94% (using Data+mayfly+ANN) and 90.17% (using Data+ANN) were attained after experimentation. In comparison to existing models, the proposed model yielded better results in terms of accuracy, sensitivity, specificity, and F1-score metrics. The model's sturdiness can be attributed to the use of mayfly techniques, which harness the strength in PSO, GA and FA for selecting optimal feature subsets. The results of this research provide a reliable dimensionality reduction model that may be used in the field of computer networks for intrusion detection and enhancement of security in computer networking environments.