Ensemble Feature Selection Algorithm Research Articles

An ensemble technique to predict Parkinson's disease using machine learning algorithms

Parkinson's Disease (PD) is a progressive neurodegenerative disorder affecting motor and non-motor symptoms. Its symptoms develop slowly, making early identification difficult. Machine learning has a significant potential to predict Parkinson's disease on features hidden in voice data. This work aimed to identify the most relevant features from a high-dimensional dataset, which helps accurately classify Parkinson's Disease with less computation time. Three individual datasets with various medical features based on voice have been analyzed in this work. An Ensemble Feature Selection Algorithm (EFSA) technique based on filter, wrapper, and embedding algorithms that pick highly relevant features for identifying Parkinson's Disease is proposed, and the same has been validated on three different datasets based on voice. These techniques can shorten training time to improve model accuracy and minimize overfitting. We utilized different ML models such as K-Nearest Neighbors (KNN), Random Forest, Decision Tree, Support Vector Machine (SVM), Bagging Classifier, Multi-Layer Perceptron (MLP) Classifier, and Gradient Boosting. Each of these models was fine-tuned to ensure optimal performance within our specific context. Moreover, in addition to these established classifiers, we proposed an ensemble classifier is found on a high optimal majority of the votes. Dataset-I achieves classification accuracy with 97.6 %, F1-score 97.9 %, precision with 98 % and recall with 98 %. Dataset-II achieves classification accuracy 90.2 %, F1-score 90.2 %, precision 90.2 %, and recall 90.5 %. Dataset-III achieves 83.3 % accuracy, F1-score 83.3 %, precision 83.5 % and recall 83.3 %. These results have been taken using 13 out of 23, 45 out of 754, and 17 out of 46 features from respective datasets. The proposed EFSA model has performed with higher accuracy and is more efficient than other models for each dataset.

An ensemble learning-based feature selection algorithm for identification of biomarkers of renal cell carcinoma.

Feature selection plays a crucial role in classification tasks as part of the data preprocessing process. Effective feature selection can improve the robustness and interpretability of learning algorithms, and accelerate model learning. However, traditional statistical methods for feature selection are no longer practical in the context of high-dimensional data due to the computationally complex. Ensemble learning, a prominent learning method in machine learning, has demonstrated exceptional performance, particularly in classification problems. To address the issue, we propose a three-stage feature selection algorithm framework for high-dimensional data based on ensemble learning (EFS-GINI). Firstly, highly linearly correlated features are eliminated using the Spearman coefficient. Then, a feature selector based on the F-test is employed for the first stage selection. For the second stage, four feature subsets are formed using mutual information (MI), ReliefF, SURF, and SURF* filters in parallel. The third stage involves feature selection using a combinator based on GINI coefficient. Finally, a soft voting approach is proposed to employ for classification, including decision tree, naive Bayes, support vector machine (SVM), k-nearest neighbors (KNN) and random forest classifiers. To demonstrate the effectiveness and efficiency of the proposed algorithm, eight high-dimensional datasets are used and five feature selection methods are employed to compare with our proposed algorithm. Experimental results show that our method effectively enhances the accuracy and speed of feature selection. Moreover, to explore the biological significance of the proposed algorithm, we apply it on the renal cell carcinoma dataset GSE40435 from the Gene Expression Omnibus database. Two feature genes, NOP2 and NSUN5, are selected by our proposed algorithm. They are directly involved in regulating m5c RNA modification, which reveals the biological importance of EFS-GINI. Through bioinformatics analysis, we shows that m5C-related genes play an important role in the occurrence and progression of renal cell carcinoma, and are expected to become an important marker to predict the prognosis of patients.

A feature selection method based on multiple feature subsets extraction and result fusion for improving classification performance

Directly applying high-dimensional data to machine learning leads to dimensionality disasters and may induce model overfitting. Feature selection can effectively reduce feature size. However, a single feature selection algorithm has instability and poor generalization ability problems. The ensemble feature selection algorithm is complex to find a suitable feature subset aggregation strategy. To solve these two problems, we propose a feature selection method based on multiple feature subsets extraction and result fusion (FSM). Generate multiple feature subsets to improve stability. This method uses mutual information to mine the relationship between features and categories. Fast non-dominated sorting combines this correlation to distribute similar features in the same layer. A layer optimization algorithm is proposed to combine the layered features to generate multiple different feature subsets. To evaluate the excellence of feature subsets, FSM uses precision, recall, and F-Score comprehensively to assess and remove ineffective feature subsets. The idea of fusion is put on the output of the results. Multiple superior feature subsets train various classifiers. The results of numerous classifiers are fused as the final output according to the voting method, simplifying the ensemble feature selection method’s aggregation process. Experiments on 20 well-known datasets show that FSM can effectively reduce the data dimension and improve the classification performance compared with the original datasets. FSM performs well in classification performance and efficiency compared with other dimensionality reduction algorithms.

Research on Feature Extraction of Underwater Acoustic Target Radiation Noise Based on Machine Learning Algorithm

Underwater acoustic target recognition is a very important technology in the field of underwater acoustics, with great economic and military value. Feature extraction technology for underwater acoustic target radiation noise signals is the key to achieving acoustic target recognition. This study aims at the feature extraction task of acoustic targets and extracts 10 types of 252-dimensional feature vectors from three domains: time domain, frequency domain, and auditory domain. Through 7 machine learning algorithms for classification and recognition experiments, the experimental results show that the recognition performance of the ensemble classifier is much better than that of a single classifier. For different types of features, this study combines three ensemble learning algorithms and feature selection algorithms to select the original 252-dimensional features. The feature selection experiment shows that the wrapper feature selection algorithm has the best effect, and the feature vector dimension can be reduced to 40 dimensions. The recognition accuracy rate is not less than 92.8%, which provides feature extraction guidance for acoustic target recognition based on feature extraction.

An adaptive and enhanced framework for daily stock market prediction using feature selection and ensemble learning algorithms

ABSTRACT Even a slight increase in accuracy when predicting the direction of stock movements can have a significant impact on the rate of returns. However, determining the most suitable variables, methods, and parameters to predict price changes is extremely challenging due to the multitude of variables influencing these changes. This paper presents an innovative prediction framework that combines ensemble learning and feature selection algorithms to effectively capture daily stock movements. The study focuses on predicting the change between the opening and closing prices of the subsequent day and employs a daily sliding window cross-validation methodology. The framework comprises fourteen variable groups encompassing a range of financial and operational indicators. Experimental findings indicate that a competitive performance was achieved for stocks within the Borsa Istanbul 30 index. Light Gradient Boosting Machines and Shapley Additive Explanations emerges as the optimal model combination and exhibits superior performance compared to a buy-and-hold strategy.

Multiparticipant Federated Feature Selection Algorithm With Particle Swarm Optimization for Imbalanced Data Under Privacy Protection

In real world, data describing the same learning task may be distributed in different institutions (called participants), and these participants cannot share their own data due to the need of privacy protection. How to select an optimal feature subset on the basis of ensuring the data privacy of each participant is an important and challenging topic. At the same time, data on part participants show imbalanced or even miss some classes, further increasing the difficulty of solving this kind of problem. In view of this, by introducing a trusted third party to process and integrate optimal information from participants, this paper proposes a multi-participant federated evolutionary feature selection algorithm for imbalanced data under privacy protection. Firstly, a multi-level joint sample filling strategy with multiple participants (called sampling-rough selection-fine tuning strategy) is proposed to fill imbalanced or empty classes on each participant while ensuring data privacy. Then, a federated evolutionary feature selection method based on particle swarm optimization (PSO) is proposed by periodically sharing the optimal feature subsets obtained by PSOs on participants. Finally, the proposed algorithm is applied to 15 test datasets, and compared with several typical imbalanced feature selection algorithms and two kinds of ensemble feature selection algorithms. The experimental results show that the proposed algorithm can significantly promote the ability of each participant on processing imbalanced datasets, and improve the classification accuracy of obtained feature subsets while protecting the participants' data privacy.

A novel approach for flood hazard assessment using hybridized ensemble models and feature selection algorithms

Identifying flood-prone regions is critical for effective management of flood hazards as floods are among the most devastating natural disasters globally. However, accurate modeling and prediction of floods are challenging due to their complexity. The current research has proposed a novel approach for Flood Hazard (FH) prediction using hybrid Machine Learning (ML) models that integrate ensemble ML models with several Feature Selection (FS) algorithms. An optimum set of Flood Influential Factors (FIFs) was determined using the Simulated Annealing (SA) and Information Gain (IG) FS algorithms. The ensemble ML models employed include AdaboostM1 (ABM), Boosted Generalized Linear Model (BGLM), and Stochastic Gradient Boosting (SGB) algorithms. In addition, the hyper-parameters of the hybrid models were optimized using the random search (RS) method and repeated cross-validation technique. The proposed hybrid models were trained using flood inventory map and FIFs obtained from a spatial database. The results verified that the SA and IG algorithms detect 9 and 13 factors as FIFs in the FH assessment, respectively. Moreover, rainfall, distance to river, altitude, and lithology FIFs have a greater impact than the other factors in the Sardabroud watershed, Mazandaran Province, Iran. Several robust indicators, such as the area under curve (AUC) in relative operating characteristic (ROC) curves and statistical measurements, were employed to assess the robustness of hybrid models. SA-ABM model had the highest AUC value (0.983), while IG-ABM, SA-BGLM, SA-SGB, IG-BGLM, and IG-SGB had lower values (0.952 to 0.973). Finally, the SA-ABM hybrid model classified 27% of the study area as having a high hazard of floods.

Legitimate and spam SMS classification employing novel Ensemble feature selection algorithm

Legitimate and spam SMS classification employing novel Ensemble feature selection algorithm

Supervised learning applied to classifying fallers versus non-fallers among older adults with cancer

IntroductionSupervised machine learning approaches are increasingly used to analyze clinical data, including in geriatric oncology. This study presents a machine learning approach to understand falls in a cohort of older adults with advanced cancer starting chemotherapy, including fall prediction and identification of contributing factors. Materials and MethodsThis secondary analysis of prospectively collected data from the GAP 70+ Trial (NCT02054741; PI: Mohile) enrolled patients aged ≥70 with advanced cancer and ≥ 1 geriatric assessment domain impairment who planned to start a new cancer treatment regimen. Of ≥2000 baseline variables (“features”) collected, 73 were selected based on clinical judgment. Machine learning models to predict falls at three months were developed, optimized, and tested using data from 522 patients. A custom data preprocessing pipeline was implemented to prepare data for analysis. Both undersampling and oversampling techniques were applied to balance the outcome measure. Ensemble feature selection was applied to identify and select the most relevant features. Four models (logistic regression [LR], k-nearest neighbor [kNN], random forest [RF], and MultiLayer Perceptron [MLP]) were trained and subsequently tested on a holdout set. Receiver operating characteristic (ROC) curves were generated and area under the curve (AUC) was calculated for each model. SHapley Additive exPlanations (SHAP) values were utilized to further understand individual feature contributions to observed predictions. ResultsBased on the ensemble feature selection algorithm, the top eight features were selected for inclusion in the final models. Selected features aligned with clinical intuition and prior literature. The LR, kNN, and RF models performed equivalently well in predicting falls in the test set, with AUC values 0.66–0.67, and the MLP model showed AUC 0.75. Ensemble feature selection resulted in improved AUC values compared to using LASSO alone. SHAP values, a model-agnostic technique, revealed logical associations between selected features and model predictions. DiscussionMachine learning techniques can augment hypothesis-driven research, including in older adults for whom randomized trial data are limited. Interpretable machine learning is particularly important, as understanding which features impact predictions is a critical aspect of decision-making and intervention. Clinicians should understand the philosophy, strengths, and limitations of a machine learning approach applied to patient data.

Machine Learning-Based Ensemble Band Selection for Early Water Stress Identification in Groundnut Canopy Using UAV-Based Hyperspectral Imaging

This paper presents the early identification of water stress in groundnut canopy using unmanned aerial vehicle (UAV) based hyperspectral imaging (in 385-1020 nm) and machine learning (ML) techniques. An efficient hyperspectral imaging (HSI) data analysis pipeline was presented which includes image quality assessment, denoising, band selection, and classification. A novel ML-based ensemble feature selection (FS) algorithm has been proposed for optimal water stress sensitive waveband selection. The data analysis pipeline and the selected bands were validated on HSI data acquired at two different water stress levels. Wavelengths 515.05, 552.16, 711.92, 724.75, and 931.92 nm were identified as optimal water stress sensitive bands in groundnut canopy, using which we could identify early stress with 96.46% accuracy. The proposed data analysis pipeline and ensemble FS algorithm will benefit crop phenotyping applications such as early abiotic stress detection.

DEVELOPMENT OF A NEURAL NETWORK STRUCTURE FOR INTRUSION DETECTION ANALYSIS

The right security solutions in the information and communication world are critical to network security by providing real-time network protection against network vulnerabilities and data usage. An effective intrusion detection strategy is able to use a holistic approach to protect critical systems from unauthorized access or attacks. The paper examines the latest scientific achievements and research related to the analysis of detection of network intrusions using machine learning (ML) methods. The article describes a complex security solution based on machine learning (ML) for network intrusion detection using a complex controlled ML structure and ensemble feature selection methods. In addition, a comparative analysis of several MH models and function selection methods is provided. The article develops a general mechanism for detecting and achieving higher accuracy with a minimum frequency of false positive results (FPR). The paper uses datasets and the results show that the detection model can successfully identify 99.3% of intrusions with the lowest error rate of 0.5%, which shows better performance compared to existing solutions. The article combines the selection of ensemble functions and ensemble machine learning approaches as a detection mechanism in SBB to detect network anomalies. An experimental study was conducted with feature sets obtained from nine feature selection methods, and then these feature sets were combined to obtain the minimum number of features using majority voting. A comparative analysis of sets of functions was carried out. Controlled methods are used, which are more efficient with a balanced data set. To make the training dataset balanced, the data type (benign or attacking) with the minimum number of data instances in that training dataset was first selected. An ensemble feature selection and ensemble classification algorithm is implemented to improve the overall performance of the proposed machine learning model. Prospects for the development of further research are proposed.

BIO-INSPIRED ENSEMBLE FEATURE SELECTION (BIEFS) AND ENSEMBLE MULTIPLE DEEP LEARNING (EMDL) CLASSIFIER FOR BREAST CANCER DIAGNOSIS

Breast cancer is a major disease diagnosed in women, affecting 2.1 million women every year, and is the reason for most cancer-related mortality in women, as per the World Health Organization (WHO). For cancer researchers, accurately forecasting the life expectancy of breast cancer patients is a serious challenge. Bio-Inspired Ensemble Feature Selection (BIEFS) algorithm is introduced uses three feature selection methods such as Adaptive Mutation Enhanced Elephant Herding Optimization (AMEHO), Adaptive Mutation Butterfly Optimization Algorithm (AMBOA), and Adaptive Salp Swarm Algorithm (ASSA) and integrates their normalized outputs for getting quantitative ensemble importance. BIEFS algorithm depends upon the aggregation of multiple FS techniques by Pearson Correlation Coefficient (PCC). Ensemble Multiple Deep Learning (EMDL) classifier is introduced which combines several individual models such as AdaBoost-Convolutional Neural Network (A-CNN), Long Short-Term Memory Network (LSTM), and Deep Auto-Encoder (DAE) is introduced to obtain better generalization performance. Then, these three classifiers such as A-CNN, LSTM, and DAE are used for ensemble classification using a Weight Majority Voting (WMV) mechanism. Wisconsin Diagnosis Breast Cancer (WOBC), and Wisconsin Diagnosis Breast Cancer (WDBC) are collected from University of California, Irvine (UCI) repository for experimentation. Evaluation metrics like Precision, Recall, F-measure, Accuracy, and Area Under Curve (AUC) are used to compare the results of proposed system and existing classifiers. All experiments are executed within a simulation environment and conducted in MATrix LABoratory R 2014 a (MATLAB 2014a) tool. The proposed EMDL classifier is showing better performance as compared to the traditional classification models.

Bio-inspired ensemble feature selection (biefs) and kernel extreme learning machine classifier for breast cancer diagnosis

Breast cancer is a major disease identified in women, affecting 2.1 million women every year, and is the reason for most cancer-related mortality in women, as per the World Health Organization (WHO). For cancer researchers, accurately forecasting the life expectancy of breast cancer patients is a serious challenge. Machine Learning (ML) has acknowledged much interest in the hope of providing correct results, but due to irrelevant features, its modelling methodologies and prediction performance are still a difficulty. To solve this issue, Feature Selection (FS) was also done to verify whether comparable accuracy can be achieved even with lesser number of features or not. Bio-Inspired Ensemble Feature Selection (BIEFS) algorithm is introduced aimed at selecting a subset of features that increase the prediction performance of subsequent classification models while also simplifying their interpretability. BIEFS algorithm uses three feature selection methods such as Adaptive Mutation Enhanced Elephant Herding Optimization (AMEHO), Adaptive Mutation Butterfly Optimization Algorithm (AMBOA), and Adaptive Salp Swarm Algorithm (ASSA) and integrates their normalized outputs for getting quantitative ensemble importance. BIEFS algorithm depends upon the aggregation of multiple FS techniques by Pearson Correlation Coefficient (PCC).This BIEFS algorithm can improve the accuracy of analysis (benign and malignant).

R-GEFS: Condorcet Rank Aggregation with Graph Theoretic Ensemble Feature Selection Algorithm for Classification

In the last few years, ensemble learning has received more interest primarily for the task of classification. It is based on the postulation that combining the output of multiple experts is better than the output of any individual expert. Ensemble feature selection may improve the performance of the learning algorithms and has the ability to obtain more stable and robust results. However, during the process of feature aggregation and selection, selected feature subset may contain high levels of inter-feature redundancy. To address this issue, a novel algorithm based on feature rank aggregation and graph theoretic technique for ensemble feature selection (R-GEFS) with the fusion of Pearson and Spearman correlation metrics is proposed. The method works by aggregation of the profile of preferences of five feature rankers as the base feature selectors. Then similar features are grouped into clusters using graph theoretic approach. The most representative feature strongly co-related to target decision classes is drawn from each cluster. The efficiency and effectiveness of the R-GEFS algorithm are evaluated through an empirical study. Extensive experiments on 15 diverse benchmark datasets are carried out to compare R-GEFS with seven state-of-the-art feature selection models with respect to four popular classifiers, namely decision tree, k nearest neighbor, random forest, and support vector machine. The proposed method turns out to be effective by selecting smaller feature subsets with lesser computational complexities and it assists in increasing the classification accuracy.

Optimization-Based Ensemble Feature Selection Algorithm and Deep Learning Classifier for Parkinson's Disease.

PD (Parkinson's Disease) is a severe malady that is painful and incurable, affecting older human beings. Identifying PD early in a precise manner is critical for the lengthened survival of patients, where DMTs (data mining techniques) and MLTs (machine learning techniques) can be advantageous. Studies have examined DMTs for their accuracy using Parkinson's dataset and analyzing feature relevance. Recent studies have used FMBOAs for feature selections and relevance analyses, where the selection of features aims to find the optimal subset of features for classification tasks and combine the learning of FMBOAs. EFSs (ensemble feature selections) are viable solutions for combining the benefits of multiple algorithms while balancing their drawbacks. This work uses OBEFSs (optimization-based ensemble feature selections) to select appropriate features based on agreements. Ensembles have the ability to combine results from multiple feature selection approaches, including FMBOAs, LFCSAs (Lévy flight cuckoo search algorithms), and AFAs (adaptive firefly algorithms). These approaches select optimized feature subsets, resulting in three feature subsets, which are subsequently matched for correlations by ensembles. The optimum features are generated by OBEFSs the trained on FCBi-LSTMs (fuzzy convolution bi-directional long short-term memories) for classifications. This work's suggested model uses the UCI (University of California-Irvine) learning repository, and the methods are evaluated using LOPO-CVs (Leave-One-Person-Out-Cross Validations) in terms of accuracies, F-measure values, and MCCs (Matthews correlation coefficients).

An ensemble method for feature selection and an integrated approach for mitigation of distributed denial of service attacks

AbstractDistributed denial of service attacks (DDoS) penetrate numerous computer system and implant malicious codes thereby making them ready for launching a collaborative attack. These attacks paralyze the target system mainly the web server by exhausting their network resources of the target server. The threats posed by DDoS attacks on the Internet demands for effective detection and mitigation methods of these attacks. In the paper, we proposed an integrated method for detection and mitigation of DDoS attack using machine learning and a line of defenses respectively. The detection phase consists of feature selection through ensemble feature selection algorithm and classification using machine learning algorithm. Feature selection algorithms are important as they reduce the dimension of the dataset. The selection of an efficient classification model will improve the detection rate of the proposed system. In the mitigation phase, we introduce two lines of defense to minimize the exhaustion of the victim server's resources. Using the existing dataset, we show experimentally that it is possible to detect the presence of attacks and mitigate them to a minimum level. The proposed integrated method yields an accuracy of 97.8% in detecting the attacks and able to reduce the utilization of processors upto an average of 25.95%.

An ensemble feature selection algorithm based on PageRank centrality and fuzzy logic

One of the crucial processes in machine learning algorithms to improve the performance as well as, in some cases, to reduce the computational cost is feature selection. In other words, when there are many features, some are irrelevant; thus, irrelevant features will badly affect the learning process, so selecting the most fitting features can lead to increasing the accuracy of models in high-dimensional datasets. Recent researches benefit from an integration of various feature selection methods to find relevant features more accurately. Despite the fact that applying several feature selection methods can be efficient, the accuracy of current methods and combining the ranks of different feature selection methods are still challenging problems. In this paper, a novel ensemble of feature selection based on weighted PageRank and fuzzy logic, named EFSPF, is proposed to overcome these drawbacks. First, an M×M matrix, called Euclidean Distance Matrix (EDM), is set up, where M is the number of features. Secondly, a fuzzy Type-I is applied individually to tackle uncertainties in ranks. In other words, to specify a reliability rate for each feature, fuzzy Type-I is utilized. Then, EFSPF forms a reliable weighted graph where each feature is considered as a node. Furthermore, the weight between two nodes expresses a combination of their Euclidean Distance in EDM and the calculated reliability rates through fuzzy Type-I. Finally, the rank of each graph node is calculated via the PageRank algorithm. To demonstrate the performance of EFSPF, we have compared it with five ensemble algorithms and eight primary feature selection algorithms. The proposed method is evaluated based on Accuracy, Precision, Recall, and F1 score metrics. The analysis results show that the proposed method outperforms its rivals. The source code of the proposed method is accessible from https://github.com/mehdijoodaki/EFSPF.

Framework for the Ensemble of Feature Selection Methods

Feature selection (FS) has attracted the attention of many researchers in the last few years due to the increasing sizes of datasets, which contain hundreds or thousands of columns (features). Typically, not all columns represent relevant values. Consequently, the noise or irrelevant columns could confuse the algorithms, leading to a weak performance of machine learning models. Different FS algorithms have been proposed to analyze highly dimensional datasets and determine their subsets of relevant features to overcome this problem. However, very often, FS algorithms are biased by the data. Thus, methods for ensemble feature selection (EFS) algorithms have become an alternative to integrate the advantages of single FS algorithms and compensate for their disadvantages. The objective of this research is to propose a conceptual and implementation framework to understand the main concepts and relationships in the process of aggregating FS algorithms and to demonstrate how to address FS on datasets with high dimensionality. The proposed conceptual framework is validated by deriving an implementation framework, which incorporates a set of Phyton packages with functionalities to support the assembly of feature selection algorithms. The performance of the implementation framework was demonstrated in several experiments discovering relevant features in the Sonar, SPECTF, and WDBC datasets. The experiments contrasted the accuracy of two machine learning classifiers (decision tree and logistic regression), trained with subsets of features generated either by single FS algorithms or the set of features selected by the ensemble feature selection framework. We observed that for the three datasets used (Sonar, SPECTF, and WD), the highest precision percentages (86.95%, 74.73%, and 93.85%, respectively) were obtained when the classifiers were trained with the subset of features generated by our framework. Additionally, the stability of the feature sets generated using our ensemble method was evaluated. The results showed that the method achieved perfect stability for the three datasets used in the evaluation.

A hybrid ensemble-filter wrapper feature selection approach for medical data classification

Background and objectiveMedical data plays a decisive role in disease diagnosis. The classification accuracy of high-dimensional datasets is often diminished by several redundant and irrelevant features. In this context, feature selection becomes an indispensable process. Feature selection primarily intends to identify a feature subspace which retains the classification accuracy while reducing the high computational cost of learning model as well as eliminating noise. The suitability of an appropriate feature selection approach heavily depends upon the capability of that approach to match the problem framework and to discover the intrinsic patterns within the data. The prime objective of this paper is to develop an ensemble-filter based hybrid feature selection model for disease detection. MethodsIn this paper, a four-step hybrid ensemble feature selection algorithm has been introduced. Firstly, the dataset is partitioned using the cross-validation procedure. Secondly, in the filter step, various filter methods based on weighted scores were ensembled to generate a ranking of features, and thirdly sequential forward selection algorithm is utilized as a wrapper technique to obtain an optimal subset of features. Finally, the resulting optimal subset is processed for subsequent classification tasks. ResultsExperiments have been performed on twenty benchmark medical datasets with different dimensionalities. The performance of the proposed hybrid approach is compared with fourteen state-of-the-art feature selection algorithms on four benchmark classifiers namely, Naïve Bayes, Support Vector Machine with Radial Basis Function, Random Forest and k-Nearest Neighbor. The empirical results clearly demonstrate the superiority of the proposed hybrid methodology over the competing methods with respect to accuracy, sensitivity, specificity, f1-score, area under curve evaluation measures and number of selected features. The statistical analysis of the obtained results shows the outperformance and the competitiveness of the proposed hybrid method with respect to various state-of-the-art algorithms. ConclusionsThis study concludes that the proposed hybrid approach proves to be more effective and reliable feature selection technique in selecting highly discriminative features. The framework can be utilized as a promising tool by both clinicians and researchers in enhancing the classification performance of medical datasets.

A multi-surrogate-assisted dual-layer ensemble feature selection algorithm

Ensemble feature selection has attracted much attention of scholars because of its good robustness. However, existing methods on dealing with high-dimensional or large-scale data still have some shortcomings, such as high computational cost or high feature redundancy. In view of it, this paper proposes a new ensemble feature selection algorithm for large-scale data, called Multi-surrogate-assisted Dual-layer Ensemble Feature Selection (MDEFS). In MDEFS, a filter ensemble feature selection method with fast search ability is firstly developed to remove irrelevant or weakly-relevant features in the first layer. In the second layer, a particle swarm-based ensemble method with global search ability is proposed to select optimal feature subset from those remaining features. Furthermore, a multi-surrogate-assisted search mechanism of swarm is developed to reduce the cost of MDEFS on processing large-scale data, where the whole original dataset is replaced by multiple types of representative samples. Finally, the proposed algorithm is applied to 13 datasets and compared with 7 feature selection algorithms. Experimental results show that the multi-surrogate-assisted search mechanism can obviously reduce the running time of MDEFS, and the proposed ensemble approach can make MDEFS obtain better feature subsets, whose average accuracy is 0.72% higher than the best comparison algorithm on each data set. All results indicate that MDEFS is a robust and competitive feature selection algorithm.