Well-known Feature Selection Methods Research Articles

A novel Swarm Optimized Clustering based genetic algorithm for medical decision support system

The clinical and diagnostic levels of medical decision-making may benefit from use of machine learning techniques. Algorithms for feature selection provide an basis about machine learning. The most discriminating health-related traits from the initial feature set may be quickly and effectively identified in a medical context by using feature selection. Choosing the most relevant attributes of data classes and improving classification performance are the two main goals of feature selection algorithms. The process of feature selection not only identifies the most useful characteristics but also aids in lowering the dataset's overall dimensions. Hence in this article, we propose a novel algorithm based on machine learning. Initially, the dataset is collected and preprocessed using normalisation method. The features are extracted using Linear Discriminant Analysis (LDA) and the relevant features are selected using the proposed Swarm Optimized Clustering based Genetic Algorithm (SOC-GA). On the healthcare datasets, we implemented the suggested method into action. Using Random Forest (RF) and Support Vector Machine (SVM) classifiers, performances of chosen feature subsets are assessed based on accuracy. The empirical findings from our studies in this research are competitive in terms of accuracy and outperformed the other well-known feature selection methods. This research offers remedies that might improve the accurate, efficient and trustworthy decision-making process in healthcare systems for targeted medical treatments.

A high-quality feature selection method based on frequent and correlated items for text classification

The feature selection problem is a significant challenge in pattern recognition, especially for classification tasks. The quality of the selected features plays a critical role in building effective models, and poor-quality data can make this process more difficult. This work explores the use of association analysis in data mining to select meaningful features, addressing the issue of duplicated information in the selected features. A novel feature selection technique for text classification is proposed, based on frequent and correlated items. This method considers both relevance and feature interactions, using association as a metric to evaluate the relationship between the target and features. The technique was tested using the SMS spam collecting dataset from the UCI machine learning repository and compared with well-known feature selection methods. The results showed that the proposed technique effectively reduced redundant information while achieving high accuracy (95.155%) using only 6% of the features.

Optimizing prognostic factors of five-year survival in gastric cancer patients using feature selection techniques with machine learning algorithms: a comparative study

BackgroundGastric cancer is the most common malignant tumor worldwide and a leading cause of cancer deaths. This neoplasm has a poor prognosis and heterogeneous outcomes. Survivability prediction may help select the best treatment plan based on an individual’s prognosis. Numerous clinical and pathological features are generally used in predicting gastric cancer survival, and their influence on the survival of this cancer has not been fully elucidated. Moreover, the five-year survivability prognosis performances of feature selection methods with machine learning (ML) classifiers for gastric cancer have not been fully benchmarked. Therefore, we adopted several well-known feature selection methods and ML classifiers together to determine the best-paired feature selection-classifier for this purpose.MethodsThis was a retrospective study on a dataset of 974 patients diagnosed with gastric cancer in the Ayatollah Talleghani Hospital, Abadan, Iran. First, four feature selection algorithms, including Relief, Boruta, least absolute shrinkage and selection operator (LASSO), and minimum redundancy maximum relevance (mRMR) were used to select a set of relevant features that are very informative for five-year survival prediction in gastric cancer patients. Then, each feature set was fed to three classifiers: XG Boost (XGB), hist gradient boosting (HGB), and support vector machine (SVM) to develop predictive models. Finally, paired feature selection-classifier methods were evaluated to select the best-paired method using the area under the curve (AUC), accuracy, sensitivity, specificity, and f1-score metrics.ResultsThe LASSO feature selection algorithm combined with the XG Boost classifier achieved an accuracy of 89.10%, a specificity of 87.15%, a sensitivity of 89.42%, an AUC of 89.37%, and an f1-score of 90.8%. Tumor stage, history of other cancers, lymphatic invasion, tumor site, type of treatment, body weight, histological type, and addiction were identified as the most significant factors affecting gastric cancer survival.ConclusionsThis study proved the worth of the paired feature selection-classifier to identify the best path that could augment the five-year survival prediction in gastric cancer patients. Our results were better than those of previous studies, both in terms of the time required to form the models and the performance measurement criteria of the algorithms. These findings may be very promising and can, therefore, inform clinical decision-making and shed light on future studies.

Boosting capuchin search with stochastic learning strategy for feature selection

The technological revolution has made available a large amount of data with many irrelevant and noisy features that alter the analysis process and increase time processing. Therefore, feature selection (FS) approaches are used to select the smallest subset of relevant features. Feature selection is viewed as an optimization process for which meta-heuristics have been successfully applied. Thus, in this paper, a new feature selection approach is proposed based on an enhanced version of the Capuchin search algorithm (CapSA). In the developed FS approach, named ECapSA, three modifications have been introduced to avoid a lack of diversity, and premature convergence of the basic CapSA: (1) The inertia weight is adjusted using the logistic map, (2) sine cosine acceleration coefficients are added to improve convergence, and (3) a stochastic learning strategy is used to add more diversity to the movement of Capuchin and a levy random walk. To demonstrate the performance of ECapSA, different datasets are used, and it is compared with other well-known FS methods. The results provide evidence of the superiority of ECapSA among the tested datasets and competitive methods in terms of performance metrics.

Bankruptcy prediction using fuzzy convolutional neural networks

We propose a combined method for bankruptcy prediction based on fuzzy set qualitative comparative analysis (fsQCA) and convolutional neural networks (CNN). Currently, CNNs are being applied to various fields, and in some areas are providing higher performance than traditional models. In our proposed method, a CNN uses calibrated variables from fuzzy sets to improve performance accuracy. In addition, there are no published studies on the effect of feature selection at the input level of convolutional neural networks. Therefore, this study compares four well-known feature selection methods used in financial distress prediction, (t-test, stepdisc discriminant analysis, stepwise logistic regression and partial least square discriminant analysis) to investigate their effect on classification performance. The results show that fuzzy convolutional neural networks (FCNN) lead to better performance than when using traditional methods.

Co-Operative Binary Bat Optimizer with Rough Set Reducts for Text Feature Selection

The process of eliminating irrelevant, redundant and noisy features while trying to maintain less information loss is known as a feature selection problem. Given the vast amount of the textual data generated and shared on the internet such as news reports, articles, tweets and product reviews, the need for an effective text-feature selection method becomes increasingly important. Recently, stochastic optimization algorithms have been adopted to tackle this problem. However, the efficiency of these methods is decreased when tackling high-dimensional problems. This decrease could be attributed to premature convergence where the population diversity is not well maintained. As an innovative attempt, a cooperative Binary Bat Algorithm (BBACO) is proposed in this work to select the optimal text feature subset for classification purposes. The proposed BBACO uses a new mechanism to control the population’s diversity during the optimization process and to improve the performance of BBA-based text-feature selection method. This is achieved by dividing the dimension of the problem into several parts and optimizing each of them in a separate sub-population. To evaluate the generality and capability of the proposed method, three classifiers and two standard benchmark datasets in English, two in Malay and one in Arabic were used. The results show that the proposed method steadily improves the classification performance in comparison with other well-known feature selection methods. The improvement is obtained for all of the English, Malay and Arabic datasets which indicates the generality of the proposed method in terms of the dataset language.

A Quantum-Based Chameleon Swarm for Feature Selection

The Internet of Things is widely used, which results in the collection of enormous amounts of data with numerous redundant, irrelevant, and noisy features. In addition, many of these features need to be managed. Consequently, developing an effective feature selection (FS) strategy becomes a difficult goal. Many FS techniques, based on bioinspired metaheuristic methods, have been developed to tackle this problem. However, these methods still suffer from limitations; so, in this paper, we developed an alternative FS technique, based on integrating operators of the chameleon swarm algorithm (Cham) with the quantum-based optimization (QBO) technique. With the use of eighteen datasets from various real-world applications, we proposed that QCham is investigated and compared to well-known FS methods. The comparisons demonstrate the benefits of including a QBO operator in the Cham because the proposed QCham can efficiently and accurately detect the most crucial features. Whereas the QCham achieves nearly 92.6%, with CPU time(s) nearly 1.7 overall the tested datasets. This indicates the advantages of QCham among comparative algorithms and high efficiency of integrating the QBO with the operators of Cham algorithm that used to enhance the process of balancing between exploration and exploitation.

A Spectral Feature Selection Approach With Kernelized Fuzzy Rough Sets

Feature evaluation is an important issue in constructing a feature selection algorithm in kernelized fuzzy rough sets, which has been proven to be an effective approach to deal with nonlinear classification tasks and uncertainty in learning problems. However, the feature evaluation function developed with kernelized fuzzy rough sets cannot better reflect the affinity relationship of samples and is time-consuming. To overcome these drawbacks, in this article, the problem of feature selection with kernelized fuzzy rough sets is studied based on the spectral graph theory. First, the within-class and between-class sample similarity matrices by using kernelized fuzzy approximation operators are constructed. Two operators, which can capture the affinity relationship of samples, are then introduced based on the sample similarity matrices. The proposed operator can be regarded as the sum of the weighted kernelized fuzzy approximation operators. Second, based on the ratio criterion, a feature evaluation function and its corresponding feature selection algorithm FRKF are presented, which can effectively evaluate the importance of features. Third, to illustrate the performance of the proposed algorithm, extensive experiments have been carried out to compare FRKF and other well-known feature selection methods, including the feature ranking methods and feature subset selection methods on various classification tasks. The experimental results on real-world datasets demonstrate that FRKF achieves the high performances in terms of the robustness, efficiency, and effectiveness.

Simulated annealing-based dynamic step shuffled frog leaping algorithm: Optimal performance design and feature selection

The shuffled frog leaping algorithm is a new optimization algorithm proposed to solve the combinatorial optimization problem, which effectively combines the memetic algorithm based on a memetic algorithm and the particle swarm algorithm based on population behavior. The algorithm is widely used because it is easy to implement and requires few parameters to be adjusted. However, there are still some characteristics of this method that need to be improved because it is easy to fall into local optimization or poor search ability. To alleviate this limitation, a new version of the improved SFLA is proposed in this paper, which incorporates a dynamic step size adjustment strategy based on historical information, a specular reflection learning mechanism, and a simulated annealing mechanism based on chaotic mapping and levy flight. Firstly, the dynamic step size adjustment strategy based on historical information effectively helps to balance local exploration and global exploitation and alleviates the problem of falling into local optimum. Second, the specular reflection learning mechanism increases the possibility of searching for valid solutions in feasible domains and enhances the search ability of individuals in the population. Finally, an improved simulated annealing strategy is executed for each memetic, which improves the efficiency of local exploitation. In order to test the performance of the proposed algorithm, 31 test functions were selected from IEEE CEC2014 and 23 essential benchmark functions, and comparative experiments were carried out from the two dimensions of 30 and 100. Furthermore, a series of competing algorithms are selected, which involve nine classical standard algorithms, including PSO, BA, SSA, FA, SCA, WOA, GWO, MFO, and SFLA, as well as, six well-known improved algorithms, including LSFLA, DDSFLA, GOTLBO, ALCPSO, BLPSO, CLPSO. Furthermore, Wilcoxon signed-rank test and Friedman test are used as testing tools to illustrate the scalability of the proposed algorithm. From the analysis of the results, it can be seen that this proposed method has been effectively improved concerning stability and the quality of the optimal solution obtained from the search, both in low and high dimensions, and the ability to jump out of the local optimum has also been enhanced. In addition, to prove that this method has a reliable performance in discrete problems and continuous problems, DSSRLFLA is mapped into a discrete space, and 24 UCI data sets are also selected to evaluate the performance of the new feature selection method. The experimental results illustrate that this improved method can obtain fewer features and higher classification accuracy than some well-known feature selection methods.

ST-AL: a hybridized search based metaheuristic computational algorithm towards optimization of high dimensional industrial datasets.

The rapid growth of data generated by several applications like engineering, biotechnology, energy, and others has become a crucial challenge in the high dimensional data mining. The large amounts of data, especially those with high dimensions, may contain many irrelevant, redundant, or noisy features, which may negatively affect the accuracy and efficiency of the industrial data mining process. Recently, several meta-heuristic optimization algorithms have been utilized to evolve feature selection techniques for dealing with the vast dimensionality problem. Despite optimization algorithms’ ability to find the near-optimal feature subset of the search space, they still face some global optimization challenges. This paper proposes an improved version of the sooty tern optimization (ST) algorithm, namely the ST-AL method, to improve the search performance for high-dimensional industrial optimization problems. ST-AL method is developed by boosting the performance of STOA by applying four strategies. The first strategy is the use of a control randomization parameters that ensure the balance between the exploration–exploitation stages during the search process; moreover, it avoids falling into local optimums. The second strategy entails the creation of a new exploration phase based on the Ant lion (AL) algorithm. The third strategy is improving the STOA exploitation phase by modifying the main equation of position updating. Finally, the greedy selection is used to ignore the poor generated population and keeps it from diverging from the existing promising regions. To evaluate the performance of the proposed ST-AL algorithm, it has been employed as a global optimization method to discover the optimal value of ten CEC2020 benchmark functions. Also, it has been applied as a feature selection approach on 16 benchmark datasets in the UCI repository and compared with seven well-known optimization feature selection methods. The experimental results reveal the superiority of the proposed algorithm in avoiding local minima and increasing the convergence rate. The experimental result are compared with state-of-the-art algorithms, i.e., ALO, STOA, PSO, GWO, HHO, MFO, and MPA and found that the mean accuracy achieved is in range 0.94–1.00.

Optimizing feature selection methods by removing irrelevant features using sparse least squares

Feature (variable) selection is recognized as an integral part of model construction in machine learning. One can use feature selection to remove redundant and irrelevant features. This in turn can help overcome the curse of dimensionality, reduce overfitting, and come up with interpretable models. In this paper, we propose Sparse Least Squares method (SLS) based on singular value decomposition and least squares to remove irrelevant features. We show that augmenting well-known feature selection methods with SLS significantly reduces the running time while improving or maintaining the prediction accuracy of the model.

Comparison of five supervised feature selection algorithms leading to top features and gene signatures from multi-omics data in cancer

BackgroundAs many complex omics data have been generated during the last two decades, dimensionality reduction problem has been a challenging issue in better mining such data. The omics data typically consists of many features. Accordingly, many feature selection algorithms have been developed. The performance of those feature selection methods often varies by specific data, making the discovery and interpretation of results challenging.Methods and resultsIn this study, we performed a comprehensive comparative study of five widely used supervised feature selection methods (mRMR, INMIFS, DFS, SVM-RFE-CBR and VWMRmR) for multi-omics datasets. Specifically, we used five representative datasets: gene expression (Exp), exon expression (ExpExon), DNA methylation (hMethyl27), copy number variation (Gistic2), and pathway activity dataset (Paradigm IPLs) from a multi-omics study of acute myeloid leukemia (LAML) from The Cancer Genome Atlas (TCGA). The different feature subsets selected by the aforesaid five different feature selection algorithms are assessed using three evaluation criteria: (1) classification accuracy (Acc), (2) representation entropy (RE) and (3) redundancy rate (RR). Four different classifiers, viz., C4.5, NaiveBayes, KNN, and AdaBoost, were used to measure the classification accuary (Acc) for each selected feature subset. The VWMRmR algorithm obtains the best Acc for three datasets (ExpExon, hMethyl27 and Paradigm IPLs). The VWMRmR algorithm offers the best RR (obtained using normalized mutual information) for three datasets (Exp, Gistic2 and Paradigm IPLs), while it gives the best RR (obtained using Pearson correlation coefficient) for two datasets (Gistic2 and Paradigm IPLs). It also obtains the best RE for three datasets (Exp, Gistic2 and Paradigm IPLs). Overall, the VWMRmR algorithm yields best performance for all three evaluation criteria for majority of the datasets. In addition, we identified signature genes using supervised learning collected from the overlapped top feature set among five feature selection methods. We obtained a 7-gene signature (ZMIZ1, ENG, FGFR1, PAWR, KRT17, MPO and LAT2) for EXP, a 9-gene signature for ExpExon, a 7-gene signature for hMethyl27, one single-gene signature (PIK3CG) for Gistic2 and a 3-gene signature for Paradigm IPLs.ConclusionWe performed a comprehensive comparison of the performance evaluation of five well-known feature selection methods for mining features from various high-dimensional datasets. We identified signature genes using supervised learning for the specific omic data for the disease. The study will help incorporate higher order dependencies among features.

An optimal approach for text feature selection

Traditionally, feature selection is conducted by first deriving a candidate list of features, then ranking and selecting the top features based on predefined threshold. These methods are highly dependent on the choice of the threshold, and therefore lead to sub-optimal text categorization results. In this paper, we address the selection problem by suggesting a one-step method designed to optimally select the subset of features. The selection is formulated mathematically as an optimization problem with the objective of maximizing classification accuracy while simultaneously deriving and choosing the most discriminative features. Our method, MFX, is applicable to many of the conventional methods, with two distinguishing aspects. First, it is based on considering all documents from the same category as one extended document, instead of analyzing individual documents. Second, it considers choosing the most discriminative terms that are frequent and common across all documents of the same category, and minimally present in other categories. Moreover, MFX is language-independent. It was tested on the well-known benchmark Reuters RCV1 dataset. To showcase its language independence, MFX was also tested on Arabic datasets extracted from Arabic news sources. The results indicated that MFX always performed similar to or better than other well-known feature selection methods. MFX with a Support Vector Machine (SVM) classifier was also shown to outperform recent text classification algorithms based on neural networks and word embeddings.

Feature Selection: A Review and Comparative Study

Feature selection (FS) is an important research topic in the area of data mining and machine learning. FS aims at dealing with the high dimensionality problem. It is the process of selecting the relevant features and removing the irrelevant, redundant and noisy ones, intending to obtain the best performing subset of original features without any transformation. This paper provides a comprehensive review of FS literature intending to supplement insights and recommendations to help readers. Moreover, an empirical study of six well-known feature selection methods is presented so as to critically analyzing their applicability.

EARLY PREDICTION OF CERVICAL CANCER USING MACHINE LEARNING TECHNIQUES

According to recent studies and statistics, Cervical Cancer (CC) is one of the most common causes of death worldwide, and mainly in the developing countries. CC has a mortality rate around 60%, in less developing countries and the percentages could go even higher, due to poor screening processes, lack of sensitization, and several other reasons. Therefore, this paper aims to utilize the high capabilities of machine learning techniques in the early prediction of CC. In specific, three well-known feature selection and ranking methods have been used to identify the most significant features that help in the diagnosis process. Also, eighteen different classifiers that belong to six learning strategies have been trained and extensively evaluated against a primary data which consists of five hundred images. Moreover, an investigation regarding the problem of imbalance class distribution which is common in medical dataset is conducted. The results revealed that LWNB and RandomForest classifiers showed the best performance in general, and considering four different evaluation metrics. Also, LWNB and Logistic classifiers were the best choices to handle the problem of imbalance class distribution which is common in medical diagnosis task. The final conclusion could be made is that using an ensemble model which consists of several classifiers such as LWNB, RandomForest, and Logistic is the best solution to handle this type of problems.

Orthogonally constrained matrix factorization for robust unsupervised feature selection with local preserving

Feature selection is a significant preprocessing technique that involves discarding redundant and irrelevant features, so as to reduce the data dimensionality and build a better understanding of data. Pruning superfluous features tend to build better generalization models while improving the computation efficiency extremely. In practices, obtaining the labels of data is time-consuming and labor-intensive, which brings great challenges for feature selection. In this paper, we present a novel robust unsupervised feature selection method for unlabeled dimensionality reduction, which obtains feature importance information by predicting cluster labels of data with matrix factorization. The orthogonal constraints on two decomposing matrices facilitate achieving more accurate class labels of clusters so as to select features with highly discrimination power. Then, the local preserving term is integrated into the projected data for selecting features with local retention ability. Independent of that, an alternative iterative algorithm is incorporated into the optimization of ℓ2,1-norm objective function for efficient and robust feature selection. Extensive comparative experiments are carried out on six benchmark datasets to evaluate the performance of the proposed method. The results show that our method outperforms several well-known unsupervised feature selection methods in terms of both clustering accuracy and normalized mutual information.

Enhancing Big Data Feature Selection Using a Hybrid Correlation-Based Feature Selection

This study proposes an alternate data extraction method that combines three well-known feature selection methods for handling large and problematic datasets: the correlation-based feature selection (CFS), best first search (BFS), and dominance-based rough set approach (DRSA) methods. This study aims to enhance the classifier’s performance in decision analysis by eliminating uncorrelated and inconsistent data values. The proposed method, named CFS-DRSA, comprises several phases executed in sequence, with the main phases incorporating two crucial feature extraction tasks. Data reduction is first, which implements a CFS method with a BFS algorithm. Secondly, a data selection process applies a DRSA to generate the optimized dataset. Therefore, this study aims to solve the computational time complexity and increase the classification accuracy. Several datasets with various characteristics and volumes were used in the experimental process to evaluate the proposed method’s credibility. The method’s performance was validated using standard evaluation measures and benchmarked with other established methods such as deep learning (DL). Overall, the proposed work proved that it could assist the classifier in returning a significant result, with an accuracy rate of 82.1% for the neural network (NN) classifier, compared to the support vector machine (SVM), which returned 66.5% and 49.96% for DL. The one-way analysis of variance (ANOVA) statistical result indicates that the proposed method is an alternative extraction tool for those with difficulties acquiring expensive big data analysis tools and those who are new to the data analysis field.

BRNS + SSFSM-DTI: A hybrid method for drug-target interaction prediction based on balanced reliable negative samples and semi-supervised feature selection

De novo drug discovery is a time-consuming and costly process. Drug repositioning, which means finding new applications for existing drugs, is one of the most effective approaches to reduce time and cost of detecting a new drug. Predicting drug-target interactions (DTIs) can facilitate the drug repositioning and consequently accelerate the process of drug discovery. The diversity of drug descriptors and protein features as well as the lack of access to experimentally-confirmed non-interacting drug-target pairs as negative samples are the major challenges in predicting new DTIs. In this study, we present a modified algorithm for extracting balanced reliable negative samples named BRNS. Moreover, we propose a semi-supervised feature selection method called SSFSM-DTI and compare the performance of our hybrid predictive model with the models constructed by random selection of negative samples and the other well-known feature selection methods on benchmark DTI datasets. The results show that the combination of BRNS and SSFSM-DTI is superior to the random selection of negative samples and the state-of-the-art feature selection methods in most cases. Hence, it can be used as a guideline in the drug repositioning process. The source codes and all datasets used in this study are available at https://github.com/LBDSoft/BRNS.

Identification of important features and data mining classification techniques in predicting employee absenteeism at work

Employees absenteeism at the work costs organizations billions a year. Prediction of employees’ absenteeism and the reasons behind their absence help organizations in reducing expenses and increasing productivity. Data mining turns the vast volume of human resources data into information that can help in decision-making and prediction. Although the selection of features is a critical step in data mining to enhance the efficiency of the final prediction, it is not yet known which method of feature selection is better. Therefore, this paper aims to compare the performance of three well-known feature selection methods in absenteeism prediction, which are relief-based feature selection, correlation-based feature selection and information-gain feature selection. In addition, this paper aims to find the best combination of feature selection method and data mining technique in enhancing the absenteeism prediction accuracy. Seven classification techniques were used as the prediction model. Additionally, cross-validation approach was utilized to assess the applied prediction models to have more realistic and reliable results. The used dataset was built at a courier company in Brazil with records of absenteeism at work. Regarding experimental results, correlationbased feature selection surpasses the other methods through the performance measurements. Furthermore, bagging classifier was the best-performing data mining technique when features were selected using correlation-based feature selection with an accuracy rate of (92%).

A novel filter feature selection method for text classification: Extensive Feature Selector

As the huge dimensionality of textual data restrains the classification accuracy, it is essential to apply feature selection (FS) methods as dimension reduction step in text classification (TC) domain. Most of the FS methods for TC contain several number of probabilities. In this study, we proposed a new FS method named as Extensive Feature Selector (EFS), which benefits from corpus-based and class-based probabilities in its calculations. The performance of EFS is compared with nine well-known FS methods, namely, Chi-Squared (CHI2), Class Discriminating Measure (CDM), Discriminative Power Measure (DPM), Odds Ratio (OR), Distinguishing Feature Selector (DFS), Comprehensively Measure Feature Selection (CMFS), Discriminative Feature Selection (DFSS), Normalised Difference Measure (NDM) and Max–Min Ratio (MMR) using Multinomial Naive Bayes (MNB), Support-Vector Machines (SVMs) and k-Nearest Neighbour (KNN) classifiers on four benchmark data sets. These data sets are Reuters-21578, 20-Newsgroup, Mini 20-Newsgroup and Polarity. The experiments were carried out for six different feature sizes which are 10, 30, 50, 100, 300 and 500. Experimental results show that the performance of EFS method is more successful than the other nine methods in most cases according to micro- F1 and macro- F1 scores.