Solve Feature Selection Problems Research Articles

A Recurrence Population-based Great Deluge Algorithm with Independent Quality Estimation for Feature Selection from Academician Data

ABSTRACT Great deluge (GD) algorithm same as other metaheuristics can solve feature selection problem. The GD imitates that in a great deluge someone climbing a hill and attempt to progress in any direction that does not get his/her feet wet in the expectation of discovering a way up when the water Level rises. The drawbacks of GD are: 1) a local search, which may lead the algorithm toward a local optima and 2) a challenging estimation of quality of the final solution in solving most of the problems. In this paper, for the first issue, a population-based great deluge (popGD) algorithm with additional recurrence operation is proposed. This operation is an imitation of no progress of hill climber after a long time; the climber tries to move small steps even downward in hope of finding better way to climb. For the second problem, a technique with an automate alteration of the Level is proposed. The statistical analysis of the results from 25 test functions and 18 benchmark feature selection problems supports the ability of the method. Finally a real-world academician data are employed to perform feature selection and execute classification result with selected features.

Differential search algorithm for biojective feature selection

Feature selection (FS) is a complex optimization problem with important real-world applications. Generally, the main target of FS is to reduce the dimensionality of features and enhance the effectiveness of solving problems. Due to the population characteristics, various evolutionary algorithms (EAs) have been proposed to solve feature selection problems over the past decades. However, the majority of them only consider single-objective optimization, while many real-world problems have multiple objectives, therefore, it is need to design more suitable and effective EAs to deal with multi-objective FS. In this paper, a biobjective FS algorithm based on differential search algorithm (DSA) is designed to solve FS problems, which minimizes the number of selected features and maximizes the classification accuracy. The results of simulation experiments and statistical analysis on 15 classification datasets compared with other four state-of-the-art EAs show that the proposed DSA algorithm can not only obtain better optimization performance, but also achieve competitive convergence accuracy.

Fingerprint Intramodal Biometric System Based on ABC Feature Fusion

Unimodal biometrics system (UBS) drawbacks include noisy data, intra-class variance, inter-class similarities, non-universality, which all affect the system's classification performance. Intramodal fingerprint fusion can overcome the limitations imposed by UBS when features are fused at the feature level as it is a good approach to boost the performance of the biometric system. However, feature level fusion leads to high dimensionality of feature space which can be overcame by Feature Selection (FS). FS improves the performance of classification by selecting only relevant and useful information from extracted feature sets being an optimization problem. Artificial Bee Colony (ABC) is an optimizing algorithm that has been frequently used in solving FS problems because of its simple concept, use of few control parameters, easy implementation and good exploration characteristics. ABC was proposed for optimized feature selection prior to the classification of Fingerprint Intramodal Biometric System (FIBS). Performance evaluation of ABC-based FIBS showed the system had a Sensitivity of 97.69% and RA of 96.76%. The developed ABC optimized feature selection reduced the high dimensionality of features space prior to classification tasks thereby increasing sensitivity and recognition accuracy of FIBS.

An hybrid particle swarm optimization with crow search algorithm for feature selection

The recent advancements in science, engineering, and technology have facilitated huge generation of datasets. These huge datasets contain noisy, redundant, and irrelevant features which negatively affects the performance of classification techniques in machine learning and data mining process. Feature selection is a pre-processing stage for reducing the dimensionality of datasets by selecting the most important attributes while increasing the accuracy of classification at the same time. In this paper, we present a novel hybrid binary version of enhanced chaotic crow search and particle swarm optimization algorithm (ECCSPSOA) to solve feature selection problems. In the proposed ECCSPSOA, in order to navigate the feature space, we hybridized the enhanced version of the CSA algorithm which has a better search strategy and particle swarm optimization (PSO) which is capable of converging into the best global solution in the search field. We further embed opposition-based learning technique in the local search of the hybrid algorithm. The ECCSPSOA was compared using 15 datasets from the UCI repository with four well-known optimization algorithms, such as particle swarm optimization (PSO), binary particle swarm optimization (BPSO), crow search algorithm (CSA), and chaotic crow search algorithm (CCSA). In the experiments with k-Nearest Neighbour (KNN) as a classifier, six different performance metrics were used. To tackle the over-fitting problem, each dataset is divided into training and testing data using K-fold cross-validation. The computational findings demonstrate that the proposed algorithm obtains an average accuracy rate of 89.67 % over 15 datasets, indicating that our technique exceeds state-of-the-art findings in 12 of the 15 datasets studied. Furthermore, the suggested approach outperforms state-of-the-art methods in terms of fitness value and standard deviation, obtaining the lowest value in 13 and 8 of the datasets studied respectively.

Improved Equilibrium Optimization Algorithm Using Elite Opposition-Based Learning and New Local Search Strategy for Feature Selection in Medical Datasets

The rapid growth in biomedical datasets has generated high dimensionality features that negatively impact machine learning classifiers. In machine learning, feature selection (FS) is an essential process for selecting the most significant features and reducing redundant and irrelevant features. In this study, an equilibrium optimization algorithm (EOA) is used to minimize the selected features from high-dimensional medical datasets. EOA is a novel metaheuristic physics-based algorithm and newly proposed to deal with unimodal, multi-modal, and engineering problems. EOA is considered as one of the most powerful, fast, and best performing population-based optimization algorithms. However, EOA suffers from local optima and population diversity when dealing with high dimensionality features, such as in biomedical datasets. In order to overcome these limitations and adapt EOA to solve feature selection problems, a novel metaheuristic optimizer, the so-called improved equilibrium optimization algorithm (IEOA), is proposed. Two main improvements are included in the IEOA: The first improvement is applying elite opposite-based learning (EOBL) to improve population diversity. The second improvement is integrating three novel local search strategies to prevent it from becoming stuck in local optima. The local search strategies applied to enhance local search capabilities depend on three approaches: mutation search, mutation–neighborhood search, and a backup strategy. The IEOA has enhanced the population diversity, classification accuracy, and selected features, and increased the convergence speed rate. To evaluate the performance of IEOA, we conducted experiments on 21 biomedical benchmark datasets gathered from the UCI repository. Four standard metrics were used to test and evaluate IEOA’s performance: the number of selected features, classification accuracy, fitness value, and p-value statistical test. Moreover, the proposed IEOA was compared with the original EOA and other well-known optimization algorithms. Based on the experimental results, IEOA confirmed its better performance in comparison to the original EOA and the other optimization algorithms, for the majority of the used datasets.

S-shaped and V-shaped gaining-sharing knowledge-based algorithm for feature selection

In machine learning, searching for the optimal feature subset from the original datasets is a very challenging and prominent task. The metaheuristic algorithms are used in finding out the relevant, important features, that enhance the classification accuracy and save the resource time. Most of the algorithms have shown excellent performance in solving feature selection problems. A recently developed metaheuristic algorithm, gaining-sharing knowledge-based optimization algorithm (GSK), is considered for finding out the optimal feature subset. GSK algorithm was proposed over continuous search space; therefore, a total of eight S-shaped and V-shaped transfer functions are employed to solve the problems into binary search space. Additionally, a population reduction scheme is also employed with the transfer functions to enhance the performance of proposed approaches. It explores the search space efficiently and deletes the worst solutions from the search space, due to the updation of population size in every iteration. The proposed approaches are tested over twenty-one benchmark datasets from UCI repository. The obtained results are compared with state-of-the-art metaheuristic algorithms including binary differential evolution algorithm, binary particle swarm optimization, binary bat algorithm, binary grey wolf optimizer, binary ant lion optimizer, binary dragonfly algorithm, binary salp swarm algorithm. Among eight transfer functions, V4 transfer function with population reduction on binary GSK algorithm outperforms other optimizers in terms of accuracy, fitness values and the minimal number of features. To investigate the results statistically, two non-parametric statistical tests are conducted that concludes the superiority of the proposed approach.

A self-adaptive Harris Hawks optimization algorithm with opposition-based learning and chaotic local search strategy for global optimization and feature selection

Harris Hawks Optimization is a recently proposed algorithm inspired by the cooperative manner and chasing behavior of harris. However, from the experimental results, it can be noticed that HHO may fall in local optima or have a slow convergence curve in some complex optimization tasks. In this paper, an improved version of HHO called IHHO is proposed which enhances the performance of HHO by combining HHO with opposition-based learning (OBL), Chaotic Local Search (CLS), and a self-adaptive technique. In order to show the performance of the proposed algorithm, several experiments are conducted using the Standard IEEE CEC 2017 benchmark. IHHO is compared with the classical HHO and other 10 state-of-art algorithms. Moreover, IHHO is used to solve 5 constrained engineering problems. IHHO has also been applied to solve feature selection problem using 7 UCI dataset. The numerical results and analysis show the superiority of IHHO in solving real-world problems.

Classification of Parkinson disease using binary Rao optimization algorithms

AbstractRao algorithms are recently proposed optimization algorithms used to solve optimization problems. These algorithms are based on the best and the worst solutions, which are computed during the optimization process. However, these algorithms apply to continuous problems only. In this article, the binary versions of Rao algorithms are proposed, which can be used for solving feature selection problems. These are applied to four publicly available Parkinson's disease datasets. Besides providing an optimal set of features, the k parameter of the k‐nearest neighbour classifier is also optimized by the proposed approach. The performance of these algorithms has been measured taking an average of 30 independent runs using a 10‐fold cross‐validation procedure. Also, a comparison of the performance has been made with the other state of the art methods. Significance analysis of these algorithms has been made with the Friedman rank test.

Multi-objective particle swarm optimization with adaptive strategies for feature selection

Feature selection is a multi-objective optimization problem since it has two conflicting objectives: maximizing the classification accuracy and minimizing the number of the selected features. Due to the lack of selection pressures, most feature selection algorithms based on multi-objective optimization obtain many optimal solutions around the center of Pareto fronts. The penalty boundary interaction (PBI) decomposition approach provides fixed selection pressures for the population, but fixed selection pressures are hard to solve feature selection problems with complicated Pareto fronts. This paper proposes a novel feature selection algorithm based on multi-objective particle swarm optimization with adaptive strategies (MOPSO-ASFS) to improve the selection pressures of the population. An adaptive penalty mechanism based on PBI parameter adjusts penalty values adaptively to enhance the selection pressures of the archive. An adaptive leading particle selection based on feature information combines the opposite mutation and the feature frequencies to improve the selection pressure of each particle. The proposed algorithm is compared with 6 related algorithms on 14 benchmark UCI datasets and 6 gene datasets. The experimental results show that MOPSO-ASFS can find optimal solutions with better convergence and diversity than comparison algorithms especially on the high dimensional datasets.

Pinhole-imaging-based learning butterfly optimization algorithm for global optimization and feature selection

Butterfly optimization algorithm (BOA), as a recently proposed meta-heuristic optimization technique, performs competitively in solving numerical optimization problems as well as real-world applications. However, BOA has low precision, slow convergence and may be prone to local optimum, when solving complex or high-dimensional optimization problems. To overcome these defects, a modified BOA (called PIL-BOA) with adaptive gbest-guided search strategy and pinhole-imaging-based learning is proposed. Firstly, a modified position updated equation by introducing the global best (gbest) solution and the inertia weight is designed to efficiently improve the exploitation capability and the solution precision. Secondly, a novel pinhole-imaging learning strategy based on the principle of optics is presented to effectively search the unknown regions and avoid premature convergence. 23 classical problems and 60 complex optimization tasks from CEC 2014 and CEC 2017 are used to further investigate the effectiveness of PIL-BOA. The comparison results demonstrate that PIL-BOA has better performance than most compared algorithms on benchmark test functions. Finally, PIL-BOA is applied to solve feature selection problems and fault diagnosis in real-world wind turbine. The results show that PIL-BOA is superior to other competitors in term of classification accuracy.

Metaheuristic Algorithms on Feature Selection: A Survey of One Decade of Research (2009-2019)

Feature selection is a critical and prominent task in machine learning. To reduce the dimension of the feature set while maintaining the accuracy of the performance is the main aim of the feature selection problem. Various methods have been developed to classify the datasets. However, metaheuristic algorithms have achieved great attention in solving numerous optimization problem. Therefore, this paper presents an extensive literature review on solving feature selection problem using metaheuristic algorithms which are developed in the ten years (2009-2019). Further, metaheuristic algorithms have been classified into four categories based on their behaviour. Moreover, a categorical list of more than a hundred metaheuristic algorithms is presented. To solve the feature selection problem, only binary variants of metaheuristic algorithms have been reviewed and corresponding to their categories, a detailed description of them explained. The metaheuristic algorithms in solving feature selection problem are given with their binary classification, name of the classifier used, datasets and the evaluation metrics. After reviewing the papers, challenges and issues are also identified in obtaining the best feature subset using different metaheuristic algorithms. Finally, some research gaps are also highlighted for the researchers who want to pursue their research in developing or modifying metaheuristic algorithms for classification. For an application, a case study is presented in which datasets are adopted from the UCI repository and numerous metaheuristic algorithms are employed to obtain the optimal feature subset.

A New Population Initialization of Particle Swarm Optimization Method Based on PCA for Feature Selection

In many fields such as signal processing, machine learning, pattern recognition and data mining, it is common practice to process datasets containing huge numbers of features. In such cases, Feature Selection (FS) is often involved. Meanwhile, owing to their excellent global search ability, evolutionary computation techniques have been widely employed to the FS. So, as a powerful global search method and calculation fast than other EC algorithms, PSO can solve features selection problems well. However, when facing a large number of feature selection, the efficiency of PSO drops significantly. Therefore, plenty of works have been done to improve this situation. Besides, many studies have shown that an appropriate population initialization can effectively help to improve this problem. So, basing on PSO, this paper introduces a new feature selection method with filter-based population. The proposed algorithm uses Principal Component Analysis (PCA) to measure the importance of features first, then based on the sorted feature information, a population initialization method using the threshold selection and the mixed initialization is proposed. The experiments were performed on several datasets and compared to several other related algorithms. Experimental results show that the accuracy of PSO to solve feature selection problems is significantly improved after using proposed method.

RoPM: An Algorithm for Computing Typical Testors Based on Recursive Reductions of the Basic Matrix

Feature selection plays an important role in pattern recognition and smart computing. The full set of typical testors constitutes a useful tool for solving feature selection problems, especially those problems in which the objects are described by both quantitative and qualitative features. However, finding the typical testors involves a high computational cost. That is why even the most efficient methods become unsuitable to solve some problems. In this work, a new algorithm was introduced in order to reduce the long runtimes involved in the search of typical testors. The performance of the proposed algorithm was evaluated by means of several tests, which use both real-world and simulation data. MATLAB and Java language on Eclipse SDK platform were used to build the simulation dataset and to perform the tests, respectively. The runtimes achieved by the proposed algorithm were significantly shorter than those obtained by fast-BR and GCreduct (the two fastest algorithms) mainly when the latter ones exhibited excessively long runtimes.

Binary Political Optimizer for Feature Selection Using Gene Expression Data.

DNA Microarray technology is an emergent field, which offers the possibility of obtaining simultaneous estimates of the expression levels of several thousand genes in an organism in a single experiment. One of the most significant challenges in this research field is to select high relevant genes from gene expression data. To address this problem, feature selection is a well-known technique to eliminate unnecessary genes in order to ensure accurate classification results. This paper proposes a binary version of Political Optimizer (PO) to solve feature selection problem using gene expression data. Two transfer functions are used to design a binary PO. The first one is based on Sigmoid function and will be noted as BPO-S, while the second one is based on V-shaped function and will be noted as BPO-V. The proposed methods are evaluated using 9 biological datasets and compared with 8 binary well-known metaheuristics. The comparative results show the prevalent performance of the BPO methods especially BPO-V in comparison with other techniques.

Solving feature selection problems by combining mutation and crossover operations with the monarch butterfly optimization algorithm

Feature selection (FS) is used to solve hard optimization problems in artificial intelligence and data mining. In the FS process, some, rather than all of the features of a dataset are selected in order to both maximize classification accuracy and minimize the time required for computation. In this paper a FS wrapper method that uses K-nearest Neighbor (KSN) classification is subjected to two modifications using a current improvement algorithm, the Monarch Butterfly Optimization (MBO) algorithm. The first modification, named MBOICO, involves the utilization of an enhanced crossover operator to improve FS. The second, named MBOLF, integrates the Levy flight distribution into the MBO to improve convergence speed. Experiments are carried out on 25 benchmark data sets using the original MBO, MBOICO and MBOLF. The results show that MBOICO is superior, so its performance is also compared against that of four metaheuristic algorithms (PSO, ALO, WOASAT, and GA). The results indicate that it has a high classification accuracy rate of 93% on average for all datasets and significantly reduces the selection size. Hence, the findings demonstrate that the MBOICO outperforms the other algorithms in terms of classification accuracy and number of features chosen (selection size).

An Asymmetric Chaotic Competitive Swarm Optimization Algorithm for Feature Selection in High-Dimensional Data

This paper presents a method for feature selection in a high-dimensional classification context. The proposed method finds a candidate solution based on quality criteria using subset searching. In this study, the competitive swarm optimization (CSO) algorithm was implemented to solve feature selection problems in high-dimensional data. A new asymmetric chaotic function was proposed and used to generate the population and search for a CSO solution. Its histogram is right-skewed. The proposed method is named an asymmetric chaotic competitive swarm optimization algorithm (ACCSO). According to the asymmetrical property of the proposed chaotic map, ACCSO prefers zero than one. Therefore, the solution is very compact and can achieve high classification accuracy with a minimal feature subset for high-dimensional datasets. The proposed method was evaluated on 12 datasets, with dimensions ranging from 4 to 10,304. ACCSO was compared to the original CSO algorithm and other metaheuristic algorithms. Experimental results show that the proposed method can increase accuracy and it reduces the number of selected features. Compared to different optimization algorithms with other wrappers, the proposed method exhibits excellent performance.

A novel binary gaining–sharing knowledge-based optimization algorithm for feature selection

To obtain the optimal set of features in feature selection problems is the most challenging and prominent problem in machine learning. Very few human-related metaheuristic algorithms were developed and solved this type of problem. It motivated us to check the performance of recently developed gaining–sharing knowledge-based optimization algorithm (GSK), which is based on the concept of gaining and sharing knowledge of humans throughout their lifespan. It depends on two stages: beginners–intermediate gaining and sharing stage and intermediate–experts gaining and sharing stage. In this study, two approaches are proposed to solve feature selection problems: FS-BGSK: a novel binary version of GSK algorithm that relies on these two stages with knowledge factor 1 and FS-pBGSK: a population reduction technique that is employed on BGSK algorithm to enhance the exploration and exploitation quality of FS-BGSK. The proposed approaches are checked on twenty two feature selection benchmark datasets from UCI repository that contains small, medium and large dimensions datasets. The obtained results are compared with seven state-of-the-art metaheuristic algorithms; binary differential evolution, binary particle swarm optimization algorithm, binary bat algorithm, binary grey wolf optimizer, binary ant lion optimizer, binary dragonfly algorithm and binary salp swarm algorithm. It concludes that FS-pBGSK and FS-BGSK outperform other algorithms in terms of accuracy, convergence and robustness in most of the datasets.

Improved coral reefs optimization with adaptive $$\beta $$-hill climbing for feature selection

For any classification problem, the dimension of the feature vector used for classification has great importance. This is because, in a high-dimensional feature vector, it is found that some are non-informative or even redundant as they do not contribute to the learning process of the classifier. Rather, they may be the reason for low classification accuracy and high training time of the learning model. To address this issue, researchers apply various feature selection (FS) methods as found in the literature. In recent years, meta-heuristic algorithms have been proven to be effective in solving FS problems. The Coral Reefs Optimizer (CRO) which is a cellular type evolutionary algorithms has good tuning between its exploration and exploitation ability. This has motivated us to present an improved version of CRO with the inclusion of adaptive $$\beta $$ -hill climbing to increase the exploitation ability of CRO. The proposed method is assessed on 18 standard UCI-datasets by means of three distinct classifiers, KNN, Random Forest and Naive Bayes classifiers. It is also analyzed with 10 state-of-the-art meta-heuristics FS procedure, and the outputs show an excellent performance of the proposed FS method reaching better results than the previous methods considered here for comparison. The source code of this work is publicly available at https://github.com/ahmed-shameem/Projects .

A Duplication Analysis-Based Evolutionary Algorithm for Biobjective Feature Selection

Feature selection is a complex optimization problem with important real-world applications. Normally, its main target is to reduce the dimensionality of the dataset and increase the effectiveness of the classification. Owing to the population-inspired characteristics, different evolutionary algorithms (EAs) have been proposed to solve feature selection problems over the past decades. However, the majority of them only consider single-objective optimization while many real-world problems have multiple objectives, which creates a genuine demand for designing more suitable and effective EAs to handle multiobjective feature selection. A multiobjective feature selection problem usually consists of two objectives: one is to minimize the number of selected features and the other is to minimize the error of classification. In this article, we propose a duplication analysis-based EA (DAEA) for biobjective feature selection in classification. In the proposed algorithm, we make improvements on the basic dominance-based EA framework in three aspects: first, the reproduction process is modified to improve the quality of offspring; second, a duplication analysis method is proposed to filter out the redundant solutions; and third, a diversity-based selection method is adopted to further select the reserved solutions. In the experiments, we have compared the proposed algorithm with five state-of-the-art multiobjective EAs (MOEAs) and tested them on 20 classification datasets, using two widely used performance metrics. According to the empirical results, DAEA performs the best on most datasets, indicating that DAEA not only gains outstanding optimization performance but also obtains good classification and generalization results.

The monarch butterfly optimization algorithm for solving feature selection problems

Feature selection (FS) is considered to be a hard optimization problem in data mining and some artificial intelligence fields. It is a process where rather than studying all of the features of a whole dataset, some associated features of a problem are selected, the aim of which is to increase classification accuracy and reduce computational time. In this paper, a recent optimization algorithm, the monarch butterfly optimization (MBO) algorithm, is implemented with a wrapper FS method that uses the k-nearest neighbor (KNN) classifier. Experiments were implemented on 18 benchmark datasets. The results showed that, in comparison with four metaheuristic algorithms (WOASAT, ALO, GA and PSO), MBO was superior, giving a high rate of classification accuracy of, on average, 93% for all datasets as well as reducing the selection size significantly. Therefore, the use of the MBO to solve the FS problems has been proven through the results obtained to be effective and highly efficient in this field, and the results have also proven the strength of the balance between global and local search of MBO.