Optimal Feature Selection Algorithm Research Articles

An Electric Fish-Based Arithmetic Optimization Algorithm for Feature Selection.

With the widespread use of intelligent information systems, a massive amount of data with lots of irrelevant, noisy, and redundant features are collected; moreover, many features should be handled. Therefore, introducing an efficient feature selection (FS) approach becomes a challenging aim. In the recent decade, various artificial methods and swarm models inspired by biological and social systems have been proposed to solve different problems, including FS. Thus, in this paper, an innovative approach is proposed based on a hybrid integration between two intelligent algorithms, Electric fish optimization (EFO) and the arithmetic optimization algorithm (AOA), to boost the exploration stage of EFO to process the high dimensional FS problems with a remarkable convergence speed. The proposed EFOAOA is examined with eighteen datasets for different real-life applications. The EFOAOA results are compared with a set of recent state-of-the-art optimizers using a set of statistical metrics and the Friedman test. The comparisons show the positive impact of integrating the AOA operator in the EFO, as the proposed EFOAOA can identify the most important features with high accuracy and efficiency. Compared to the other FS methods whereas, it got the lowest features number and the highest accuracy in 50% and 67% of the datasets, respectively.

Two-stage improved Grey Wolf optimization algorithm for feature selection on high-dimensional classification

In recent years, evolutionary algorithms have shown great advantages in the field of feature selection because of their simplicity and potential global search capability. However, most of the existing feature selection algorithms based on evolutionary computation are wrapper methods, which are computationally expensive, especially for high-dimensional biomedical data. To significantly reduce the computational cost, it is essential to study an effective evaluation method. In this paper, a two-stage improved gray wolf optimization (IGWO) algorithm for feature selection on high-dimensional data is proposed. In the first stage, a multilayer perceptron (MLP) network with group lasso regularization terms is first trained to construct an integer optimization problem using the proposed algorithm for pre-selection of features and optimization of the hidden layer structure. The dataset is compressed using the feature subset obtained in the first stage. In the second stage, a multilayer perceptron network with group lasso regularization terms is retrained using the compressed dataset, and the proposed algorithm is employed to construct the discrete optimization problem for feature selection. Meanwhile, a rapid evaluation strategy is constructed to mitigate the evaluation cost and improve the evaluation efficiency in the feature selection process. The effectiveness of the algorithm was analyzed on ten gene expression datasets. The experimental results show that the proposed algorithm not only removes almost more than 95.7% of the features in all datasets, but also has better classification accuracy on the test set. In addition, the advantages of the proposed algorithm in terms of time consumption, classification accuracy and feature subset size become more and more prominent as the dimensionality of the feature selection problem increases. This indicates that the proposed algorithm is particularly suitable for solving high-dimensional feature selection problems.

A comparative analysis of meta-heuristic optimization algorithms for feature selection and feature weighting in neural networks

A comparative analysis of meta-heuristic optimization algorithms for feature selection and feature weighting in neural networks

Metaheuristic Search Based Feature Selection Methods for Classification of Cancer

Cancer is a cluster of diseases caused due to unusual cell growth. This paper aims to discover cancer prediction from the microarray gene expression data using the selected features. The metaheuristic search algorithms select the global and local optimal features using population and neighbourhood based algorithms. Although the ant colony optimization and genetic algorithm search for the global optimal features from the dataset entails enhanced classification, sometimes there occur some challenges in the selection of neighbourhood features. Against this background, two feature selection algorithms are proposed to hybridize tabu search, a neighbourhood based search algorithm with global optimal feature selection algorithm. Those are (1) Ant Colony Optimization and Tabu search with Fuzzy Rough set for Optimal feature selection (ACTFRO) algorithm, (2) Genetic algorithm and Tabu search with Fuzzy Rough set for Optimal feature selection (GATFRO) algorithm. The performance of proposed feature selection algorithms is assessed through a fuzzy rough nearest neighbour classifier using ten-fold cross validation. Four cancer medical datasets and one non-medical dataset are used to analyse the performance of the proposed algorithms in terms of classification accuracy, computation time, sensitivity, specificity, f-measure, receiver operation characteristics and positive predicted value. Results derived from the different performance metrics confirm that the proposed algorithms evidence effective global and local feature selection hybridization with improved results.

Adaptive Salp Swarm Algorithm as Optimal Feature Selection for Power Quality Disturbance Classification

Power quality disturbance (PQD) is an influential situation that significantly declines the reliability of electrical distribution systems. Therefore, PQD classification is an important process for preventing system reliability degradation. This paper introduces a novel algorithm called “adaptive salp swarm algorithm (SSA)” as an optimal feature selection algorithm for PQD classification. Feature extraction and classifier of the proposed classification system were based on the discrete wavelet and the probabilistic neural network, respectively. The classification was focused on the 13 types of power quality signals. The optimal number of selected features for the proposed classification system was firstly determined. Then, it demonstrated that the optimally selected features resulted in the highest classification accuracy of 98.77%. High performance of the proposed classification system in the noisy environment, as well as based on the real dataset was also verified. Furthermore, the proposed SSA indicates a very high convergence rate compared to other well-known algorithms. A comparison of the proposed classification system’s performance to existing works was also carried out, revealing that the proposed system’s accuracy is on a high-range scale. Hence, the adaptive SSA becomes another efficient optimal feature selection algorithm for PQD classification.

Predicting the Risk Genes of Autism Spectrum Disorders.

Autism spectrum disorder (ASD) refers to a wide spectrum of neurodevelopmental disorders that emerge during infancy and continue throughout a lifespan. Although substantial efforts have been made to develop therapeutic approaches, core symptoms persist lifelong in ASD patients. Identifying the brain temporospatial regions where the risk genes are expressed in ASD patients may help to improve the therapeutic strategies. Accordingly, this work aims to predict the risk genes of ASD and identify the temporospatial regions of the brain structures at different developmental time points for exploring the specificity of ASD gene expression in the brain that would help in possible ASD detection in the future. A dataset consisting of 13 developmental stages ranging from 8 weeks post-conception to 8 years from 26 brain structures was retrieved from the BrainSpan atlas. This work proposes a support vector machine–based risk gene prediction method ASD-Risk to distinguish the risk genes of ASD and non-ASD genes. ASD-Risk used an optimal feature selection algorithm called inheritable bi-objective combinatorial genetic algorithm to identify the brain temporospatial regions for prediction of the risk genes of ASD. ASD-Risk achieved a 10-fold cross-validation accuracy, sensitivity, specificity, area under a receiver operating characteristic curve, and a test accuracy of 81.83%, 0.84, 0.79, 0.84, and 72.27%, respectively. We prioritized the temporospatial features according to their contribution to the prediction accuracy. The top identified temporospatial regions of the brain for risk gene prediction included the posteroventral parietal cortex at 13 post-conception weeks feature. The identified temporospatial features would help to explore the risk genes that are specifically expressed in different brain regions of ASD patients.

Secure and Usable Handshake Based Pairing for Wrist-Worn Smart Devices on Different Users

Wrist-worn smart devices are being used to share various sensitive personal information in various fields such as social, medical treatment, sports, etc. Secure pairing establishing a trusted channel between involved devices is a prerequisite to ensure data transmission security. Handshake has been employed to realize secure pairing between devices worn by different users without pre-shared knowledge, the participation of third parties or complicated user interactions. However, existing schemes cannot meet the practical requirement in terms of time delay and security. In this paper, we present a secure and usable pairing scheme utilizing the handshake acceleration data. Specifically, we propose an optimal feature selection algorithm based on Euclidean distance sorting which improves the success rate and security of the system. In addition, we use Pearson correlation coefficient based feature sequence similarity measurement approach to enhance the accuracy of pairing scheme. Theoretical and experimental security analysis indicates that our solution can resist active and passive attacks. What’s more, the experimental results demonstrate that the proposed solution has a high key generation rate of 87 bits per second, and completes security pairing in less than 4 s.

Detection of white blood cells using optimized qGWO

This paper presents an optimized quantum Grey Wolf Optimization algorithm (qGWO), which is an enhanced version of the Grey Wolf optimization algorithm for feature selection of blood cells, which can further used for the detection of WBCs. White blood cells count in the human body determines the immune system of the human body. A deviation in the count of WBCs from the normal cell count in the human body may indicate an abnormal condition. The proposed model uses a quantum grey wolf optimization algorithm for the detection of White Blood cells among the dataset of various types of blood cells. Quantum Grey Wolf algorithm is used to find the minimal set of optimal features from the set of available features to detect the White Blood Cells optimally. As the ordinary Grey Wolf Optimization algorithm also used to find the minimal set of optimal features, but the features selected by qGWO are better in terms of computational time. Further, several classification algorithms such as Support Vector Machine (SVM), Random Forest algorithm, K Nearest Neighbor(KNN) algorithm are applied to the model to predict its accuracy for the selected subset of features after feature selection. The performance of several classifiers is compared, and the model attained the maximum accuracy of 97.8% using KNN with minimum computational time. The result obtained shows that the algorithm proposed is capable of finding an optimal subset of features and maximizing the accuracy.

LAGOA: Learning automata based grasshopper optimization algorithm for feature selection in disease datasets

In predictive modelling it is important to use any feature selection methods as irrelevant features when used with powerful classifiers can lead to over-fitting and thus create models which fail to perform as good as when these features are not used. Particularly it is important in case of disease datasets where various features or attributes are available through the patients’ medical records and many features in these datasets may not be relevant to the diagnosis of some specific disease. Wrong models in this case can be disastrous and lead to wrong diagnosis, and maybe in extreme cases lead to loss of life. To this end, we have used a wrapper based feature selection model for the said purpose. In recent years, Grasshopper Optimization Algorithm (GOA) has proved its superiority over other optimization algorithms in different research areas. In this paper, we propose an improved version of GOA, called (LAGOA), which uses Learning Automata (LA) for adjusting the parameters of GOA in an adaptive way, and two-phase mutation for enhancing exploitation capability of the algorithm. LA is used for adjusting the parameter values of each grasshopper in the population individually. In two-phase mutation the first phase reduces the number of selected features while preserving high classification accuracy, while the second phase adds relevant features which increase the classification accuracy. Proposed method has been applied to Breast Cancer (Wisconsin), Breast Cancer (Diagnosis), Statlog (Heart), Lung Cancer, SpectF Heart and Hepatitis datasets taken from UCI Machine Learning Repository. Experimental results confirm its superiority over state-of-the-art methods considered here for comparison.

A Hybrid Supervised Machine Learning Classifier System for Breast Cancer Prognosis Using Feature Selection and Data Imbalance Handling Approaches

Nowadays, breast cancer is the most frequent cancer among women. Early detection is a critical issue that can be effectively achieved by machine learning (ML) techniques. Thus in this article, the methods to improve the accuracy of ML classification models for the prognosis of breast cancer are investigated. Wrapper-based feature selection approach along with nature-inspired algorithms such as Particle Swarm Optimization, Genetic Search, and Greedy Stepwise has been used to identify the important features. On these selected features popular machine learning classifiers Support Vector Machine, J48 (C4.5 Decision Tree Algorithm), Multilayer-Perceptron (a feed-forward ANN) were used in the system. The methodology of the proposed system is structured into five stages which include (1) Data Pre-processing; (2) Data imbalance handling; (3) Feature Selection; (4) Machine Learning Classifiers; (5) classifier’s performance evaluation. The dataset under this research experimentation is referred from the UCI Machine Learning Repository, named Breast Cancer Wisconsin (Diagnostic) Data Set. This article indicated that the J48 decision tree classifier is the appropriate machine learning-based classifier for optimum breast cancer prognosis. Support Vector Machine with Particle Swarm Optimization algorithm for feature selection achieves the accuracy of 98.24%, MCC = 0.961, Sensitivity = 99.11%, Specificity = 96.54%, and Kappa statistics of 0.9606. It is also observed that the J48 Decision Tree classifier with the Genetic Search algorithm for feature selection achieves the accuracy of 98.83%, MCC = 0.974, Sensitivity = 98.95%, Specificity = 98.58%, and Kappa statistics of 0.9735. Furthermore, Multilayer Perceptron ANN classifier with Genetic Search algorithm for feature selection achieves the accuracy of 98.59%, MCC = 0.968, Sensitivity = 98.6%, Specificity = 98.57%, and Kappa statistics of 0.9682.

High-Accuracy Power Quality Disturbance Classification Using the Adaptive ABC-PSO as Optimal Feature Selection Algorithm

Power quality disturbance (PQD) is an important issue in electrical distribution systems that needs to be detected promptly and identified to prevent the degradation of system reliability. This work proposes a PQD classification using a novel algorithm, comprised of the artificial bee colony (ABC) and the particle swarm optimization (PSO) algorithms, called “adaptive ABC-PSO” as the feature selection algorithm. The proposed adaptive technique is applied to a combination of ABC and PSO algorithms, and then used as the feature selection algorithm. A discrete wavelet transform is used as the feature extraction method, and a probabilistic neural network is used as the classifier. We found that the highest classification accuracy (99.31%) could be achieved through nine optimally selected features out of all 72 extracted features. Moreover, the proposed PQD classification system demonstrated high performance in a noisy environment, as well as the real distribution system. When comparing the presented PQD classification system’s performance to previous studies, PQD classification accuracy using adaptive ABC-PSO as the optimal feature selection algorithm is considered to be at a high-range scale; therefore, the adaptive ABC-PSO algorithm can be used to classify the PQD in a practical electrical distribution system.

Novel self-adjusted particle swarm optimization algorithm for feature selection

Due to the ever increasing number of features in the most practical application fields, i.e, expert and intelligent systems, feature selection (FS) has become a promising pre-processing step for a particular task (i.e., classification and regression) in the last few decades. FS aims at selecting the optimal feature subset from the original feature dataset by removing redundant and irrelevant features, which improve the performance of the learning models. In this paper, a novel self-adjusted particle swarm optimization algorithm (SAPSO) is proposed for selecting the optimal feature subset for classification datasets. In SAPSO, we make three improvements: First, a new learning model of particles, which can extract much more useful knowledge from multiple information providers, is used to enhance the diversity of particles. Second, one-flip neighborhood search strategy is adopted to strengthen the local search ability of a swarm when the swarm enters a period of stagnation. Finally, a population replacement process is conducted, which bases on the part of new particles generated by the neighborhood search strategy, to enhance the diversity of the swarm. Moreover, the k-nearest neighbor method is used as a classifier to evaluate the classification accuracy of a particle. The proposed method is benchmarked on 10 well-known UCI datasets and the results are compared with 9 state-of-the-art wrapper-based FS methods. From the results, it is observed that the proposed approach significantly outperforms others on most the 10 datasets.

A Robust Multi-Objective Feature Selection Model Based on Local Neighborhood Multi-Verse Optimization

Classification tasks often include, among the large number of features to be processed in the datasets, many irrelevant and redundant ones, which can even decrease the efficiency of classifiers. Feature Selection (FS) is the most common preprocessing technique utilized to overcome the drawbacks of the high dimensionality of datasets and often has two conflicting objectives: The first function aims to maximize the classification performance or reduce the error rate of the classifier. In contrast, the second function is designed to minimize the number of features. However, the majority of wrapper FS techniques are developed for single-objective scenarios. Multi-verse optimizer (MVO) is considered as one of the well-regarded optimization approaches in recent years. In this paper, the binary multi-objective variant of MVO (MOMVO) is proposed to deal with feature selection tasks. The standard MOMVO suffers from local optima stagnation, so we propose an improved binary MOMVO to deal with this issue using the memory concept and personal best of the universes. The experimental results and comparisons indicate that the proposed binary MOMVO approach can effectively eliminate irrelevant and/or redundant features and maintain a minimum classification error rate when dealing with different datasets compared with the most popular feature selection techniques. Furthermore, the 14 benchmark datasets showed that the proposed approach outperforms the stat-of-art multi-objective optimization algorithms for feature selection.

EAOA: An Enhanced Archimedes Optimization Algorithm for Feature Selection in Classification

Feature selection plays a crucial role in order to mitigate the high dimensional feature space in different classification problems. The computational cost is reduced, and the accuracy of the classification is improved by reducing the dimension of feature space. Hence, in the classification task, finding the optimal subset of features is of utmost importance. Metaheuristic techniques have proved their efficacy in solving many real-world optimization issues. One of the recently introduced physics-inspired optimization methods is Archimedes Optimization Algorithm (AOA). This paper proposes an Enhanced Archimedes Optimization Algorithm (EAOA) by adding a new parameter that depends on the step length of each individual while revising the individual location. The EAOA algorithm is proposed to improve the AOA exploration and exploitation balance and enhance the classification performance for the feature selection issue in real-world data sets. Experiments were performed on twenty-three standard benchmark functions and sixteen real-world data sets to investigate the performance of the proposed EAOA algorithm. The experimental results based on the standard benchmark functions show that the EAOA algorithm provides very competitive results compared to the basic AOA algorithm and five well-known optimization algorithms in terms of improved exploitation, exploration, local optima avoidance, and convergence rate. In addition, the results based on sixteen real-world data sets ascertain that reduced feature subset yields higher classification performance when compared with the other feature selection methods.

A search space enhanced modified whale optimization algorithm for feature selection in large-scale microarray datasets

Objectives: To enhance the microarray data classification accuracy, to accelerate the convergence speed of classifier, and Modified Whale Optimization Algorithm (MWOA), refine the best balance among local exploitation and global exploration, a Search space enhanced Modified Whale Optimization Algorithm (SMWOA) is the proposed task. Methods: The SMWOA selects the optimal features stands on the Levy flight method and quadratic interpolation method. Levy flight which employs for acceleration convergence speed of SMWOA andalso holds the result from local optima builds up by the population assortment.A quadratic interpolation takes up the exploitation stage for deeper searching within the search area. Finding: In addition to this, a self-adaptive control parameter is introduced to make a clear variation to the solution quality. Itrefines the best equity among the local exploitation method by global exploration method. After selection of features, those are processed in Naïve Bayes (NB), Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Artificial Neural Network (ANN) classifiers for cancer detection. Novelty: The classification accuracy is improved by processing the most discriminative features in the classifiers. The overall accuracy, specificity, sensitivity, F1-score and average error of SMWOA-ANN are 6.7%, 5.6%, 7.3% and 5.6% greater than MWOA-ANN respectively for cancer detection. Keywords: Gene expression data; dimensionality reduction; feature selection; modified whale optimization algorithm (MWOA); search space enhanced modified whale optimization algorithm (WOA)

An Asymmetric Chaotic Competitive Swarm Optimization Algorithm for Feature Selection in High-Dimensional Data

This paper presents a method for feature selection in a high-dimensional classification context. The proposed method finds a candidate solution based on quality criteria using subset searching. In this study, the competitive swarm optimization (CSO) algorithm was implemented to solve feature selection problems in high-dimensional data. A new asymmetric chaotic function was proposed and used to generate the population and search for a CSO solution. Its histogram is right-skewed. The proposed method is named an asymmetric chaotic competitive swarm optimization algorithm (ACCSO). According to the asymmetrical property of the proposed chaotic map, ACCSO prefers zero than one. Therefore, the solution is very compact and can achieve high classification accuracy with a minimal feature subset for high-dimensional datasets. The proposed method was evaluated on 12 datasets, with dimensions ranging from 4 to 10,304. ACCSO was compared to the original CSO algorithm and other metaheuristic algorithms. Experimental results show that the proposed method can increase accuracy and it reduces the number of selected features. Compared to different optimization algorithms with other wrappers, the proposed method exhibits excellent performance.

A novel binary gaining–sharing knowledge-based optimization algorithm for feature selection

To obtain the optimal set of features in feature selection problems is the most challenging and prominent problem in machine learning. Very few human-related metaheuristic algorithms were developed and solved this type of problem. It motivated us to check the performance of recently developed gaining–sharing knowledge-based optimization algorithm (GSK), which is based on the concept of gaining and sharing knowledge of humans throughout their lifespan. It depends on two stages: beginners–intermediate gaining and sharing stage and intermediate–experts gaining and sharing stage. In this study, two approaches are proposed to solve feature selection problems: FS-BGSK: a novel binary version of GSK algorithm that relies on these two stages with knowledge factor 1 and FS-pBGSK: a population reduction technique that is employed on BGSK algorithm to enhance the exploration and exploitation quality of FS-BGSK. The proposed approaches are checked on twenty two feature selection benchmark datasets from UCI repository that contains small, medium and large dimensions datasets. The obtained results are compared with seven state-of-the-art metaheuristic algorithms; binary differential evolution, binary particle swarm optimization algorithm, binary bat algorithm, binary grey wolf optimizer, binary ant lion optimizer, binary dragonfly algorithm and binary salp swarm algorithm. It concludes that FS-pBGSK and FS-BGSK outperform other algorithms in terms of accuracy, convergence and robustness in most of the datasets.

Vocal acoustic analysis and machine learning for the identification of schizophrenia

Psychiatry still needs objective biomarkers. In the context of schizophrenia, there are speech abnormalities such as tangentiality, derailment, alogia, neologisms, poverty of speech, and aprosodia. There is a growing interest in speech signals features as possible indicators of schizophrenia. This article aims to develop an intelligent tool for detection of schizophrenia using vocal patterns and machine learning techniques. The main advantages of this type of solution are the low cost, high performance, and for being non-invasive. Thirty-one individuals over 18 years old were selected, 20 with previous diagnosis of schizophrenia, and 11 healthy controls. Their speech was audio-recorded in naturalistic settings, during a routine medical assessment for psychiatric patients. In the case of healthy patients, the recordings were made in different environments. Recordings were pre-processed, excluding non-participant voices. We extracted 33 features. We used the particle swarm optimization algorithm for feature selection. The classifiers’ performance was analyzed with four metrics: accuracy, sensibility, specificity, and kappa index. Best results were achieved when considering all 33 extracted features. Within machine models, support vector machines (SVM) models provided the greatest classification performance, with mean accuracy of 91.76% for PUK kernel. Our results outperform those from most studies published so far for the detection of schizophrenia based on acoustic patterns. The use of machine learning classifiers using vocal parameters, in particular SVM, has shown to be very promising for the detection of schizophrenia. Nevertheless, further experiments with a larger sample will be necessary to validate our findings.

A novel quantum grasshopper optimization algorithm for feature selection

Feature selection is an indispensable work to make the data mining more effective. It reduces the computational complexity and effectively improves the performance of learning models. The exhaustive algorithm and the greedy algorithm cannot adapt to the current increasing number of features when finding the potential optimal feature subset. Therefore, the feasible way for feature selection called swarm intelligence algorithms becomes popular. The grasshopper optimization algorithm is a novel swarm intelligence algorithm which has good performance. In this study, we improve the grasshopper optimization algorithm by applying quantum method. A dynamic population quantum binary grasshopper optimization algorithm based on mutual information and rough set theory (DQBGOA_MR) for feature selection is proposed. Through the quantization of grasshopper individuals, the search scope of feature space is improved, and a good balance is achieved between exploration and exploitation. The premature and catastrophe strategies are used to avoid converge prematurely and fall into a local optimum. Moreover, the rough set and mutual information based evaluation criterion is defined which considers both effectiveness of selected features and the relation among unselected features, selected features, and classes. The proposed method is evaluated by extensive experiments in twenty UCI datasets. Experimental results show that DQBGOA_MR has better feature reduction rate, higher classification accuracy and more stable results compared to other swarm intelligence algorithms.

Multi-objective symbiotic organism search algorithm for optimal feature selection in brain computer interfaces

Feature selection is crucial to develop a brain computer interface (BCI) system which has high classification accuracy and less computational complexity in especially a large feature space. Feature selection (FS) problem has been solved by many various methods. Among these methods, especially evolutionary computation (EC) techniques have gained a lot of attention in recent years. However, there are very few studies in the literature that consider FS problem as a multi-objective problem to find the optimal trade-off between classification accuracy and the number of selected features. Therefore, in this paper, a non-dominated sorting multi-objective symbiotic organism search (NSMOSOS) algorithm is proposed to generate the optimal feature subset in BCI. The efficiency and robustness of the proposed algorithm as a feature selection method is investigated in two datasets based on motor imagery. The highest classification accuracies of NSMOSOS for dataset 1 and dataset 2 are obtained 97.86% with 11 features and 96.57% with average 19 features, respectively. The obtained results demonstrate that the proposed method achieves satisfying results with regard to both the classification accuracy improvement and feature reduction rates for both datasets. The superiority of the proposed method is verified compared with the existing methods for both datasets. Besides, three different versions of symbiotic search organism (SOS) algorithm are improved, and pros and cons of these algorithms are evaluated compared with each other. In conclusion, the paper indicates that the proposed NSMOSOS algorithm is an efficient and practicable technique for FS problem and could be helpful in developing the BCI applications.