Issue Of Feature Selection Research Articles

Dipper Throated Optimization Algorithm for Unconstrained Function and Feature Selection

Dipper throated optimization (DTO) algorithm is a novel with a very efficient metaheuristic inspired by the dipper throated bird. DTO has its unique hunting technique by performing rapid bowing movements. To show the efficiency of the proposed algorithm, DTO is tested and compared to the algorithms of Particle Swarm Optimization (PSO), Whale Optimization Algorithm (WOA), Grey Wolf Optimizer (GWO), and Genetic Algorithm (GA) based on the seven unimodal benchmark functions. Then, ANOVA and Wilcoxon rank-sum tests are performed to confirm the effectiveness of the DTO compared to other optimization techniques. Additionally, to demonstrate the proposed algorithm's suitability for solving complex real-world issues, DTO is used to solve the feature selection problem. The strategy of using DTOs as feature selection is evaluated using commonly used data sets from the University of California at Irvine (UCI) repository. The findings indicate that the DTO outperforms all other algorithms in addressing feature selection issues, demonstrating the proposed algorithm's capabilities to solve complex real-world situations.

Computational Intelligence-Based Financial Crisis Prediction Model Using Feature Subset Selection with Optimal Deep Belief Network.

At present times, financial decisions are mainly based on the classifier technique, which is utilized to allocate a collection of observations into fixed groups. A diverse set of data classifier approaches were presented for forecasting the financial crisis of an institution using the past data. An essential process toward the design of a precise financial crisis prediction (FCP) approach comprises the choice of proper variables (features) that are related to the issues at hand. This is termed as a feature selection (FS) issue that assists to improvise the classifier results. Besides, computational intelligence techniques can be used as a classification model to determine the financial crisis of an organization. In this view, this article introduces a new FS using elephant herd optimization (EHO) with modified water wave optimization (MWWO) algorithm-based deep belief network (DBN) for FCP. The EHO algorithm is applied as a feature selector, and MWWO-DBN is utilized for the classification process. The application of the MWWO algorithm helps to tune the parameters of the DBN model, and the choice of optimal feature subset from the EHO algorithm leads to enhanced classification performance. The experimental results of the proposed model are tested against three benchmark data sets, namely AnalcatData, German Credit, and Australian Credit. The obtained simulation results indicated the superior performance of the proposed model by attaining maximum classification performance.

Anomaly Intrusion Detection Using SVM and C4.5 Classification With an Improved Particle Swarm Optimization (I-PSO)

In the last decade, many researchers have proposed several models of classification algorithms for enhancing the accuracy performance of IDSs. However, there is a minor issue arising in the classifier's incapability to process high-dimensional data. Using several classifiers always outperforms a single classifier's performance. This paper proposes a novel intrusion detection system by classifying data with SVM as well as C4.5 decision tree algorithm. The NSL-KDD dataset is first preprocessed with principal component analysis (PCA) and later feature selected with an improved particle swarm optimization (I-PSO). This framework improved the time consumption and inaccurate feature selection issues in other methodologies. Upon simplifying features more effectively, the outcomes display an excellent agreement with the conventional PSO techniques and their results, and also produce enhanced outcomes when compared to only single classifier. The results demonstrate better performance when subject to different attack-scenarios and can be used for enterprise network security applications.

A Comparison of Feature Selection and Forecasting Machine Learning Algorithms for Predicting Glycaemia in Type 1 Diabetes Mellitus

Type 1 diabetes mellitus (DM1) is a metabolic disease derived from falls in pancreatic insulin production resulting in chronic hyperglycemia. DM1 subjects usually have to undertake a number of assessments of blood glucose levels every day, employing capillary glucometers for the monitoring of blood glucose dynamics. In recent years, advances in technology have allowed for the creation of revolutionary biosensors and continuous glucose monitoring (CGM) techniques. This has enabled the monitoring of a subject’s blood glucose level in real time. On the other hand, few attempts have been made to apply machine learning techniques to predicting glycaemia levels, but dealing with a database containing such a high level of variables is problematic. In this sense, to the best of the authors’ knowledge, the issues of proper feature selection (FS)—the stage before applying predictive algorithms—have not been subject to in-depth discussion and comparison in past research when it comes to forecasting glycaemia. Therefore, in order to assess how a proper FS stage could improve the accuracy of the glycaemia forecasted, this work has developed six FS techniques alongside four predictive algorithms, applying them to a full dataset of biomedical features related to glycaemia. These were harvested through a wide-ranging passive monitoring process involving 25 patients with DM1 in practical real-life scenarios. From the obtained results, we affirm that Random Forest (RF) as both predictive algorithm and FS strategy offers the best average performance (Root Median Square Error, RMSE = 18.54 mg/dL) throughout the 12 considered predictive horizons (up to 60 min in steps of 5 min), showing Support Vector Machines (SVM) to have the best accuracy as a forecasting algorithm when considering, in turn, the average of the six FS techniques applied (RMSE = 20.58 mg/dL).

Auto-HMM-LMF: feature selection based method for prediction of drug response via autoencoder and hidden Markov model

BackgroundPredicting the response of cancer cell lines to specific drugs is an essential problem in personalized medicine. Since drug response is closely associated with genomic information in cancer cells, some large panels of several hundred human cancer cell lines are organized with genomic and pharmacogenomic data. Although several methods have been developed to predict the drug response, there are many challenges in achieving accurate predictions. This study proposes a novel feature selection-based method, named Auto-HMM-LMF, to predict cell line-drug associations accurately. Because of the vast dimensions of the feature space for predicting the drug response, Auto-HMM-LMF focuses on the feature selection issue for exploiting a subset of inputs with a significant contribution.ResultsThis research introduces a novel method for feature selection of mutation data based on signature assignments and hidden Markov models. Also, we use the autoencoder models for feature selection of gene expression and copy number variation data. After selecting features, the logistic matrix factorization model is applied to predict drug response values. Besides, by comparing to one of the most powerful feature selection methods, the ensemble feature selection method (EFS), we showed that the performance of the predictive model based on selected features introduced in this paper is much better for drug response prediction. Two datasets, the Genomics of Drug Sensitivity in Cancer (GDSC) and Cancer Cell Line Encyclopedia (CCLE) are used to indicate the efficiency of the proposed method across unseen patient cell-line. Evaluation of the proposed model showed that Auto-HMM-LMF could improve the accuracy of the results of the state-of-the-art algorithms, and it can find useful features for the logistic matrix factorization method.ConclusionsWe depicted an application of Auto-HMM-LMF in exploring the new candidate drugs for head and neck cancer that showed the proposed method is useful in drug repositioning and personalized medicine. The source code of Auto-HMM-LMF method is available in https://github.com/emdadi/Auto-HMM-LMF.

EAOA: An Enhanced Archimedes Optimization Algorithm for Feature Selection in Classification

Feature selection plays a crucial role in order to mitigate the high dimensional feature space in different classification problems. The computational cost is reduced, and the accuracy of the classification is improved by reducing the dimension of feature space. Hence, in the classification task, finding the optimal subset of features is of utmost importance. Metaheuristic techniques have proved their efficacy in solving many real-world optimization issues. One of the recently introduced physics-inspired optimization methods is Archimedes Optimization Algorithm (AOA). This paper proposes an Enhanced Archimedes Optimization Algorithm (EAOA) by adding a new parameter that depends on the step length of each individual while revising the individual location. The EAOA algorithm is proposed to improve the AOA exploration and exploitation balance and enhance the classification performance for the feature selection issue in real-world data sets. Experiments were performed on twenty-three standard benchmark functions and sixteen real-world data sets to investigate the performance of the proposed EAOA algorithm. The experimental results based on the standard benchmark functions show that the EAOA algorithm provides very competitive results compared to the basic AOA algorithm and five well-known optimization algorithms in terms of improved exploitation, exploration, local optima avoidance, and convergence rate. In addition, the results based on sixteen real-world data sets ascertain that reduced feature subset yields higher classification performance when compared with the other feature selection methods.

BEPO: A novel binary emperor penguin optimizer for automatic feature selection

Emperor Penguin Optimizer (EPO) is a metaheuristic algorithm which is recently developed and illustrates the emperor penguin’s huddling behaviour. However, the original version of the EPO will fix issues that are continuing in fact but not discrete. The eight separate EPO variants have been provided in this article. Four transfer features, s-shaped and v-shaped, that are used in order to map the search space into a separate research space are considered in the proposed algorithm. The output of the proposed algorithm is validated using 25 standard benchmark functions. It also analyses the statistical sense of the proposed algorithm. Experimental findings and comparisons suggest that the proposed algorithm performs better than other algorithms. The solution also applies to the issue of feature selection. The findings reveal the supremacy of the binary emperor penguin optimization algorithm.

Selecting critical features for data classification based on machine learning methods

Feature selection becomes prominent, especially in the data sets with many variables and features. It will eliminate unimportant variables and improve the accuracy as well as the performance of classification. Random Forest has emerged as a quite useful algorithm that can handle the feature selection issue even with a higher number of variables. In this paper, we use three popular datasets with a higher number of variables (Bank Marketing, Car Evaluation Database, Human Activity Recognition Using Smartphones) to conduct the experiment. There are four main reasons why feature selection is essential. First, to simplify the model by reducing the number of parameters, next to decrease the training time, to reduce overfilling by enhancing generalization, and to avoid the curse of dimensionality. Besides, we evaluate and compare each accuracy and performance of the classification model, such as Random Forest (RF), Support Vector Machines (SVM), K-Nearest Neighbors (KNN), and Linear Discriminant Analysis (LDA). The highest accuracy of the model is the best classifier. Practically, this paper adopts Random Forest to select the important feature in classification. Our experiments clearly show the comparative study of the RF algorithm from different perspectives. Furthermore, we compare the result of the dataset with and without essential features selection by RF methods varImp(), Boruta, and Recursive Feature Elimination (RFE) to get the best percentage accuracy and kappa. Experimental results demonstrate that Random Forest achieves a better performance in all experiment groups.

Target spectrum based feature selection (TSFS): A new method based on chain coding for target detection problems

One of the controversial issues in hyperspectral remote sensing methods for target detection is whether the feature selection will be useful. Generally, feature selection methods are divided into variance based and wavelength based methods. Variance based feature selection methods like information-theory-based methods may eliminate the distinctive features because the distinctive features probably are not statistical principal components. Distinctive features are crucial to distinguish target from background and are maintained in wavelength based methods which concentrate on wavelength information. However, beside to wavelength-based information, target spectrum fluctuations are also critical for target detection. In addition, the wavelength based methods are often time consuming iterative methods with high computational cost. This paper introduces a new feature selection method considering hyperspectral target spectrum. The proposed algorithm has been developed based on Chain coding idea. We proposed Chain Filtering, Chain Encoding, and Chain Statistics as filter, embedded, and wrapper feature selection methods. In this paper, Chain filtering, Chain statistics, and Chain encoding approaches have been compared with different types of feature selection methods such as Principle Component Analysis (PCA) and Minimum Noise Fraction (MNF). Numerical tests have been executed using 4 datasets including Cuprite Nevada and Jasper Ridge datasets from AVIRIS, Botswana, and local datasets of Isfahan province from Hyperion sensor and using Constrained Energy Minimization (CEM) target detection. The results show the accuracy of target detection applying the proposed feature selection method increases from 85% to about 92% for Kaolinite and from 77% to 96% for Buddingtonite in comparison with PCA for Cuprite dataset. Furthermore, the increments more than 5% and 17.5% will be achieved in comparison with MNF, respectively. The results have shown that not only the proposed method overcomes the accuracy decrement issue of feature selection in target detection, but also it improves target detection accuracy by eliminating non-informative features for target detection applications. So feature selection will be an efficient tool for target detection if the applied feature selection method picks out the distinctive features well.

Under-Sampling and Feature Selection Algorithms for S2SMLP

Imbalance learning is a hot topic in the data mining and machine learning domains. Data-level, algorithm-level and ensemble solutions are the three main methods proposed thus far to address imbalance learning. To alleviate the issues of data explosion and feature selection in multilayer perceptron based on simultaneous two-sample representation(S2SMLP), in this paper, firstly, spectral clustering is exploited to select majority samples so as to construct a smaller training dataset for the classifier. We divide all majority samples into many clusters through spectral clustering, extract different numbers of representative samples from a cluster according to the size of each cluster, the average distance between the minority class and all samples of the cluster, then construct the training dataset of the classifier by combining these extracted samples from the majority class and all minority samples. Secondly, we propose a novel feature selection method based on the pairwise samples distance constraint, which considers the class labels of paired samples, select the features which push two similar samples closer together and pull two different samples farther apart. Finally, we conduct extensive experiments on 44 two-class imbalanced datasets and four high-dimensional DNA microarray datasets. The experimental results demonstrate that our proposed algorithms outperform some state-of-the-art algorithms in terms of $F\textrm {-measure},G\textrm {-mean}$ and AUC.

Feature selection for probabilistic load forecasting via sparse penalized quantile regression

Probabilistic load forecasting (PLF) is able to present the uncertainty information of the future loads. It is the basis of stochastic power system planning and operation. Recent works on PLF mainly focus on how to develop and combine forecasting models, while the feature selection issue has not been thoroughly investigated for PLF. This paper fills the gap by proposing a feature selection method for PLF via sparse L_1-norm penalized quantile regression. It can be viewed as an extension from point forecasting-based feature selection to probabilistic forecasting-based feature selection. Since both the number of training samples and the number of features to be selected are very large, the feature selection process is casted as a large-scale convex optimization problem. The alternating direction method of multipliers is applied to solve the problem in an efficient manner. We conduct case studies on the open datasets of ten areas. Numerical results show that the proposed feature selection method can improve the performance of the probabilistic forecasting and outperforms traditional least absolute shrinkage and selection operator method.

Autoencoder Based Feature Selection Method for Classification of Anticancer Drug Response.

Anticancer drug responses can be varied for individual patients. This difference is mainly caused by genetic reasons, like mutations and RNA expression. Thus, these genetic features are often used to construct classification models to predict the drug response. This research focuses on the feature selection issue for the classification models. Because of the vast dimensions of the feature space for predicting drug response, the autoencoder network was first built, and a subset of inputs with the important contribution was selected. Then by using the Boruta algorithm, a further small set of features was determined for the random forest, which was used to predict drug response. Two datasets, GDSC and CCLE, were used to illustrate the efficiency of the proposed method.

Drug sensitivity prediction with high-dimensional mixture regression.

This paper proposes a mixture regression model-based method for drug sensitivity prediction. The proposed method explicitly addresses two fundamental issues in drug sensitivity prediction, namely, population heterogeneity and feature selection pertaining to each of the subpopulations. The mixture regression model is estimated using the imputation-conditional consistency algorithm, and the resulting estimator is consistent. This paper also proposes an average-BIC criterion for determining the number of components for the mixture regression model. The proposed method is applied to the CCLE dataset, and the numerical results indicate that the proposed method can make a drastic improvement over the existing ones, such as random forest, support vector regression, and regularized linear regression, in both drug sensitivity prediction and feature selection. The p-values for the comparisons in drug sensitivity prediction can reach the order O(10−8) or lower for the drugs with heterogeneous populations.

Volatile compounds profiling by using proton transfer reaction-time of flight-mass spectrometry (PTR-ToF-MS). The case study of dark chocolates organoleptic differences.

Direct-injection mass spectrometry (DIMS) techniques have evolved into powerful methods to analyse volatile organic compounds (VOCs) without the need of chromatographic separation. Combined to chemometrics, they have been used in many domains to solve sample categorization issues based on volatilome determination. In this paper, different DIMS methods that have largely outperformed conventional electronic noses (e-noses) in classification tasks are briefly reviewed, with an emphasis on food-related applications. A particular attention is paid to proton transfer reaction mass spectrometry (PTR-MS), and many results obtained using the powerful PTR-time of flight-MS (PTR-ToF-MS) instrument are reviewed. Data analysis and feature selection issues are also summarized and discussed. As a case study, a challenging problem of classification of dark chocolates that has been previously assessed by sensory evaluation in four distinct categories is presented. The VOC profiles of a set of 206 chocolate samples classified in the four sensory categories were analysed by PTR-ToF-MS. A supervised multivariate data analysis based on partial least squares regression-discriminant analysis allowed the construction of a classification model that showed excellent prediction capability: 97% of a test set of 62 samples were correctly predicted in the sensory categories. Tentative identification of ions aided characterisation of chocolate classes. Variable selection using dedicated methods pinpointed some volatile compounds important for the discrimination of the chocolates. Among them, the CovSel method was used for the first time on PTR-MS data resulting in a selection of 10 features that allowed a good prediction to be achieved. Finally, challenges and future needs in the field are discussed.

Feature Selection and Its Use in Big Data: Challenges, Methods, and Trends

Feature selection has been an important research area in data mining, which chooses a subset of relevant features for use in the model building. This paper aims to provide an overview of feature selection methods for big data mining. First, it discusses the current challenges and difficulties faced when mining valuable information from big data. A comprehensive review of existing feature selection methods in big data is then presented. Herein, we approach the review from two aspects: methods specific to a particular kind of big data with certain characteristics and applications of methods in classification analysis, which are significantly different to the existing review work. This paper also highlights the current issues of feature selection in big data and suggests the future research directions.

A survey on online feature selection with streaming features

In the era of big data, the dimensionality of data is increasing dramatically in many domains. To deal with high dimensionality, online feature selection becomes critical in big data mining. Recently, online selection of dynamic features has received much attention. In situations where features arrive sequentially over time, we need to perform online feature selection upon feature arrivals. Meanwhile, considering grouped features, it is necessary to deal with features arriving by groups. To handle these challenges, some state-of-the-art methods for online feature selection have been proposed. In this paper, we first give a brief review of traditional feature selection approaches. Then we discuss specific problems of online feature selection with feature streams in detail. A comprehensive review of existing online feature selection methods is presented by comparing with each other. Finally, we discuss several open issues in online feature selection.

A feature selection approach towards progressive vector transmission over the Internet

WebGIS has been applied for visualizing and sharing geospatial information popularly over the Internet. In order to improve the efficiency of the client applications, the web-based progressive vector transmission approach is proposed. Important features should be selected and transferred firstly, and the methods for measuring the importance of features should be further considered in the progressive transmission. However, studies on progressive transmission for large-volume vector data have mostly focused on map generalization in the field of cartography, but rarely discussed on the selection of geographic features quantitatively. This paper applies information theory for measuring the feature importance of vector maps. A measurement model for the amount of information of vector features is defined based upon the amount of information for dealing with feature selection issues. The measurement model involves geometry factor, spatial distribution factor and thematic attribute factor. Moreover, a real-time transport protocol (RTP)-based progressive transmission method is then presented to improve the transmission of vector data. To clearly demonstrate the essential methodology and key techniques, a prototype for web-based progressive vector transmission is presented, and an experiment of progressive selection and transmission for vector features is conducted. The experimental results indicate that our approach clearly improves the performance and end-user experience of delivering and manipulating large vector data over the Internet.

ℓ2,1 norm regularized multi-kernel based joint nonlinear feature selection and over-sampling for imbalanced data classification

High dimensionality and classification of imbalanced data sets are two of the most interesting machine learning challenges. Both issues have been independently studied in the literature. In order to simultaneously explore the both issues of feature selection and oversampling, we efficiently combine two different methodological approaches in an unified kernel framework. Specifically, we proposed a novel ℓ2,1 norm balanced multiple kernel feature selection (ℓ2,1 MKFS), and designed a proximal based optimization algorithm for efficiently learning the model. Moreover, multiple kernel oversampling (MKOS) was developed to generate synthetic instances in the optimal kernel space induced by ℓ2,1 MKFS, so as to compensate for the class imbalanced distribution. Our experimental results on multiple UCI data and two real medical application demonstrate that jointly operating nonlinear feature selection and oversampling with ℓ2,1 norm multi-kernel learning framework (ℓ2,1 MKFSOS) can lead to a promising classification performance.

Feature Mining and Health Assessment for Gearboxes Using Run-Up/Coast-Down Signals.

Vibration signals measured in the run-up/coast-down (R/C) processes usually carry rich information about the health status of machinery. However, a major challenge in R/C signals analysis lies in how to exploit more diagnostic information, and how this information could be properly integrated to achieve a more reliable maintenance decision. Aiming at this problem, a framework of R/C signals analysis is presented for the health assessment of gearbox. In the proposed methodology, we first investigate the data preprocessing and feature selection issues for R/C signals. Based on that, a sparsity-guided feature enhancement scheme is then proposed to extract the weak phase jitter associated with gear defect. In order for an effective feature mining and integration under R/C, a generalized phase demodulation technique is further established to reveal the evolution of modulation feature with operating speed and rotation angle. The experimental results indicate that the proposed methodology could not only detect the presence of gear damage, but also offer a novel insight into the dynamic behavior of gearbox.

Feature selection and approximate reasoning of large-scale set-valued decision tables based on α-dominance-based quantitative rough sets

Set-valued data are a common type of data for characterizing uncertain and missing information. Traditional dominance-based rough sets can not efficiently deal with large-scale set-valued decision tables and usually neglect the disjunctive semantics of sets. In this paper, we propose a general framework of feature selection and approximate reasoning for large-scale set-valued information tables by integrating quantitative rough sets and dominance-based rough sets. Firstly, we define two new partial orders for set-valued data via the conjunctive and disjunctive semantics of a set. Secondly, based on α-disjunctive dominance relation and α-conjunctive dominance relation defined by the inclusion measure, we present α-dominance-based quantitative rough set models for these two types of set-valued decision tables. Furthermore, we study the issue of feature selection in set-valued decision tables by employing α-dominance-based quantitative rough set models and discuss the relationships between the relative reductions and discernibility matrices. We also present approximate reasoning models based on α-dominance-based quantitative rough sets. Finally, the application of the approach is illustrated by some real-world data sets.