Good Feature Selection Research Articles

Feature selection for hybrid information systems based on fuzzy $$\beta $$ covering and fuzzy evidence theory

AbstractFeature selection plays a crucial role in machine learning, as it eliminates data noise and redundancy, thereby significantly reducing computational complexity and enhancing the overall performance of the model. The challenges of feature selection for hybrid information systems stem from the difficulty in quantifying the disparities among nominal attribute values. Furthermore, a significant majority of the current methodologies exhibit sensitivity to noise. This paper introduces techniques that address the aforementioned issues from the perspective of fuzzy evidence theory. First of all, a new distance incorporating decision attributes is defined, and then a relation between fuzzy evidence theory and fuzzy $$\beta $$ β covering with an anti-noise mechanism is established. In this framework, two robust feature selection algorithms for hybrid data are proposed based on fuzzy belief and fuzzy plausibility. Experiments on 10 data sets of various types show that compared with the other 6 state-of-the-art algorithms, the proposed algorithms improve the anti-noise ability by at least 6% with higher average classification accuracy. Therefore, it can be concluded that the proposed algorithms have excellent anti-noise ability while maintaining good feature selection ability.

Feature selection by Universum embedding

Feature selection in classification is an important task in machine learning. Inspired by the success of Universum support vector machine proposed by Weston et al. on improving the classification ability of classical support vector machine, this paper considers a special type of Universum and further lets it play its role in both useful feature identification and separating hyperplane construction, aiming to improve both the feature selection ability and classification performance of Universum support vector machine. By introducing this special Universum, a redundant feature can be identified by observing whether some Universum sample is useful. In fact, we prove that by observing the dual solution of the optimization problem, useful features can be selected from a set satisfying some properties. Due to the introduction of these extra Universum samples, it needs to cope with a large-scale optimization problem. To improve the training efficiency, we modify the sequential minimal optimization algorithm and further combine it with the coordinate descent technique to solve the proposed model. Experimental results on artificial datasets, benchmark datasets, and text classification datasets demonstrate that the proposed method improves the classification performance of support vector machine and Universum support vector machine, and also has good feature selection ability.

Feature selection for hybrid information systems based on fuzzy β covering and fuzzy evidence theory

Feature selection can remove data noise and redundancy and reduce computational complexity, which is vital for machine learning. Because the difference between nominal attribute values is difficult to measure, feature selection for hybrid information systems faces challenges. In addition, many existing feature selection methods are susceptible to noise, such as Fisher, LASSO, random forest, mutual information, rough-set-based methods, etc. This paper proposes some techniques that consider the above problems from the perspective of fuzzy evidence theory. Firstly, a new distance incorporating decision attributes is defined, and then a relation between fuzzy evidence theory and fuzzy β covering with an anti-noise mechanism is established. Based on fuzzy belief and fuzzy plausibility, two robust feature selection algorithms for hybrid data are proposed in this framework. Experiments on 10 datasets of various types have shown that the proposed algorithms achieved the highest classification accuracy 11 times out of 20 experiments, significantly surpassing the performance of the other 6 state-of-the-art algorithms, achieved dimension reduction of 84.13% on seven UCI datasets and 99.90% on three large-scale gene datasets, and have a noise tolerance that is at least 6% higher than the other 6 state-of-the-art algorithms. Therefore, it can be concluded that the proposed algorithms have excellent anti-noise ability while maintaining good feature selection ability.

Attribute reduction algorithms with an anti-noise mechanism for hybrid data based on fuzzy evidence theory

Attribute reduction can remove data noise and redundancy, thus reducing computational complexity, which is very important for machine learning. Because the difference between nominal attribute values is difficult to measure, attribute reduction for hybrid data faces challenges. In addition, most of the existing methods are sensitive to noise due to the lack of an anti-noise mechanism. Decision attribute contains the most important information of data. This paper proposes some techniques that consider the above problems from the perspective of fuzzy evidence theory. First of all, a new distance incorporating decision attributes is defined, and then a new fuzzy relation with an anti-noise mechanism is defined. Furthermore, fuzzy belief and fuzzy plausibility are defined based on the defined new distance and new fuzzy relation. In this framework, two anti-noise attribute reduction algorithms for hybrid data are proposed. Experiments on 12 data sets of various types show that compared with the other 8 state-of-the-art algorithms, the proposed algorithms improve the classification accuracy by at least 2% and the anti-noise ability by at least 11%. Therefore, it can be concluded that the proposed algorithms have excellent anti-noise ability while maintaining good feature selection ability.

A review of feature selection methods based on meta-heuristic algorithms

ABSTRACT Feature selection is a real-world problem that finds a minimal feature subset from an original feature set. A good feature selection method, in addition to selecting the most relevant features with less redundancy, can also reduce computational costs and increase classification performance. One of the feature selection approaches is using meta-heuristic algorithms. This work provides a summary of some meta-heuristic feature selection methods proposed from 2018 to 2022 that were designed and implemented on a wide range of different data for solving feature selection problem. Evaluation criteria, fitness functions and classifiers used and the time complexity of each method are also depicted. The results of the study showed that some meta-heuristic algorithms alone cannot perfectly solve the feature selection problem on all types of datasets with an acceptable speed. In other words, depending on dataset, a special meta-heuristic algorithm should be used. The results of this study and the identified research gaps can be used by researchers in this field.

An optimized features selection approach based on Manta Ray Foraging Optimization (MRFO) method for parasite malaria classification.

Malaria is a serious and lethal disease that has been reported by the World Health Organization (WHO), with an estimated 219 million new cases and 435,000 deaths globally. The most frequent malaria detection method relies mainly on the specialists who examine the samples under a microscope. Therefore, a computerized malaria diagnosis system is required. In this article, malaria cell segmentation and classification methods are proposed. The malaria cells are segmented using a color-based k-mean clustering approach on the selected number of clusters. After segmentation, deep features are extracted using pre-trained models such as efficient-net-b0 and shuffle-net, and the best features are selected using the Manta-Ray Foraging Optimization (MRFO) method. Two experiments are performed for classification using 10-fold cross-validation, the first experiment is based on the best features selected from the pre-trained models individually, while the second experiment is performed based on the selection of best features from the fusion of extracted features using both pre-trained models. The proposed method provided an accuracy of 99.2% for classification using the linear kernel of the SVM classifier. An empirical study demonstrates that the fused features vector results are better as compared to the individual best-selected features vector and the existing latest methods published so far.

SPDPOA: Student Psychology Dragonfly Political Optimizer Algorithm-Based Soil Moisture and Heat-Level Prediction for Plant Health Monitoring in Internet of Things

Abstract This article devised an effective Student Psychology-based Dragonfly Political Optimizer (SPDPOA) for predicting heat level and soil moisture to monitor plant health in the Internet of Things (IoT). The developed SPDPOA is modeled by integrating the Student Psychology-based Optimization (SPBO) algorithm, Dragonfly Algorithm (DA) and Political optimizer (PO), respectively. The prediction process is done in the base station (BS), which gathers the IoT nodes’ information through optimal Cluster Head (CH) using Deep Recurrent Neural Network (Deep RNN). Moreover, the CH selection and routing process are established using a developed SPDPOA scheme. The data transformation and feature selection processes are done based on Box-Cox transformation and wrapper model, correspondingly, which helps in the selection of best features. Moreover, the developed SPDPOA scheme attained better performance in Packet Delivery Ratio (PDR), energy and testing accuracy of 0.7232, 0.6342 J and 0.9372, respectively.

Unsupervised feature selection with robust data reconstruction (UFS-RDR) and outlier detection

In unsupervised learning, the traditional feature selection methods are not always efficient and their feature selection performance can be severely affected in the presence of outliers and noise. To address this issue, we propose a novel robust unsupervised feature selection method, called Unsupervised Feature Selection with Robust Data Reconstruction (UFS-RDR), that minimizes the graph regularized weighted data reconstruction error function. For the detection of outliers, the well-known Mahalanobis distance is used and further determine the Huber-type weight function using these Mahalanobis distances. This weight function downweights the clustering observations that have large distance. Our experimental results on both synthetic and real-world datasets indicate that the proposed UFS-RDR approach has good feature selection performance and also outperforms the competitive non-robust unsupervised feature selection methods in the presence of contamination in the unlabeled data.

A Fast Hybrid Classification Algorithm with Feature Reduction for Medical Images.

In this paper, we are introducing a fast hybrid fuzzy classification algorithm with feature reduction for medical images. We incorporated the quantum-based grasshopper computing algorithm (QGH) with feature extraction using fuzzy clustering technique (C-means). QGH integrates quantum computing into machine learning and intelligence applications. The objective of our technique is to the integrate QGH method, specifically into cervical cancer detection that is based on image processing. Many features such as color, geometry, and texture found in the cells imaged in Pap smear lab test are very crucial in cancer diagnosis. Our proposed technique is based on the extraction of the best features using a more than 2600 public Pap smear images and further applies feature reduction technique to reduce the feature space. Performance evaluation of our approach evaluates the influence of the extracted feature on the classification precision by performing two experimental setups. First setup is using all the extracted features which leads to classification without feature bias. The second setup is a fusion technique which utilized QGH with the fuzzy C-means algorithm to choose the best features. In the setups, we allocate the assessment to accuracy based on the selection of best features and of different categories of the cancer. In the last setup, we utilized a fusion technique engaged with statistical techniques to launch a qualitative agreement with the feature selection in several experimental setups.

FSCAM: CAM-Based Feature Selection for Clustering scRNA-seq.

Cell type determination based on transcriptome profiles is a key application of single-cell RNA sequencing (scRNA-seq). It is usually achieved through unsupervised clustering. Good feature selection is capable of improving the clustering accuracy and is a crucial component of single-cell clustering pipelines. However, most current single-cell feature selection methods are univariable filter methods ignoring gene dependency. Even the multivariable filter methods developed in recent years only consider "one-to-many" relationship between genes. In this paper, a novel single-cell feature selection method based on convex analysis of mixtures (FSCAM) is proposed, which takes into account "many-to-many" relationship. Compared to the previous "one-to-many" methods, FSCAM selects genes with a combination of relevancy, redundancy and completeness. Pertinent benchmarking is conducted on the real datasets to validate the superiority of FSCAM. Through plugging into the framework of partition around medoids (PAM) clustering, a single-cell clustering algorithm based on FSCAM method (SCC_FSCAM) is further developed. Comparing SCC_FSCAM with existing advanced clustering algorithms, the results show that our algorithm has advantages in both internal criteria (clustering number) and external criteria (adjusted Rand index) and has a good stability.

Human Gait Recognition: A Deep Learning and Best Feature Selection Framework

Background—Human Gait Recognition (HGR) is an approach based on biometric and is being widely used for surveillance. HGR is adopted by researchers for the past several decades. Several factors are there that affect the system performance such as the walking variation due to clothes, a person carrying some luggage, variations in the view angle. Proposed—In this work, a new method is introduced to overcome different problems of HGR. A hybrid method is proposed or efficient HGR using deep learning and selection of best features. Four major steps are involved in this work-preprocessing of the video frames, manipulation of the pre-trained CNN model VGG-16 for the computation of the features, removing redundant features extracted from the CNN model, and classification. In the reduction of irrelevant features Principal Score and Kurtosis based approach is proposed named PSbK. After that, the features of PSbK are fused in one materix. Finally, this fused vector is fed to the One against All Multi Support Vector Machine (OAMSVM) classifier for the final results. Results—The system is evaluated by utilizing the CASIA B database and six angles 00°, 18°, 36°, 54°, 72°, and 90° are used and attained the accuracy of 95.80%, 96.0%, 95.90%, 96.20%, 95.60%, and 95.50%, respectively. Conclusion—The comparison with recent methods show the proposed method work better.

Development of Data Mining Models Based on Features Ranks Voting (FRV)

Data size plays a significant role in the design and the performance of data mining models. A good feature selection algorithm reduces the problems of big data size and noise due to data redundancy. Features selection algorithms aim at selecting the best features and eliminating unnecessary ones, which in turn simplifies the structure of the data mining model as well as increases its performance. This paper introduces a robust features selection algorithm, named Features Ranking Voting Algorithm FRV. It merges the benefits of the different features selection algorithms to specify the features ranks in the dataset correctly and robustly; based on the feature ranks and voting algorithm. The FRV comprises of three different proposed techniques to select the minimum best feature set, the forward voting technique to select the best high ranks features, the backward voting technique, which drops the low ranks features (low importance feature), and the third technique merges the outputs from the forward and backward techniques to maximize the robustness of the selected features set. Different data mining models were built using obtained selected features sets from applying the proposed FVR on different datasets; to evaluate the success behavior of the proposed FRV. The high performance of these data mining models reflects the success of the proposed FRV algorithm. The FRV performance is compared with other features selection algorithms. It successes to develop data mining models for the Hungarian CAD dataset with Acc. of 96.8%, and with Acc. of 96% for the Z-Alizadeh Sani CAD dataset compared with 83.94% and 92.56% respectively in [48].

Weighted Cox regression for the prediction of heterogeneous patient subgroups

BackgroundAn important task in clinical medicine is the construction of risk prediction models for specific subgroups of patients based on high-dimensional molecular measurements such as gene expression data. Major objectives in modeling high-dimensional data are good prediction performance and feature selection to find a subset of predictors that are truly associated with a clinical outcome such as a time-to-event endpoint. In clinical practice, this task is challenging since patient cohorts are typically small and can be heterogeneous with regard to their relationship between predictors and outcome. When data of several subgroups of patients with the same or similar disease are available, it is tempting to combine them to increase sample size, such as in multicenter studies. However, heterogeneity between subgroups can lead to biased results and subgroup-specific effects may remain undetected.MethodsFor this situation, we propose a penalized Cox regression model with a weighted version of the Cox partial likelihood that includes patients of all subgroups but assigns them individual weights based on their subgroup affiliation. The weights are estimated from the data such that patients who are likely to belong to the subgroup of interest obtain higher weights in the subgroup-specific model.ResultsOur proposed approach is evaluated through simulations and application to real lung cancer cohorts, and compared to existing approaches. Simulation results demonstrate that our proposed model is superior to standard approaches in terms of prediction performance and variable selection accuracy when the sample size is small.ConclusionsThe results suggest that sharing information between subgroups by incorporating appropriate weights into the likelihood can increase power to identify the prognostic covariates and improve risk prediction.

Analisis Sentimen E-Wallet di Twitter Menggunakan Support Vector Machine dan Recursive Feature Elimination

Grouping of positive or negative sentiments in text reviews is increasingly being done automatically for identification. The selection of features in the classification is a problem that is often not solved. Most of the feature selection related to sentiment classification techniques is insurmountable in terms of evaluating significant features that reduce classification performance. Good feature selection technique can improve sentiment classification performance in machine learning approach. First, two sets of customer review data are labeled with sentiment and then retrieved, processed for evaluation. Next, the supports vector machine (svm-rfe) method is created and tested on the dataset. Svm-rfe will be run to measure the importance of the feature by rating the feature iteratively. For sentiment classification, only the top features of the ranking feature sequence will be used. Finally, performance is measured using accuracy, precision, recall, and f1-score. The experimental results show promising performance with an accuracy rate of 81%. This level of reduction is significant in making optimal use of computing resources while maintaining the efficiency of classification performance

A New Feature Selection Method Based on a Self-Variant Genetic Algorithm Applied to Android Malware Detection

In solving classification problems in the field of machine learning and pattern recognition, the pre-processing of data is particularly important. The processing of high-dimensional feature datasets increases the time and space complexity of computer processing and reduces the accuracy of classification models. Hence, the proposal of a good feature selection method is essential. This paper presents a new algorithm for solving feature selection, retaining the selection and mutation operators from traditional genetic algorithms. On the one hand, the global search capability of the algorithm is ensured by changing the population size, on the other hand, finding the optimal mutation probability for solving the feature selection problem based on different population sizes. During the iteration of the algorithm, the population size does not change, no matter how many transformations are made, and is the same as the initialized population size; this spatial invariance is physically defined as symmetry. The proposed method is compared with other algorithms and validated on different datasets. The experimental results show good performance of the algorithm, in addition to which we apply the algorithm to a practical Android software classification problem and the results also show the superiority of the algorithm.

Improved 18-FDG PET/CT diagnosis of multiple myeloma diffuse disease by radiomics analysis.

In multiple myeloma, the diagnosis of diffuse bone marrow infiltration on 18-FDG PET/CT can be challenging. We aimed to develop a PET/CT radiomics-based model that could improve the diagnosis of multiple myeloma diffuse disease on 18-FDG PET/CT. We prospectively performed PET/CT and whole-body diffusion-weighted MRI in 30 newly diagnosed multiple myeloma. MRI was the reference standard for diffuse disease assessment. Twenty patients were randomly assigned to a training set and 10 to an independent test set. Visual analysis of PET/CT was performed by two nuclear medicine physicians. Spine volumes were automatically segmented, and a total of 174 Imaging Biomarker Standardisation Initiative-compliant radiomics features were extracted from PET and CT. Selection of best features was performed with random forest features importance and correlation analysis. Machine-learning algorithms were trained on the selected features with cross-validation and evaluated on the independent test set. Out of the 30 patients, 18 had established diffuse disease on MRI. The sensitivity, specificity and accuracy of visual analysis were 67, 75 and 70%, respectively, with a moderate kappa coefficient of agreement of 0.6. Five radiomics features were selected. On the training set, random forest classifier reached a sensitivity, specificity and accuracy of 93, 86 and 91%, respectively, with an area under the curve of 0.90 (95% confidence interval, 0.89-0.91). On the independent test set, the model achieved an accuracy of 80%. Radiomics analysis of 18-FDG PET/CT images with machine-learning overcame the limitations of visual analysis, providing a highly accurate and more reliable diagnosis of diffuse bone marrow infiltration in multiple myeloma patients.

Bio-inspired optimization of weighted-feature machine learning for strength property prediction of fiber-reinforced soil

The fiber-reinforcement of soil is an effective and reliable ground improvement technique for increasing the strength and stability of soil for various purposes (including retaining structures, embankments, foundations, slopes and pavements). Numerous scholars have developed methods to identify factors that influence the shear strength and to predict the peak friction angle of fiber-reinforced soil (FRS). The accuracy of theoretical and empirical models for predicting the shear strength (peak friction angle) of FRS is questionable because of the difficulty of using these simplified models to describe the complex mechanism of soil-fiber interaction. Solutions to this problem require ever-increasing predictive accuracy, and ML-based methods have been confirmed to provide potential solutions to real-world engineering problems. Therefore, this study develops weighted-feature least squares support vector regression (WFLSSVR) that is optimized by a novel metaheuristic algorithm, jellyfish search (JS) algorithm, to predict the peak friction angle of FRS. Analytical results demonstrate that JS-WFLSSVR outperforms baseline, ensemble, and hybrid machine learning models as well as empirical methods in literature. Notably, analysis of the weight values that were obtained by JS-WFLSSVR enables the identification of new feature combinations that provide much higher accuracy than current models. Therefore, the JS-WFLSSVR model not only significantly provides better predictive accuracy than methods in the literature; it is also a good feature selection method, and can help geotechnical engineers in estimating the shear strength of FRS. Geotechnical engineers can use the proposed model to predict the shear strength and control the quality of FRS structures.

A Financial Distress Prediction Model Based on Sparse Algorithm and Support Vector Machine

Classification learning is a very important issue in machine learning, which has been widely used in the field of financial distress warning. Some researches show that the prediction model framework based on sparse algorithm has better performance than the traditional model. In this paper, we explore the financial distress prediction based on grouping sparsity. Feature selection of sparse algorithm plays an important role in classification learning, because many redundant and irrelevant features will degrade performance. A good feature selection algorithm would reduce computational complexity and improve classification accuracy. In this study, we propose an algorithm for feature selection classification prediction based on feature attributes and data source grouping. The existing financial distress prediction model usually only uses the data from financial statement and ignores the timeliness of company sample in practice. Therefore, we propose a corporate financial distress prediction model that is better in line with the practice and combines the grouping sparse principal component analysis of financial data, corporate governance characteristics, and market transaction data with support vector machine. Experimental results show that this method can improve the prediction efficiency of financial distress with fewer characteristic variables.

Feature Selection to Optimize Credit Banking Risk Evaluation Decisions for the Example of Home Equity Loans

Measuring credit risk is essential for financial institutions because there is a high risk level associated with incorrect credit decisions. The Basel II agreement recommended the use of advanced credit scoring methods in order to improve the efficiency of capital allocation. The latest Basel agreement (Basel III) states that the requirements for reserves based on risk have increased. Financial institutions currently have exhaustive datasets regarding their operations; this is a problem that can be addressed by applying a good feature selection method combined with big data techniques for data management. A comparative study of selection techniques is conducted in this work to find the selector that reduces the mean square error and requires the least execution time.

A RELATIVE INVESTIGATORY ANALYSIS ON ANT COLONY ALGORITHM AND GENETIC ALGORITHM FOR FEATURE SELECTION

Feature subset selection is used as a common technique in data pre-processing for pattern recognition, machine learning and data mining has attracted much attention in recent years. A good feature selection method can reduce the cost of feature measurement and increase classifier efficiency and classification accuracy. One approach in the feature selection area is employing population based optimization algorithms such as Genetic Algorithm (GA)-based method and Ant Colony Optimization (ACO) based method. In this paper ant colony optimization algorithm (ACO) is compared to Genetic algorithm (GA) using feature selection. Finally a comparative study is done to know the pros and cons of the work done.