Compact Feature Subset Research Articles

Ensemble Feature Selection Using Neighbourhood Rough Set–Based Multicriterion Fusion

Feature selection refers to finding a compact feature subset in which redundant and irrelevant information is reduced. It is generally aimed at simplifying the learning model and improving the learning performance simultaneously. To further enhance the traditional feature selection, ensemble feature selection has been developed vigorously in recent years. In this study, a novel ensemble feature selection algorithm that fuses multiple scores induced by multiple neighbourhood rough set–based feature evaluation criteria is presented. In detail, three different neighbourhood rough set–based feature evaluation criteria are applied, including neighbourhood approximation quality, neighbourhood recognition rate, and neighbourhood discrimination index. These representative measures serve as ensemble bases to induce multiple feature scores. Then, an aggregation method is proposed to fuse these scores for quantifying the feature importance quality. Finally, a searching algorithm based on this neighbourhood score by multicriterion fusion is devised. Experimental results on 16 UCI data sets prove that the proposed method outperforms other feature selection algorithms in classification task.

Structural regularization based discriminative multi-view unsupervised feature selection

Multi-view unsupervised feature selection (MUFS) has recently aroused considerable attention, which can select the compact representative feature subset from original multi-view data. Despite the promising preliminary performance, most previous MUFS methods fail to explore the discriminative ability of multi-view data. In addition, they usually utilize spectral analysis to maintain the geometrical structure, which will inevitably increase the difficulty of parameter selection. To address these issues, we present a novel MUFS method, named structural regularization based discriminative multi-view unsupervised feature selection (SDFS). Specifically, we calculate the similarity matrix of sample space from different views and automatically weight each view-specific graph to learn a consensus similarity graph, in which these two types of graphs can promote each other. Further, we treat the learned latent representation as the cluster indicator, and employ a graph regularization without introducing additional parameters to maintain the geometrical structure of data. Besides, a simple yet efficient iterative updating algorithm with theoretical convergence property is developed. Extensive experiments on several benchmark datasets verify that the designed model is superior to several state-of-the-art MUFS models.

A novel multi-label feature selection method with association rules and rough set

In multi-label learning, each instance is associated with a set of labels. To improve the accuracy and efficiency of multi-label learning tasks, label correlations have been explored. However, existing conventional algorithms obtain label correlation directly from the raw label space, ignoring the significance of the label to the instance. In this study, a novel feature selection method was proposed to select a more relevant and compact feature subset by considering the label distribution and inter-label correlations. First, the concept of label distribution was defined to reflect the significance of each label. Second, a new algorithm for mining association rules was designed to obtain the correlation between labels by improving the existing association rules algorithm. Thus, a new information system was designed by combining the label distribution and correlation between labels. Subsequently, a novel feature selection algorithm was designed in this information system which can effectively remove irrelevant and redundant features in the feature space. Finally, the experimental results demonstrated that the proposed algorithm effectively improves the classification performance and perform better than some state-of-the-art methods.

MSINGB: A Novel Computational Method Based on NGBoost for Identifying Microsatellite Instability Status from Tumor Mutation Annotation Data.

Microsatellite instability (MSI), a vital mutator phenotype caused by DNA mismatch repair deficiency, is frequently observed in several tumors. MSI is recognized as a critical molecular biomarker for diagnosis, prognosis, and therapeutic selection in several cancers. Identifying MSI status for current gold standard methods based on experimental analysis is laborious, time-consuming, and costly. Although several computational methods based on machine learning have been proposed to identify MSI status, we need to further understand which machine learning model would favor identification for MSI and which feature subset is strongly related to MSI. On this basis, more effective machine learning-based methods can be developed to improve the performance of MSI status identification. In this work, we present MSINGB, an NGBoost-based method for identifying MSI status from tumor somatic mutation annotation data. MSINGB first evaluates the prediction performance of 11 popular machine learning algorithms and 9 deep learning models to identify MSI. Among 20 models, NGBoost, a novel natural gradient boosting method, achieves the overall best performance. MSINGB then introduces two feature selection strategies to find the compact feature subset, which is strongly related to MSI, and employs the SHAP approach to interpreting how selected features impact the model prediction. MSINGB achieves a better prediction performance on both the tenfold cross-validation test and independent test compared with state-of-the-art methods.

Multi-Population Enhanced Slime Mould Algorithm and with Application to Postgraduate Employment Stability Prediction

In this study, the authors aimed to study an effective intelligent method for employment stability prediction in order to provide a reasonable reference for postgraduate employment decision and for policy formulation in related departments. First, this paper introduces an enhanced slime mould algorithm (MSMA) with a multi-population strategy. Moreover, this paper proposes a prediction model based on the modified algorithm and the support vector machine (SVM) algorithm called MSMA-SVM. Among them, the multi-population strategy balances the exploitation and exploration ability of the algorithm and improves the solution accuracy of the algorithm. Additionally, the proposed model enhances the ability to optimize the support vector machine for parameter tuning and for identifying compact feature subsets to obtain more appropriate parameters and feature subsets. Then, the proposed modified slime mould algorithm is compared against various other famous algorithms in experiments on the 30 IEEE CEC2017 benchmark functions. The experimental results indicate that the established modified slime mould algorithm has an observably better performance compared to the algorithms on most functions. Meanwhile, a comparison between the optimal support vector machine model and other several machine learning methods on their ability to predict employment stability was conducted, and the results showed that the suggested the optimal support vector machine model has better classification ability and more stable performance. Therefore, it is possible to infer that the optimal support vector machine model is likely to be an effective tool that can be used to predict employment stability.

Elaboration of a Radiomics Strategy for the Prediction of the Re-positive Cases in the Discharged Patients With COVID-19.

Objective: A considerable part of COVID-19 patients were found to be re-positive in the SARS-CoV-2 RT-PCR test after discharge. Early prediction of re-positive COVID-19 cases is of critical importance in determining the isolation period and developing clinical protocols.Materials and Methods: Ninety-one patients discharged from Wanzhou Three Gorges Central Hospital, Chongqing, China, from February 10, 2020 to March 3, 2020 were administered nasopharyngeal swab SARS-CoV-2 tests within 12–14 days, and 50 eligible patients (32 male and 18 female) with completed data were enrolled. Average age was 48 ± 11.5 years. All patients underwent non-enhanced chest CT on admission. A total of 568 radiomics features were extracted from the CT images, and 17 clinical factors were collected based on the medical record. Student's t-test and support vector machine–based recursive feature elimination (SVM-RFE) method were used to determine an optimal subset of features for the discriminative model development.Results: After Student's t-test, 62 radiomics features showed significant inter-group differences (p < 0.05) between the re-positive and negative cases, and none of the clinical features showed significant differences. These significant features were further selected by SVM-RFE algorithm, and a more compact feature subset containing only two radiomics features was finally determined, achieving the best predictive performance with the accuracy and area under the curve of 72.6% and 0.773 for the identification of the re-positive case.Conclusion: The proposed radiomics method has preliminarily shown potential in identifying the re-positive cases among the recovered COVID-19 patients after discharge. More strategies are to be integrated into the current pipeline to improve its precision, and a larger database with multi-clinical enrollment is required to extensively verify its performance.

Robust hierarchical feature selection driven by data and knowledge

Feature selection is facing great challenges brought by the enlarging label space and the inevitable noisy data. Flat feature selection methods fail to obtain a compact feature subset because of the numerous classes. In addition, these data-driven methods are sensitive to the data outliers. Fortunately, many practical tasks usually organize the classes by a hierarchical structure in a coarse-to-fine manner and can be solved by using the divide-and-conquer strategy. In this paper, we propose a hierarchical feature selection method driven by data and knowledge (HFSDK), which is robust to the data outliers and produces compact feature subsets by splitting the original large label space. Firstly, HFSDK decomposes a large-scale classification task into a group of small subclassification tasks with multiple granularities, which is driven by knowledge of the hierarchical class structure. Then, the corresponding datasets are constructed from the bottom to the top using the class labels of data, which is a data-driven process. Finally, robust and discriminative feature subsets are selected recursively for those subtasks by eliminating the data outliers and adding a semantic relation constraint. Experiments on six real-world datasets validate the superior performance of the proposed method.

Dynamic incorporation of prior knowledge from multiple domains in biomarker discovery

BackgroundIn biomarker discovery, applying domain knowledge is an effective approach to eliminating false positive features, prioritizing functionally impactful markers and facilitating the interpretation of predictive signatures. Several computational methods have been developed that formulate the knowledge-based biomarker discovery as a feature selection problem guided by prior information. These methods often require that prior information is encoded as a single score and the algorithms are optimized for biological knowledge of a specific type. However, in practice, domain knowledge from diverse resources can provide complementary information. But no current methods can integrate heterogeneous prior information for biomarker discovery. To address this problem, we developed the Know-GRRF (know-guided regularized random forest) method that enables dynamic incorporation of domain knowledge from multiple disciplines to guide feature selection.ResultsKnow-GRRF embeds domain knowledge in a regularized random forest framework. It combines prior information from multiple domains in a linear model to derive a composite score, which, together with other tuning parameters, controls the regularization of the random forests model. Know-GRRF concurrently optimizes the weight given to each type of domain knowledge and other tuning parameters to minimize the AIC of out-of-bag predictions. The objective is to select a compact feature subset that has a high discriminative power and strong functional relevance to the biological phenotype.Via rigorous simulations, we show that Know-GRRF guided by multiple-domain prior information outperforms feature selection methods guided by single-domain prior information or no prior information. We then applied Known-GRRF to a real-world study to identify prognostic biomarkers of prostate cancers. We evaluated the combination of cancer-related gene annotations, evolutionary conservation and pre-computed statistical scores as the prior knowledge to assemble a panel of biomarkers. We discovered a compact set of biomarkers with significant improvements on prediction accuracies.ConclusionsKnow-GRRF is a powerful novel method to incorporate knowledge from multiple domains for feature selection. It has a broad range of applications in biomarker discoveries. We implemented this method and released a KnowGRRF package in the R/CRAN archive.

Predicting Entrepreneurial Intention of Students: An Extreme Learning Machine With Gaussian Barebone Harris Hawks Optimizer

This study aims to propose an effective intelligent model for predicting entrepreneurial intention, which can provide a reasonable reference for the formulation of talent training programs and the guidance of entrepreneurial intention of students. The prediction model is mainly based on the kernel extreme learning machine (KELM) optimized by the improved Harris hawk’s optimizer (HHO). In order to obtain better parameters and feature subsets, the Gaussian barebone (GB) strategy is introduced to improve the HHO algorithm, so as to strengthen the optimization ability for tuning parameters of KELM and identifying the compact feature subsets. Then, an optimal KELM model (GBHHO-KELM) is established according to the obtained optimal parameters and feature subsets to predict the entrepreneurial intention of students. In the experiment, GBHHO is compared with the other nine well-known methods in 30 CEC 2014 benchmark problems. The experimental findings suggest that the proposed GBHHO method is significantly superior to the existing methods in most problems. At the same time, GBHHO-KELM is compared with other machine learning methods in the prediction of entrepreneurial intention. The experimental results indicate that the proposed GBHHO-KELM can achieve better classification performance and higher stability in accordance with the four metrics. Therefore, we can conclude that the GBHHO-KELM model is expected to be an effective tool for the prediction of entrepreneurial intention.

A novel approach for learning label correlation with application to feature selection of multi-label data

Each example of multi-label data is represented in an object with its feature vector (i.e. an instance) while being related to multiple labels. Learning label correlation can effectively reduce the labels needed to be predicted and optimize the classification performance. For this reason, label correlation plays a crucial role in multi-label learning and has been explored by many existing algorithms. Generally, every label has its own indispensable features, and it is reasonable to assume that a higher repetitiveness of indispensable features represents higher correlations among labels. Inspired by this fact, the essential elements for each label, which are composed of indispensable features, are constructed in this paper. A method is proposed for learning label correlation and applying it to feature selection of multi-label data based on the overlap of different families of essential elements related to label. Firstly, the essential elements of each label are defined and characterized to reflect the internal connection between features and label. In addition, a process for calculating the essential elements of a single label is provided. Secondly, by considering the overlap of the essential element collections that are determined by the different labels, relevancy of label and corresponding relevance judgement matrix with the label set are described. Therefore, several labels with strong relationships are assigned to a relevant group of labels. Meanwhile, local and global label correlations can be computed. Thus a novel multi-label learning algorithm, called CLSF, is presented to select a compact subset of indispensable features for each relevant group of labels by applying local label correlation to feature selection of multi-label data. Finally, comprehensive experiments on eleven benchmark data sets clearly illustrate the effectiveness and efficiency of CLSF against five other multi-label learning algorithms.

Feature selection for blind image steganalysis using neighborhood rough sets

Feature extraction for blind image steganalysis produces much features or high dimensional data, which bring about time consuming and even a low detection percentage. As being one of the most important phases of preprocessing, feature selection can reduce these extracted features, and improve the p erformance of steganalysis. Firstly, we introduce the Neighborhood Rough Sets (NRS) to the field of blind image steganalysis. Then, some concepts of feature significance and feature reduct are presented based on NRS. Furthermore, we propose a Feature Selection approach by NRS for blind image steganalysis (FSNRS). The FSNRS has the ability to delete redundant features, meanwhile maintaining the classification accuracy of a steganalysis system. The FSNRS is a filter feature selection technique for blind image steganalysis, which filtrates extracted features depending on a positive region preserving in NRS. The compact feature subset with a shortest feature dimension for blind image steganalysis is selected. Moreover, some experiments for blind steganalysis using SVM and KNN classifiers on selected feature subset are carried out. The experimental results show that our proposed approach can obtain compact features for blind image steganalysis and the performances of classifiers on those selected features are improved. Since the FSNRS is used with an adjustable neighborhood parameter, as a result, the classification performance of selected features is better than that of original whole features in most cases.

An instance voting approach to feature selection

In this work, we address the problem of supervised feature selection (FS) for high-dimensional datasets with a small number of instances. Here, we propose a novel heuristic FS approach, Conditional Priority Coverage Maximization (CPCM) which seeks to leverage the local information provided by the small set of instances. We define the vote assigned by an instance to a feature as the local relevance of the latter. Also, we show that the proposed voting scheme is asymptotically related to the Bayes’ decision rule for minimum risk classification. Next, we exploit the instance votes for feature selection by posing it as a set-covering problem — we seek to select a subset of features such that they can together cover the instances. This approach avoids the selection of redundant features, while selecting relevant ones. In addition, we formulate the stopping criterion to select a compact subset of features. Through experiments on synthetic and real datasets, we demonstrated that CPCM outperforms other graph based FS techniques and state-of-the-art FS approaches employing mutual information (MI). Further, we evaluated the stability of CPCM to minor variations in the training data and found it to be reasonably robust.

Computational identification of deleterious synonymous variants in human genomes using a feature-based approach

BackgroundAlthough synonymous single nucleotide variants (sSNVs) do not alter the protein sequences, they have been shown to play an important role in human disease. Distinguishing pathogenic sSNVs from neutral ones is challenging because pathogenic sSNVs tend to have low prevalence. Although many methods have been developed for predicting the functional impact of single nucleotide variants, only a few have been specifically designed for identifying pathogenic sSNVs.ResultsIn this work, we describe a computational model, IDSV (Identification of Deleterious Synonymous Variants), which uses random forest (RF) to detect deleterious sSNVs in human genomes. We systematically investigate a total of 74 multifaceted features across seven categories: splicing, conservation, codon usage, sequence, pre-mRNA folding energy, translation efficiency, and function regions annotation features. Then, to remove redundant and irrelevant features and improve the prediction performance, feature selection is employed using the sequential backward selection method. Based on the optimized 10 features, a RF classifier is developed to identify deleterious sSNVs. The results on benchmark datasets show that IDSV outperforms other state-of-the-art methods in identifying sSNVs that are pathogenic.ConclusionsWe have developed an efficient feature-based prediction approach (IDSV) for deleterious sSNVs by using a wide variety of features. Among all the features, a compact and useful feature subset that has an important implication for identifying deleterious sSNVs is identified. Our results indicate that besides splicing and conservation features, a new translation efficiency feature is also an informative feature for identifying deleterious sSNVs. While the function regions annotation and sequence features are weakly informative, they may have the ability to discriminate deleterious sSNVs from benign ones when combined with other features. The data and source code are available on website http://bioinfo.ahu.edu.cn:8080/IDSV.

Incremental Perspective for Feature Selection Based on Fuzzy Rough Sets

Feature selection based on fuzzy rough sets is an effective approach to select a compact feature subset that optimally predicts a given decision label. Despite being studied extensively, most existing methods of fuzzy rough set based feature selection are restricted to computing the whole dataset in batch, which is often costly or even intractable for large datasets. To improve the time efficiency, we investigate the incremental perspective for fuzzy rough set based feature selection assuming data can be presented in sample subsets one after another. The key challenge for the incremental perspective is how to add and delete features with the subsequent arrival of sample subsets. We tackle this challenge with strategies of adding and deleting features based on the relative discernibility relations that are updated as subsets arrive sequentially. Two incremental algorithms for fuzzy rough set based feature selection are designed based on the strategies. One updates the selected features as each sample subset arrives, and outputs the final feature subset where no sample subset is left. The other updates the relative discernibility relations but only performs feature selection where there is no further subset arriving. Experimental comparisons suggest our incremental algorithms expedite fuzzy rough set based feature selection without compromising performance.

Compact feature subset-based multi-label music categorization for mobile devices

Music categorization based on acoustic features extracted from music clips and user-defined tags forms the basis of recent music recommendation applications, because relevant tags can be automatically assigned based on the feature values and their relation to tags. In practice, especially for handheld lightweight mobile devices, there is a certain limitation on the computational capacity, owing to consumers’ usage behavior or battery consumption. This also limits the maximum number of acoustic features to be extracted, and results in the necessity of identifying a compact feature subset that is used for the music categorization process. In this study, we propose an approach to compact feature subset-based multi-label music categorization for mobile music recommendation services. Experimental results using various multi-labeled music datasets reveal that the proposed approach yields better performance when compared to conventional approach.

Protein-protein interface hot spots prediction based on a hybrid feature selection strategy

BackgroundHot spots are interface residues that contribute most binding affinity to protein-protein interaction. A compact and relevant feature subset is important for building machine learning methods to predict hot spots on protein-protein interfaces. Although different methods have been used to detect the relevant feature subset from a variety of features related to interface residues, it is still a challenge to detect the optimal feature subset for building the final model.ResultsIn this study, three different feature selection methods were compared to propose a new hybrid feature selection strategy. This new strategy was proved to effectively reduce the feature space when we were building the prediction models for identifying hotspot residues. It was tested on eighty-two features, both conventional and newly proposed. According to the strategy, combining the feature subsets selected by decision tree and mRMR (maximum Relevance Minimum Redundancy) individually, we were able to build a model with 6 features by using a PSFS (Pseudo Sequential Forward Selection) process. Compared with other state-of-art methods for the independent test set, our model had shown better or comparable predictive performances (with F-measure 0.622 and recall 0.821). Analysis of the 6 features confirmed that our newly proposed feature CNSV_REL1 was important for our model. The analysis also showed that the complementarity between features should be considered as an important aspect when conducting the feature selection.ConclusionIn this study, most important of all, a new strategy for feature selection was proposed and proved to be effective in selecting the optimal feature subset for building prediction models, which can be used to predict hot spot residues on protein-protein interfaces. Moreover, two aspects, the generalization of the single feature and the complementarity between features, were proved to be of great importance and should be considered in feature selection methods. Finally, our newly proposed feature CNSV_REL1 had been proved an alternative and effective feature in predicting hot spots by our study. Our model is available for users through a webserver: http://zhulab.ahu.edu.cn/iPPHOT/.

A New Filter Feature Selection Based on Criteria Fusion for Gene Microarray Data

In machine learning and data mining, feature selection aims to seek a compact and discriminant feature subset from the original feature space. It is usually used as a preprocessing step to improve the prediction performance, understandability, scalability, and generalization capability of classifiers. A typical gene microarray data set has the characteristics of high dimensionality, limited samples, and most irrelevant features, and these characteristics make it difficult to discover a compact set of features that really contribute to the response of the model. In this paper, a score-based criteria fusion feature selection method (SCF) is proposed for cancer prediction, and this method aims at improving the prediction performance of the classification model. The SCF method is evaluated on five open gene microarray data sets and three low-dimensional data sets, and it shows superior performance over many well-known feature selection methods when employing two classifiers SVM and KNN to measure the quality of selected features. Experiments verify that SCF is able to find more discriminative features than the competing methods and can be used as a preprocessing algorithm to combine with other methods effectively.

Effective Evolutionary Multilabel Feature Selection under a Budget Constraint

Multilabel feature selection involves the selection of relevant features from multilabeled datasets, resulting in improved multilabel learning accuracy. Evolutionary search-based multilabel feature selection methods have proved useful for identifying a compact feature subset by successfully improving the accuracy of multilabel classification. However, conventional methods frequently violate budget constraints or result in inefficient searches due to ineffective exploration of important features. In this paper, we present an effective evolutionary search-based feature selection method for multilabel classification with a budget constraint. The proposed method employs a novel exploration operation to enhance the search capabilities of a traditional genetic search, resulting in improved multilabel classification. Empirical studies using 20 real-world datasets demonstrate that the proposed method outperforms conventional multilabel feature selection methods.

Solving customer insurance coverage sales plan problem using a multi-stage data mining approach

PurposeCustomer insurance coverage sales plan problem, in which the loyal customers are recognized and offered some special plans, is an essential problem facing insurance companies. On the other hand, the loyal customers who have enough potential to renew their insurance contracts at the end of the contract term should be persuaded to repurchase or renew their contracts. The aim of this paper is to propose a three-stage data-mining approach to recognize high-potential loyal insurance customers and to predict/plan special insurance coverage sales.Design/methodology/approachThe first stage addresses data cleansing. In the second stage, several filter and wrapper methods are implemented to select proper features. In the third stage, K-nearest neighbor algorithm is used to cluster the customers. The approach aims to select a compact feature subset with the maximal prediction capability. The proposed approach can detect the customers who are more likely to buy a specific insurance coverage at the end of a contract term.FindingsThe proposed approach has been applied in a real case study of insurance company in Iran. On the basis of the findings, the proposed approach is capable of recognizing the customer clusters and planning a suitable insurance coverage sales plans for loyal customers with proper accuracy level. Therefore, the proposed approach can be useful for the insurance company which helps them to identify their potential clients. Consequently, insurance managers can consider appropriate marketing tactics and appropriate resource allocation of the insurance company to their high-potential loyal customers and prevent switching them to competitors.Originality/valueDespite the importance of recognizing high-potential loyal insurance customers, little study has been done in this area. In this paper, data-mining techniques were developed for the prediction of special insurance coverage sales on the basis of customers’ characteristics. The method allows the insurance company to prioritize their customers and focus their attention on high-potential loyal customers. Using the outputs of the proposed approach, the insurance companies can offer the most productive/economic insurance coverage contracts to their customers. The approach proposed by this study be customized and may be used in other service companies.

Granular multi-label feature selection based on mutual information

Like the traditional machine learning, the multi-label learning is faced with the curse of dimensionality. Some feature selection algorithms have been proposed for multi-label learning, which either convert the multi-label feature selection problem into numerous single-label feature selection problems, or directly select features from the multi-label data set. However, the former omit the label dependency, or produce too many new labels leading to learning with significant difficulties; the latter, taking the global label dependency into consideration, usually select a few redundant or irrelevant features, because actually not all labels depend on each other, which may confuse the algorithm and degrade its classification performance. To select a more relevant and compact feature subset as well as explore the label dependency, a granular feature selection method for multi-label learning is proposed with a maximal correlation minimal redundancy criterion based on mutual information. The maximal correlation minimal redundancy criterion makes sure that the selected feature subset contains the most class-discriminative information, while in the meantime exhibits the least intra-redundancy. Granulation can help explore the label dependency. We study the relation of the label granularity and the performance on four data sets, and compare the proposed method with other three multi-label feature selection methods. The experimental results demonstrate that the proposed method can select compact and specific feature subsets, improve the classification performance and performs better than other three methods on the widely-used multi-label learning evaluation criteria.