Constrained Feature Selection Research Articles

ScGIST: gene panel design for spatial transcriptomics with prioritized gene sets

A critical challenge of single-cell spatial transcriptomics (sc-ST) technologies is their panel size. Being based on fluorescence in situ hybridization, they are typically limited to panels of about a thousand genes. This constrains researchers to build panels from only the marker genes of different cell types and forgo other genes of interest, e.g., genes encoding ligand-receptor complexes or those in specific pathways. We propose scGIST, a constrained feature selection tool that designs sc-ST panels prioritizing user-specified genes without compromising cell type detection accuracy. We demonstrate scGIST’s efficacy in diverse use cases, highlighting it as a valuable addition to sc-ST’s algorithmic toolbox.

An Empirical Evaluation of Constrained Feature Selection

While feature selection helps to get smaller and more understandable prediction models, most existing feature-selection techniques do not consider domain knowledge. One way to use domain knowledge is via constraints on sets of selected features. However, the impact of constraints, e.g., on the predictive quality of selected features, is currently unclear. This article is an empirical study that evaluates the impact of propositional and arithmetic constraints on filter feature selection. First, we systematically generate constraints from various types, using datasets from different domains. As expected, constraints tend to decrease the predictive quality of feature sets, but this effect is non-linear. So we observe feature sets both adhering to constraints and with high predictive quality. Second, we study a concrete setting in materials science. This part of our study sheds light on how one can analyze scientific hypotheses with the help of constraints.

Constrained feature selection for semisupervised color-texture image segmentation using spectral clustering

Color-texture image segmentation remains a challenging problem due to extensive color-texture variability. Thus, the limited prior knowledge that is expressed by pairwise constraints can be exploited to guide the segmentation process. We propose a new semisupervised method by combining constrained feature selection and spectral clustering (SC) to perform color-texture image segmentation. The pairwise constraints are used by the constraint feature selection to choose the most relevant features among an available set of color and texture features. For this purpose, an innovative constraint score is developed to evaluate a subset of features at one time. A specific constrained SC algorithm involving the pairwise constraints is then applied to regroup the pixels into clusters. Experimental results on four benchmark datasets show that the proposed constraint score outperforms the main state-of-the-art constraint scores and that our semisupervised segmentation method is competitive compared with supervised, semisupervised, and unsupervised state-of-the-art segmentation methods.

Unsupervised feature selection via transformed auto-encoder

As one of the fundamental research issues, feature selection plays a critical role in machine learning. By the removal of irrelevant features, it attempts to reduce computational complexities of upstream tasks, usually with computation accelerations and performance improvements. This paper proposes an auto-encoder based scheme for unsupervised feature selection. Due to the inherent consistency, this framework can solve traditional constrained feature selection problems approximately. Specifically, the proposed model takes non-negativity, orthogonality, and sparsity into account, whose internal characteristics are exploited sufficiently. It can also employ other loss functions and flexible activation functions. The former can fit a wide range of learning tasks, and the latter has the ability to play the role of regularization terms to impose regularization constraints on the model. Thereinafter, the proposed model is validated on multiple benchmark datasets, where various activation and loss functions are analyzed for finding better feature selectors. Finally, extensive experiments demonstrate the superiority of the proposed method against other compared state-of-the-arts.

A Constrained Feature Selection Approach Based on Feature Clustering and Hypothesis Margin Maximization

In this paper, we propose a semisupervised feature selection approach that is based on feature clustering and hypothesis margin maximization. The aim is to improve the classification accuracy by choosing the right feature subset and to allow building more interpretable models. Our approach handles the two core aspects of feature selection, i.e., relevance and redundancy, and is divided into three steps. First, the similarity weights between features are represented by a sparse graph where each feature can be reconstructed from the sparse linear combination of the others. Second, features are then hierarchically clustered identifying groups of the most similar ones. Finally, a semisupervised margin‐based objective function is optimized to select the most data discriminative feature from within each cluster, hence maximizing relevance while minimizing redundancy among features. Eventually, we empirically validate our proposed approach on multiple well‐known UCI benchmark datasets in terms of classification accuracy and representation entropy, where it proved to outperform four other semisupervised and unsupervised methods and competed with two widely used supervised ones.

A novel method of constrained feature selection by the measurement of pairwise constraints uncertainty

In the past decades, the rapid growth of computer and database technologies has led to the rapid growth of large-scale datasets. On the other hand, data mining applications with high dimensional datasets that require high speed and accuracy are rapidly increasing. Semi-supervised learning is a class of machine learning in which unlabeled data and labeled data are used simultaneously to improve feature selection. The goal of feature selection over partially labeled data (semi-supervised feature selection) is to choose a subset of available features with the lowest redundancy with each other and the highest relevancy to the target class, which is the same objective as the feature selection over entirely labeled data. This method actually used the classification to reduce ambiguity in the range of values. First, the similarity values of each pair are collected, and then these values are divided into intervals, and the average of each interval is determined. In the next step, for each interval, the number of pairs in this range is counted. Finally, by using the strength and similarity matrices, a new constraint feature selection ranking is proposed. The performance of the presented method was compared to the performance of the state-of-the-art, and well-known semi-supervised feature selection approaches on eight datasets. The results indicate that the proposed approach improves previous related approaches with respect to the accuracy of the constrained score. In particular, the numerical results showed that the presented approach improved the classification accuracy by about 3% and reduced the number of selected features by 1%. Consequently, it can be said that the proposed method has reduced the computational complexity of the machine learning algorithm despite increasing the classification accuracy.

Joint L2,1 Norm and Fisher Discrimination Constrained Feature Selection for Rational Synthesis of Microporous Aluminophosphates.

Feature selection has been regarded as an effective tool to help researchers understand the generating process of data. For mining the synthesis mechanism of microporous AlPOs, this paper proposes a novel feature selection method by joint l2,1 norm and Fisher discrimination constraints (JNFDC). In order to obtain more effective feature subset, the proposed method can be achieved in two steps. The first step is to rank the features according to sparse and discriminative constraints. The second step is to establish predictive model with the ranked features, and select the most significant features in the light of the contribution of improving the predictive accuracy. To the best of our knowledge, JNFDC is the first work which employs the sparse representation theory to explore the synthesis mechanism of six kinds of pore rings. Numerical simulations demonstrate that our proposed method can select significant features affecting the specified structural property and improve the predictive accuracy. Moreover, comparison results show that JNFDC can obtain better predictive performances than some other state-of-the-art feature selection methods.

Evolutionary Clustering Algorithm Using Criterion-Knowledge-Ranking for Multi-objective Optimization

There are variety of methods available to solve multi-objective optimization problems, very few utilizes criterion linkage between data objects in the searching phase, to improve final result. This article proposes an evolutionary clustering algorithm for multi-objective optimization. This paper aims to identify more relevant features based on criterion knowledge from the given data sets and also adopts neighborhood learning to improve the diversity and efficacy of the algorithm. This research is an extension of the previous work named neighborhood learning using k-means genetic algorithm (FS-NLMOGA) for multi-objective optimization which maximizes the compactness of the cluster and accuracy of the solution through constrained feature selection. The proposed objective finds the closest feature subset from the selected features of the data sets that also minimizes the cost while maintains the quality of the solution. The resultant cluster were analyzed and validated using cluster validity indexes. The proposed algorithm is tested with several UCI real-life data sets. The experimental results substantiates that the algorithm is efficient and robust .

Feature Selection Using K-Means Genetic Algorithm for Multi-objective Optimization

Abstract Multi-objective genetic -clustering algorithms are based on optimization which optimizes several objectives simultaneously. In multi-objective optimization problem (MOP), different objective function may have different properties. In the previous paper, multi-objective optimization on neighbourhood learning using k-means genetic algorithm (NLMOGA), was proposed and applied to several real-life data sets. This research paper aims to extend NLMOGA by maximizing the compactness and the accuracy of the solution through constraint feature selection on the selected sub-population. A new population is generated using NLMOGA and a constrained feature selection is applied to each sub-population. This method is developed to determine a suitable or closest set of objects from the group objects which will improve the robustness of NLMOGA for different instances of MOPs. In NLMOGA, a solution is selected from global population repository and then neighbourhood learning is made to promote the evolution of each objective for the selected solution. The effectiveness of this approach is evaluated with various real-life benchmark gene expression data sets.