Unsupervised Feature Selection Approach Research Articles

Graph autoencoder-based unsupervised feature selection with broad and local data structure preservation

Abstract Feature selection is a dimensionality reduction technique that selects a subset of representative features from high-dimensional data by eliminating irrelevant and redundant features. Recently, feature selection combined with sparse learning has attracted significant attention due to its outstanding performance compared with traditional feature selection methods that ignores correlation between features. These works first map data onto a low-dimensional subspace and then select features by posing a sparsity constraint on the transformation matrix. However, they are restricted by design to linear data transformation, a potential drawback given that the underlying correlation structures of data are often non-linear. To leverage a more sophisticated embedding, we propose an autoencoder-based unsupervised feature selection approach that leverages a single-layer autoencoder for a joint framework of feature selection and manifold learning. More specifically, we enforce column sparsity on the weight matrix connecting the input layer and the hidden layer, as in previous work. Additionally, we include spectral graph analysis on the projected data into the learning process to achieve local data geometry preservation from the original data space to the low-dimensional feature space. Extensive experiments are conducted on image, audio, text, and biological data. The promising experimental results validate the superiority of the proposed method.

Evaluation of a NeuroFuzzy Unsupervised Feature Selection Approach

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

A locality constrained self-representation approach for unsupervised feature selection

Recently, regularised self-representation (RSR) has been proposed as an efficient unsupervised feature selection algorithm. However, RSR only takes the self-representation ability of features into account, and neglects the locality structure preserving ability of features, which may degrade its performance. To overcome this limitation, a novel algorithm termed locality constrained regularised self-representation (LCRSR) is proposed in this paper. In our algorithm, a local scatter matrix is introduced to encode the locality geometric structure of high-dimensional data. Therefore, the locality information of the input database can be well preserved. Moreover, a simple yet efficient iterative update algorithm is developed to solve the proposed LCRSR. Extensive experiments are conducted on five publicly available databases (such as JAFFE, ORL, AR, COIL20 and SRBCT) to demonstrate the efficiency of the proposed algorithm. Experimental results show that LCRSR obtains better clustering performance than some other state-of-the-art approaches.

A locality constrained self-representation approach for unsupervised feature selection

A locality constrained self-representation approach for unsupervised feature selection

A new hybrid filter–wrapper feature selection method for clustering based on ranking

Feature selection is a common task in areas such as Pattern Recognition, Data Mining, and Machine Learning since it can help to improve prediction quality, reduce computation time and build more understandable models. Although feature selection for supervised classification has been widely studied, feature selection in the absence of class labels, namely feature selection for clustering or unsupervised feature selection, has been less addressed. Most existing unsupervised feature selection approaches suffer from the called “Bias of Criterion Values to Dimension,” which arises when feature subsets with different cardinality are evaluated by an internal evaluation clustering criterion. In this paper, we introduce a new hybrid filter–wrapper method for clustering, which combines the spectral feature selection framework using the Laplacian Score ranking and a modified Calinski–Harabasz index. The proposed method in the filter stage sorts the features according to their relevance, while in the wrapper stage, through our modified Calinski–Harabasz index that takes into account the cardinality of the feature subsets under evaluation, evaluates the features considering them as a subset rather than individually by using two well-known selection strategies. Experiments on different datasets show that the proposed method alleviates the “Bias of Criterion Values to Dimension” and, identifies and selects more relevant features than those selected by other reported hybrid filter–wrapper feature selection methods for clustering. Additionally, we also contrast our results against other filter and wrapper methods of the state-of-the-art.

TU‐D‐207B‐07: Radiomic Response Assessment for Recurrent Glioblastoma Treated with Bevacizumab in the BRAIN Trial

Purpose:To develop radiomic biomarkers for non‐invasive response assessment of Bevacizumab (Avastin; Genentech) treatment in recurrent glioblastoma multiforme (GBM).Methods:We analyzed prospectively acquired data from the BRAIN trial. For 167 patients, we extracted 71 radiomic features each from normalized post‐contrast T1‐weighted and fluid attenuation inversion recovery (FLAIR) sequences at baseline (pretreatment), and at follow‐ups of six and twelve weeks (post‐treatment), respectively, where available. For every imaging modality and time point, we selected 10 comprehensive features using an unsupervised feature selection approach that did not take clinical outcomes into account to limit overfitting. We investigated these features in terms of their prognostic value for overall survival (OS), progression‐free survival (PFS), as well as early and late progressors.Results:T1 and FLAIR features showed only low correlation at baseline (mean positive and negative Pearson correlation of ‐ 0.13 and 0.3) indicating complementary effects of imaging modalities at the radiomic level. We identified a textural‐heterogeneity feature (large area emphasis) that had higher prognostic value of OS at baseline as compared to conventional tumor volume (C‐index 0.71 vs 0.52, p = 0.006). Highest prognostic value was observed for the delta difference between baseline and six week follow‐up values for short‐run lengths of image intensities (Cindex 0.86, p = 2×10−8). Radiomic features generally showed similar or lower performances for PFS. Comparing radiomic feature values of early and late progressors directly showed significant differences in tumor surface and textural features after correcting for multiple‐testing (Wilcoxon rank sum test, p < 0.05).Conclusion:For the first time, our study allows the definition of radiomic response phenotypes of Bevacizumab treatment in recurrent GBM by leveraging high‐quality prospective trial data. Importantly, our data suggests the increased benefit of measuring radiomic patient profiles longitudinally after treatment has been initiated to monitor progression and resistance for immediate intervention and treatment adaptation.

Unsupervised Feature Selection with Structured Graph Optimization

Since amounts of unlabelled and high-dimensional data needed to be processed, unsupervised feature selection has become an important and challenging problem in machine learning. Conventional embedded unsupervised methods always need to construct the similarity matrix, which makes the selected features highly depend on the learned structure. However real world data always contain lots of noise samples and features that make the similarity matrix obtained by original data can't be fully relied. We propose an unsupervised feature selection approach which performs feature selection and local structure learning simultaneously, the similarity matrix thus can be determined adaptively. Moreover, we constrain the similarity matrix to make it contain more accurate information of data structure, thus the proposed approach can select more valuable features. An efficient and simple algorithm is derived to optimize the problem. Experiments on various benchmark data sets, including handwritten digit data, face image data and biomedical data, validate the effectiveness of the proposed approach.

Global and local structure preserving sparse subspace learning: An iterative approach to unsupervised feature selection

As we aim at alleviating the curse of high-dimensionality, subspace learning is becoming more popular. Existing approaches use either information about global or local structure of the data, and few studies simultaneously focus on global and local structures as the both of them contain important information. In this paper, we propose a global and local structure preserving sparse subspace learning (GLoSS) model for unsupervised feature selection. The model can simultaneously realize feature selection and subspace learning. In addition, we develop a greedy algorithm to establish a generic combinatorial model, and an iterative strategy based on an accelerated block coordinate descent is used to solve the GLoSS problem. We also provide whole iterate sequence convergence analysis of the proposed iterative algorithm. Extensive experiments are conducted on real-world datasets to show the superiority of the proposed approach over several state-of-the-art unsupervised feature selection approaches.

A graph theoretic approach for unsupervised feature selection

Feature subset selection is a major problem in data mining which can help to reduce computation time, improve prediction performance, and build understandable models. Specifically, feature selection realized in the absence of class labels, namely unsupervised feature selection, is challenging and interesting. In this paper a novel graph-theoretic approach for unsupervised feature selection has been proposed. The proposed method works in three steps. In the first step, the entire feature set is represented as a weighted graph. In the second step, the features are divided into several clusters using a community detection algorithm and finally in the third step, a novel iterative search strategy based on node centrality is developed to select the final subset of features. The proposed feature selection method offers two major advantages: first, our method groups features into different clusters based on their similarities, in which the features in the same cluster are similar to each other, and to obtain the reduced redundancy set, the final subset of features is selected from different clusters. Second, the node centrality measure and term variance are used to identify the most representative and informative feature subset; hence, the optimal size of the feature subset can be automatically determined. The performance of the proposed method has been compared to those of the state-of-the-art unsupervised and supervised feature selection methods on eight benchmark classification problems. The results show that our method has produced consistently better classification accuracies.

Unsupervised feature selection using swarm intelligence and consensus clustering for automatic fault detection and diagnosis in Heating Ventilation and Air Conditioning systems

Various sensory and control signals in a Heating Ventilation and Air Conditioning (HVAC) system are closely interrelated which give rise to severe redundancies between original signals. These redundancies may cripple the generalization capability of an automatic fault detection and diagnosis (AFDD) algorithm. This paper proposes an unsupervised feature selection approach and its application to AFDD in a HVAC system. Using Ensemble Rapid Centroid Estimation (ERCE), the important features are automatically selected from original measurements based on the relative entropy between the low- and high-frequency features. The materials used is the experimental HVAC fault data from the ASHRAE-1312-RP datasets containing a total of 49 days of various types of faults and corresponding severity. The features selected using ERCE (Median normalized mutual information (NMI)=0.019) achieved the least redundancies compared to those selected using manual selection (Median NMI=0.0199) Complete Linkage (Median NMI=0.1305), Evidence Accumulation K-means (Median NMI=0.04) and Weighted Evidence Accumulation K-means (Median NMI=0.048). The effectiveness of the feature selection method is further investigated using two well-established time-sequence classification algorithms: (a) Nonlinear Auto-Regressive Neural Network with eXogenous inputs and distributed time delays (NARX-TDNN); and (b) Hidden Markov Models (HMM); where weighted average sensitivity and specificity of: (a) higher than 99% and 96% for NARX-TDNN; and (b) higher than 98% and 86% for HMM is observed. The proposed feature selection algorithm could potentially be applied to other model-based systems to improve the fault detection performance.

SPARSE REPRESENTATION-BASED APPROACH FOR UNSUPERVISED FEATURE SELECTION

Dimension reduction methods including feature selection and feature extraction have played an important role in data mining and pattern recognition. In this study, we propose a novel unsupervised feature selection approach based on sparse representation theory, namely Sparsity Score (SS). Due to the sparse representation procedure, SS not only owns the global property of Variance Score (VS) and the local property of Laplacian Score (LS), but also possesses the discriminating nature. Experimental results, based on three well-known face datasets (Yale, ORL and CMU PIE), reveal that SS performs well in the evaluation of the feature significance, and it significantly outperforms VS and LS.

Feature selection with SVD entropy: Some modification and extension

Many approaches have been developed for dimensionality reduction. These approaches can broadly be categorized into supervised and unsupervised methods. In case of supervised dimensionality reduction, for any input vector the target value is known, which can be a class label also. In a supervised approach, our objective is to select a subset of features that has adequate discriminating power to predict the target value. This target value for an input vector is absent in case of an unsupervised approach. In an unsupervised scheme, we mainly try to find a subset that can capture the inherent “structure” of the data, such as the neighborhood relation or the cluster structure. In this work, we first study a Singular Value Decomposition (SVD) based unsupervised feature selection approach proposed by Varshavsky et al. Then we propose a modification of this method to improve its performance. An SVD-entropy based supervised feature selection algorithm is also developed in this paper. Performance evaluation of the algorithms is done on altogether 13 benchmark and one Synthetic data sets. The quality of the selected features is assessed using three indices: Sammon’s Error (SE), Cluster Preservation Index (CPI) and MisClassification Error (MCE) using a 1-Nearest Neighbor (1-NN) classifier. Besides showing the improvement of the modified unsupervised scheme over the existing one, we have also made a comparative study of the modified unsupervised and the proposed supervised algorithms with one well-known unsupervised and two popular supervised feature selection methods respectively. Our results reveal the effectiveness of the proposed algorithms in selecting relevant features.

A new unsupervised feature selection method for text clustering based on genetic algorithms

Nowadays a vast amount of textual information is collected and stored in various databases around the world, including the Internet as the largest database of all. This rapidly increasing growth of published text means that even the most avid reader cannot hope to keep up with all the reading in a field and consequently the nuggets of insight or new knowledge are at risk of languishing undiscovered in the literature. Text mining offers a solution to this problem by replacing or supplementing the human reader with automatic systems undeterred by the text explosion. It involves analyzing a large collection of documents to discover previously unknown information. Text clustering is one of the most important areas in text mining, which includes text preprocessing, dimension reduction by selecting some terms (features) and finally clustering using selected terms. Feature selection appears to be the most important step in the process. Conventional unsupervised feature selection methods define a measure of the discriminating power of terms to select proper terms from corpus. However up to now the valuation of terms in groups has not been investigated in reported works. In this paper a new and robust unsupervised feature selection approach is proposed that evaluates terms in groups. In addition a new Modified Term Variance measuring method is proposed for evaluating groups of terms. Furthermore a genetic based algorithm is designed and implemented for finding the most valuable groups of terms based on the new measure. These terms then will be utilized to generate the final feature vector for the clustering process . In order to evaluate and justify our approach the proposed method and also a conventional term variance method are implemented and tested using corpus collection Reuters-21578. For a more accurate comparison, methods have been tested on three corpuses and for each corpus clustering task has been done ten times and results are averaged. Results of comparing these two methods are very promising and show that our method produces better average accuracy and F1-measure than the conventional term variance method.

A scale space approach for unsupervised feature selection in mass spectra classification for ovarian cancer detection

BackgroundMass spectrometry spectra, widely used in proteomics studies as a screening tool for protein profiling and to detect discriminatory signals, are high dimensional data. A large number of local maxima (a.k.a. peaks) have to be analyzed as part of computational pipelines aimed at the realization of efficient predictive and screening protocols. With this kind of data dimensions and samples size the risk of over-fitting and selection bias is pervasive. Therefore the development of bio-informatics methods based on unsupervised feature extraction can lead to general tools which can be applied to several fields of predictive proteomics.ResultsWe propose a method for feature selection and extraction grounded on the theory of multi-scale spaces for high resolution spectra derived from analysis of serum. Then we use support vector machines for classification. In particular we use a database containing 216 samples spectra divided in 115 cancer and 91 control samples. The overall accuracy averaged over a large cross validation study is 98.18. The area under the ROC curve of the best selected model is 0.9962.ConclusionWe improved previous known results on the problem on the same data, with the advantage that the proposed method has an unsupervised feature selection phase. All the developed code, as MATLAB scripts, can be downloaded from

Unsupervised feature selection and general pattern discovery using Self-Organizing Maps for gaining insights into the nature of seismic wavefields

This study presents an unsupervised feature selection and learning approach for the discovery and intuitive imaging of significant temporal patterns in seismic single-station or network recordings. For this purpose, the data are parametrized by real-valued feature vectors for short time windows using standard analysis tools for seismic data, such as frequency-wavenumber, polarization, and spectral analysis. We use Self-Organizing Maps (SOMs) for a data-driven feature selection, visualization and clustering procedure, which is in particular suitable for high-dimensional data sets. Our feature selection method is based on significance testing using the Wald–Wolfowitz runs test for individual features and on correlation hunting with SOMs in feature subsets. Using synthetics composed of Rayleigh and Love waves and real-world data, we show the robustness and the improved discriminative power of that approach compared to feature subsets manually selected from individual wavefield parametrization methods. Furthermore, the capability of the clustering and visualization techniques to investigate the discrimination of wave phases is shown by means of synthetic waveforms and regional earthquake recordings.