Sparse Feature Selection Method Research Articles

Local sparse discriminative feature selection

Feature selection has been widely used in machine learning for a long time. In this paper, we propose a supervised local sparse discriminative feature selection method named LSDFS to obtain sparse features by imposing ℓ2,0-norm constraint on transformation matrix. Differently from traditional approaches, our method does not require approximation or relaxation schemes, such as ℓ2,p-norm to solve long-standing challenge. Our method is based on the trace difference form of Linear Discriminant Analysis (LDA), which can efficiently obtain discriminative information in low-dimensional space. In order to explore the local structure of data which contains more discriminative information, we adopt a sparse connections graph between anchor points and data points instead of fully-connected graph with time-consuming, and add a decay parameter to avoid trivial solutions, making the model more precisely. Extensive experiments conducted on synthetic datasets and several real-world datasets have demonstrated the advantages of our method.

Cauchy non-convex sparse feature selection method for the high-dimensional small-sample problem in motor imagery EEG decoding.

The time, frequency, and space information of electroencephalogram (EEG) signals is crucial for motor imagery decoding. However, these temporal-frequency-spatial features are high-dimensional small-sample data, which poses significant challenges for motor imagery decoding. Sparse regularization is an effective method for addressing this issue. However, the most commonly employed sparse regularization models in motor imagery decoding, such as the least absolute shrinkage and selection operator (LASSO), is a biased estimation method and leads to the loss of target feature information. In this paper, we propose a non-convex sparse regularization model that employs the Cauchy function. By designing a proximal gradient algorithm, our proposed model achieves closer-to-unbiased estimation than existing sparse models. Therefore, it can learn more accurate, discriminative, and effective feature information. Additionally, the proposed method can perform feature selection and classification simultaneously, without requiring additional classifiers. We conducted experiments on two publicly available motor imagery EEG datasets. The proposed method achieved an average classification accuracy of 82.98% and 64.45% in subject-dependent and subject-independent decoding assessment methods, respectively. The experimental results show that the proposed method can significantly improve the performance of motor imagery decoding, with better classification performance than existing feature selection and deep learning methods. Furthermore, the proposed model shows better generalization capability, with parameter consistency over different datasets and robust classification across different training sample sizes. Compared with existing sparse regularization methods, the proposed method converges faster, and with shorter model training time.

Molecular kinematic viscosity prediction of natural ester insulating oil based on sparse Machine learning models

The high viscosity property of natural ester insulating oils can be improved by means of molecular structure modification. However, it is difficult to find the optimal molecular structure that meets the conditions in the vast chemical space by relying on macroscopic experiments. Herein, the kinematic viscosity prediction models of triglyceride molecules based on sparse machine learning methods are proposed in this paper. Firstly, the molecular dynamics technique is used to simulate the kinematic viscosity (-20℃ to 40℃) of the main 20 triglyceride molecules in natural ester insulating oil, which provides extensive characterization data for subsequent model training. Secondly, the molecular descriptors are calculated for each triglyceride molecule based on density functional theory (DFT). Thirdly, four sparse feature selection methods (Boruta, RFECV, Pearson correlation coefficient, and mutual information) are used to identify the most relevant molecular descriptors for the kinematic viscosity of natural ester insulating oil and further analysis is performed. Finally, quantitative structure–property relationship (QSPR) models are constructed using automated machine learning methods to achieve efficient and accurate prediction of kinematic viscosity (average R2 reached 0.96). This study provides certain mechanistic explanations from the perspective of molecular descriptors and provides a predictive model basis for the viscosity modification of natural ester molecules, which serves as a theoretical guide to improve the kinematic viscosity of natural ester insulating oil.

Transmission line fault-cause classification based on multi-view sparse feature selection

With the development of communication and monitoring techniques, multi-source information in form of multi-view data is expected to support fault-cause identification for transmission line maintenance and fault disposal. However, the combined used of multi-view data poses a great challenge of information fusion. In respond to this challenge, this paper proposes a multi-view sparse feature selection (MVSFS) method to combine contextual and waveform features for fault-cause classification. l2,1-norm sparsity regularization is adopted to select discriminative features across two views and an ɛ-dragging technique is integrated into the regression model to enhance its discriminant ability. Subsequently, an iteration optimization algorithm is devised to solve the model. Experiments on a real-life fault dataset demonstrate that contextual and waveform features can achieve higher accuracy than single view learning only through appropriate fusion, and the proposed MVSFS can overcome the drawback of conventional feature selection methods and make a great improvement of classification performance with multi-view data.

A Novel Semisupervised Regression Method for Online Total Nitrogen Detection Using Double Structural Sparse Feature Extraction

To solve the problem of insufficient labeled samples in online total nitrogen (TN) detection, a novel semi-supervised learning method called double regularized structure graph learning (DRSGL) was proposed, which can effectively extract useful features and support the TN detection equipment to establish accurate detection models with few labeled samples. Based on temporal and spectral informativity, a high-quality graph structure method was designed firstly, which utilizes the spectrum-temporal prior knowledge hidden in the spectrum and enhances the efficiency of the graph model. Then, considering the double sparsity of TN spectrum samples, a double structural sparse feature selection method (DS²FS) was invented accordingly, which can excavate useful information from both spectral and temporal dimensions. Finally, to address the insufficient problem of labeled samples, an adaptive semi-supervised learning method was constructed by combining graph learning strategy and DS²FS, and applied to the TN rapid detection prototype, which can iteratively select important features and update graph structure of samples correspondingly. According to the experimental results based on practical application, DRSGL can effectively solve the problem brought by the insufficiency of labeled samples, which can utilize only 20% labeled samples to establish an accurate detection model satisfied national detection standard with relative error lower than 10%.

Lung cancer detection via breath by electronic nose enhanced with a sparse group feature selection approach

In the diagnosis of lung cancer, electronic nose (E-nose) has attracted much attention by detecting the volatile organic compounds (VOCs) in exhaled breath. In this research, an E-nose platform with a novel thermal desorption preconcentration subsystem is designed to verify whether analyzing VOCs can reliably differentiate lung cancer patients from healthy individuals and patients with benign pulmonary diseases. To this end, total 87 subjects (46 patients with lung cancer, 36 healthy volunteers and 5 patients with benign pulmonary diseases) are enrolled for the sensor array data collection. 13 composite features are extracted from each sensor, and some classical classifiers are established to demonstrate the feasibility of identifying lung cancer patients through VOCs. To improve the performance, considering the inherent characteristics of E-nose data, a sparse group feature selection (FS) method is applied to the raw sensor array data. It is observed that the FS method can reduce data dimensionality and improve the classification performance significantly. Statistical analysis for the effect of age and smoking habit on classification results shows that no significant influences are found.

Gene Identification from Microarray Data for Diagnosis of Acute Myeloid and Lymphoblastic Leukemia Using a Sparse Gene Selection Method

Background: Microarray experiments can simultaneously determine the expression of thousands of genes. Identification of potential genes from microarray data for diagnosis of cancer is important. This study aimed to identify genes for the diagnosis of acute myeloid and lymphoblastic leukemia using a sparse feature selection method.
 Materials and Methods: In this descriptive study, the expression of 7129 genes of 25 patients with acute myeloid leukemia (AML), and 47 patients with lymphoblastic leukemia (ALL) achieved by the microarray technology were used in this study. Then, the important genes were identified using a sparse feature selection method to diagnose AML and ALL tissues based on the machine learning methods such as support vector machine (SVM), Gaussian kernel density estimation based classifier (GKDEC), k-nearest neighbor (KNN), and linear discriminant classifier (LDC).
 Results: Diagnosis of ALL and AML was done with the accuracy of 100% using 8 genes of microarray data selected by the sparse feature selection method, GKDEC, and LDC. Moreover, the KNN classifier using 6 genes and the SVM classifier using 7 genes diagnosed AML and ALL with the accuracy of 91.18% and 94.12%, respectively. The gene with the description “Paired-box protein PAX2 (PAX2) gene, exon 11 and complete CDs” was determined as the most important gene in the diagnosis of ALL and AML.
 Conclusion: The experimental results of the current study showed that AML and ALL can be diagnosed with high accuracy using sparse feature selection and machine learning methods. It seems that the investigation of the expression of selected genes in this study can be helpful in the diagnosis of ALL and AML.

Unsupervised Feature Selection With Constrained ℓ₂,₀-Norm and Optimized Graph.

In this article, we propose a novel feature selection approach, named unsupervised feature selection with constrained l2,0 -norm (row-sparsity constrained) and optimized graph (RSOGFS), which unifies feature selection and similarity matrix construction into a general framework instead of independently performing the two-stage process; thus, the similarity matrix preserving the local manifold structure of data can be determined adaptively. Unlike those sparse learning-based feature selection methods that can only solve the relaxation or approximation problems by introducing sparsity regularization term into the objective function, the proposed method directly tackles the original l2,0 -norm constrained problem to achieve group feature selection. Two optimization strategies are provided to solve the original sparse constrained problem. The convergence and approximation guarantees for the new algorithms are rigorously proved, and the computational complexity and parameter determination are theoretically analyzed. Experimental results on real-world data sets show that the proposed method for solving a nonconvex problem is superior to the state of the arts for solving the relaxed or approximate convex problems.

Multi-View Multi-Label Learning With Sparse Feature Selection for Image Annotation

In image analysis, image samples are always represented by multiple view features and associated with multiple class labels for better interpretation. However, multiple view data may include noisy, irrelevant and redundant features, while multiple class labels can be noisy and incomplete. Due to the special data characteristic, it is hard to perform feature selection on multi-view multi-label data. To address these challenges, in this paper, we propose a novel multi-view multi-label sparse feature selection (MSFS) method, which exploits both view relations and label correlations to select discriminative features for further learning. Specifically, the multi-labeled information is decomposed into a reduced latent label representation to capture higher level concepts and correlations among multiple labels. Multiple local geometric structures are constructed to exploit visual similarities and relations for different views. By taking full advantage of the latent label representation and multiple local geometric structures, the sparse regression model with an $l_{2,1}$ -norm and an Frobenius norm ( $F$ -norm) penalty terms is utilized to perform hierarchical feature selection, where the $F$ -norm penalty performs high-level (i.e., view-wise) feature selection to preserve the informative views and the $l_{2,1}$ -norm penalty conducts low-level (i.e., row-wise) feature selection to remove noisy features. To solve the proposed formulation, we also devise a simple yet efficient iterative algorithm. Experiments and comparisons on real-world image datasets demonstrate the effectiveness and potential of MSFS.

A robust graph-based semi-supervised sparse feature selection method

Feature selection is used for excluding redundant features and enhancing learning performance. Abundant unlabeled data are existed in many applications which can be used in semi-supervised feature selection. Semi-supervised sparse feature selection methods have been presented to apply labeled and unlabeled data and consider the correlation among features. Most of the sparse methods utilize square-norm based loss function which is sensitive to outliers. In this paper, we propose a robust Graph-based Semi-Supervised Sparse Feature Selection (GS3FS) method based on the mixed convex and non-convex l2,p-norm (0 < p ≤ 1) minimization on regularization and loss function. The l2,p-norm based regularization ensures that the method selects relevant and sparse features, and the l2,p-norm based loss function makes the method robust to outliers. Proposed method takes the advantages of manifold learning and l2,p-norm minimization on regularization and loss function. The method includes the mixed l2,p-norm which is difficult to solve. To solve the proposed method for the convex (p = 1) and non-convex (0 < p <1) cases, we propose an efficient unified algorithm and prove its convergence. Extensive experiments are carried out on various classification and regression datasets for evaluating the proposed method. The results indicate that the proposed method selects the relevant features and improves the performance of classification and regression models.

Sparse feature selection: Relevance, redundancy and locality structure preserving guided by pairwise constraints

Selection of features as a pre-processing stage is an essential issue in many machine learning tasks (such as classification) to reduce data dimensionality as there are many irrelevant and redundant features that can mislead the learning process. Graph-based sparse feature selection is developed to overcome this issue. In this paper, a novel graph-based sparse feature selection method is proposed that take into account both issues: relevancy and redundancy analysis. An empirical loss function joining with ℓ1-norm regularization term is proposed to overcome the relevancy issue and the redundancy issue is overcome by introducing a regularization term that prefers uncorrelated features. Furthermore, the proposed learning procedure is guided by two different sets of supervision information as pairs of must-linked (positive) and cannot-linked (negative) constraint sets to select a discriminative feature subset. These guiding information besides the whole data points are encoded in the graph Laplacian matrix that preserves the locality structure of the original data. The graph Laplacian matrix is constructed by two different approaches. Our first approach tries to preserve the structure of the original data guided just by the positive data points (unique samples in the must-linked constraints), and our second approach applies a normalized adapted affinity matrix to embed the pairwise must-linked and cannot-linked constraints as well as the neighborhood relationships information, all together. The experimental results on a number of several datasets from the University of California-Irvine machine learning repository, in addition to several high dimensional gene expression datasets show the efficacy of the proposed methods in the classification tasks compared to several powerful feature selection methods.

Emotion recognition based on sparse learning feature selection method for social communication

At present, social communication occupies an increasingly important position in people’s lives. It is especially important to correctly identify other people’s emotional information in social communication. Therefore, emotion recognition is also a hot issue in intelligent information processing. This paper focuses on this problem, mainly based on the feature selection method in emotional recognition of EEG signals. How to select emotion-related features in high-dimensional features is a key factor to achieve fast, accurate and effective identification. This paper proposes a feature selection method based on sparse learning. The method can find a small number of features that contribute to the reconstruction of the category information from the high-dimensional feature space, so as to achieve the purpose of quickly and efficiently acquiring small and emotionally related features. The experimental results show that compared with the traditional method, the proposed method reduces the time consumption of feature selection and obtains a higher emotional five-category correct rate under the same feature dimension.

A sparse representation‐based radiomics for outcome prediction of higher grade gliomas

Accurately predicting outcome (i.e., overall survival (OS) time) for higher grade glioma (HGG) has great clinical value and would provide optimized guidelines for treatment planning. Radiomics focuses on revealing underlying pathophysiological information in biomedical images for disease analysis and demonstrates promising prognostic clinical performance. In this paper, we propose a novel sparse representation-based radiomics framework to predict if HGG patients would have long or short OS time. First, taking advantages of the scale invariant feature transform (SIFT) feature in image characterizing, we developed a sparse representation-based method to convert a local SIFT descriptor into a global tumor feature. Next, because preserving sample structure is beneficial for feature selection, we proposed a locality preserving projection and sparse representation-combined feature selection method to select more discriminative features for tumor classification. Finally, we employed a multifeature collaborative sparse representation classification to combine the information of multimodal images to classify OS time. Three experiments were performed on the two datasets provided by different institutions. Specifically, the proposed model was trained and independently tested on dataset 1 (135 subjects), on dataset 2 (86 subjects), and on the combination of dataset 1 and dataset 2, respectively. Experimental results demonstrated that the proposed method achieved encouraging prediction performance, exhibiting a testing accuracy of 93.33% on dataset 1 (one modality), 92.31% on dataset 2 (two modalities), and 87.93% on the combined dataset (one modality). The sparse representation theory provides reasonable solutions to feature extraction, feature selection, and classification for radiomics. This study provides a promising tool to enhance the prediction performance of HGG patient's outcome.

Sparse structural feature selection for multitarget regression

Feature selection is one of the most important dimensionality reduction techniques for its efficiency and interpretation. Recently, some sparse regression-based feature selection methods have obtained an increased attention from the research community. However, previous sparse regression-based feature selection methods are limited by the double-layer structure. To improve the learning performance, in this work, we propose a sparse structural feature selection model for multitarget regression, which utilizes a structure matrix to expand the double-layer to multi-layer structure. Our aim tries to explore the essential inter-target correlations. To enhance the robustness of our proposed method, we emphasize a joint ℓ2, 1-norm minimization on the loss function, regression matrix, and structure matrix. An effective optimization method with provable convergence behavior is also proposed. Extensive experimental results on multivariate prediction demonstrate the effectiveness of our proposed method.

Multiview Hessian Semisupervised Sparse Feature Selection for Multimedia Analysis

Facing a large number of unlabeled data and a small number of labeled data, semisupervised sparse feature selection has received increasing attention in recent years. However, most semisupervised feature selection algorithms are developed for single-view data and cannot naturally handle multiview data. Moreover, most existing semisupervised sparse feature selection methods are based on Laplacian regularization, which is a lack of extrapolating power. To overcome the above-mentioned drawbacks, we present a multiview Hessian semi-supervised sparse feature selection (MHSFS) framework in this paper. MHSFS can directly accomplish multiview sparse feature selection by exploiting multiview learning to reveal and leverage the correlated and complemental information among different views. In addition, MHSFS can achieve better performance based on Hessian regularization, which favors functions whose values linearly vary with respect to geodesic distance and preserves the local manifold structure well. A simple yet efficient iterative method is proposed to solve the objective function, followed by convergence analysis. We apply the proposed method into different multimedia analysis tasks, such as image annotation, video concept detection, and 3D motion analysis. The results show that MHSFS outperforms the state-of-the-art sparse feature selection methods and achieves good performance.

Compartmental sparse feature selection method for Alzheimer's disease identification.

For high-dimensional magnetic resonance imaging (MRI) data, many feature selection methods have been proposed to reduce feature dimension in the study of computer-aided Alzheimer's disease (AD) diagnosis. This paper presents a compartmental sparse feature selection method used for AD identification. Based on the derived atlas-based regions-of-interest (ROIs) of brain, the proposed method partitioned the T1-weighted MRI data into several compartments. It performs feature selection and classification compartmentally according to the local feature dimension estimation and local feature selection using sparse principal component analysis (SPCA) method followed with elastic-net logistic regression (ENLR) classifier. Experimental results showed that the proposed method improves the classification performance for small ROIs with high computational efficiency.

Hypergraph regularized sparse feature learning

As an important pre-processing stage in many machine learning and pattern recognition domains, feature selection deems to identify the most discriminate features for a compact data representation. As typical feature selection methods, Lasso and its variants using the l1-norm based regularization have received much attention in recent years. However, most of existing l1-norm based sparse feature selection methods ignore the structure information of data or only consider the pairwise relationships among samples. In this paper, we propose a hypergraph regularized sparse feature learning method, where the high-order relationships among samples are modeled and incorporated into the learning process. Specifically, we first construct a hypergraph with multiple hyperedges to capture the high-order relationships among samples, followed by the computation of a hypergraph Laplacian matrix. Then, we propose a hypergraph regularization term, and a hypergraph regularized Lasso model. We conduct a series of experiments on a number of data sets from UCI machine learning repository, and two real-world neuroimaging based classification tasks. Experimental results demonstrate that the proposed method achieves promising classification results, compared with several well known feature selection approaches.

Semi-supervised sparse feature selection based on multi-view Laplacian regularization

Semi-supervised sparse feature selection, which can exploit the large number unlabeled data and small number labeled data simultaneously, has placed an important role in web image annotation. However, most of the semi-supervised sparse feature selection methods are developed for single-view data and these methods cannot naturally deal with the multi-view data, though it has shown that leveraging information contained in multiple views can dramatically improve the feature selection performance. Recently, multi-view learning has obtained much research attention because it can reveal and leverage the correlated and complementary information between different views. So in this paper, we apply multi-view learning into semi-supervised sparse feature selection and propose a semi-supervised sparse feature selection method based on multi-view Laplacian regularization, namely, multi-view Laplacian sparse feature selection (MLSFS).11MLSFS: Multi-view Laplacian Sparse Feature Selection. MLSFS utilizes multi-view Laplacian regularization to boost semi-supervised sparse feature selection performance. A simple iterative method is proposed to solve the objective function of MLSFS. We apply MLSFS algorithm into image annotation task and conduct experiments on two web image datasets. The experimental results show that the proposed MLSFS outperforms the state-of-art single-view sparse feature selection methods.

Anomaly detection based on efficient Euclidean projection

Machine-learning algorithms are widely applied in traffic classification and anomaly detection. Due to the tremendous traffic on the network, an extremely challenging question arises: how to efficiently and accurately detect the anomalous flow from the backbone network. One solution is proposed, online anomaly-detection scheme, which is based on the sparse feature selection method, Lasso. The sparse feature selection can be efficiently solved by reformulating the problem as an optimization problem with an i¾?1-ball constraint. At the evaluation stage, the authors preprocessed the raw data trace from the trans-Pacific backbone link between Japan and the United States and generated an evaluation data set. Their empirical study shows that the feature selection step can be solved quickly by applying the efficient Euclidean projection method; indeed, doing so resolves the feature selection step faster than using three classical i¾?1-min solvers. In terms of overall accuracy, true positive rate, false positive rate, precision, and F-measure, the proposed scheme improves the quality of detection. Copyright © 2015John Wiley & Sons, Ltd.

Sparse feature selection methods identify unexpected global cellular response to strontium-containing materials

Despite the increasing sophistication of biomaterials design and functional characterization studies, little is known regarding cells' global response to biomaterials. Here, we combined nontargeted holistic biological and physical science techniques to evaluate how simple strontium ion incorporation within the well-described biomaterial 45S5 bioactive glass (BG) influences the global response of human mesenchymal stem cells. Our objective analyses of whole gene-expression profiles, confirmed by standard molecular biology techniques, revealed that strontium-substituted BG up-regulated the isoprenoid pathway, suggesting an influence on both sterol metabolite synthesis and protein prenylation processes. This up-regulation was accompanied by increases in cellular and membrane cholesterol and lipid raft contents as determined by Raman spectroscopy mapping and total internal reflection fluorescence microscopy analyses and by an increase in cellular content of phosphorylated myosin II light chain. Our unexpected findings of this strong metabolic pathway regulation as a response to biomaterial composition highlight the benefits of discovery-driven nonreductionist approaches to gain a deeper understanding of global cell-material interactions and suggest alternative research routes for evaluating biomaterials to improve their design.