Semi-supervised Feature Selection Research Articles

BRNS + SSFSM-DTI: A hybrid method for drug-target interaction prediction based on balanced reliable negative samples and semi-supervised feature selection

De novo drug discovery is a time-consuming and costly process. Drug repositioning, which means finding new applications for existing drugs, is one of the most effective approaches to reduce time and cost of detecting a new drug. Predicting drug-target interactions (DTIs) can facilitate the drug repositioning and consequently accelerate the process of drug discovery. The diversity of drug descriptors and protein features as well as the lack of access to experimentally-confirmed non-interacting drug-target pairs as negative samples are the major challenges in predicting new DTIs. In this study, we present a modified algorithm for extracting balanced reliable negative samples named BRNS. Moreover, we propose a semi-supervised feature selection method called SSFSM-DTI and compare the performance of our hybrid predictive model with the models constructed by random selection of negative samples and the other well-known feature selection methods on benchmark DTI datasets. The results show that the combination of BRNS and SSFSM-DTI is superior to the random selection of negative samples and the state-of-the-art feature selection methods in most cases. Hence, it can be used as a guideline in the drug repositioning process. The source codes and all datasets used in this study are available at https://github.com/LBDSoft/BRNS.

3-3FS: ensemble method for semi-supervised multi-label feature selection

3-3FS: ensemble method for semi-supervised multi-label feature selection

Label propagation-based semi-supervised feature selection on decoding clinical phenotypes with RNA-seq data

BackgroundClinically, behavior, cognitive, and mental functions are affected during the neurodegenerative disease progression. To date, the molecular pathogenesis of these complex disease is still unclear. With the rapid development of sequencing technologies, it is possible to delicately decode the molecular mechanisms corresponding to different clinical phenotypes at the genome-wide transcriptomic level using computational methods. Our previous studies have shown that it is difficult to distinguish disease genes from non-disease genes. Therefore, to precisely explore the molecular pathogenesis under complex clinical phenotypes, it is better to identify biomarkers corresponding to different disease stages or clinical phenotypes. So, in this study, we designed a label propagation-based semi-supervised feature selection approach (LPFS) to prioritize disease-associated genes corresponding to different disease stages or clinical phenotypes.MethodsIn this study, we pioneering put label propagation clustering and feature selection into one framework and proposed label propagation-based semi-supervised feature selection approach. LPFS prioritizes disease genes related to different disease stages or phenotypes through the alternative iteration of label propagation clustering based on sample network and feature selection with gene expression profiles. Then the GO and KEGG pathway enrichment analysis were carried as well as the gene functional analysis to explore molecular mechanisms of specific disease phenotypes, thus to decode the changes in individual behavioral and mental characteristics during neurodegenerative disease progression.ResultsLarge amounts of experiments were conducted to verify the performance of LPFS with Huntington’s gene expression data. Experimental results shown that LPFS performs better in comparison with the-state-of-art methods. GO and KEGG enrichment analysis of key gene sets shown that TGF-beta signaling pathway, cytokine-cytokine receptor interaction, immune response, and inflammatory response were gradually affected during the Huntington’s disease progression. In addition, we found that the expression of SLC4A11, ZFP474, AMBP, TOP2A, PBK, CCDC33, APSL, DLGAP5, and Al662270 changed seriously by the development of the disease.ConclusionsIn this study, we designed a label propagation-based semi-supervised feature selection model to precisely selected key genes of different disease phenotypes. We conducted experiments using the model with Huntington’s disease mice gene expression data to decode the mechanisms of it. We found many cell types, including astrocyte, microglia, and GABAergic neuron, could be involved in the pathological process.

Semi‐supervised Robust Feature Selection with ℓq‐Norm Graph for Multiclass Classification

Flexible manifold embedding (FME) is a semi-supervised dimension reduction framework. It has been extended into feature selection by using different loss functions and sparse regularization methods. However, these kind of methods used the quadratic form of graph embedding, thus the results are sensitive to noise and outliers. In this paper, we propose a general semi-supervised feature selection model that optimizes an ℓ q -norm of FME to decrease the noise sensitivity. Compare to the fixed parameter model, the ℓ q -norm graph brings flexibility to balance the manifold smoothness and the sensitivity to noise by tuning its parameter. We present an efficient iterative algorithm to solve the proposed ℓ q -norm graph embedding based semi-supervised feature selection problem, and offer a rigorous convergence analysis. Experiments performed on typical image and speech emotion datasets demonstrate that our method is effective for the multiclass classification task, and outperforms the related state-of-the-art methods.

A recursive feature retention method for semi-supervised feature selection

A recursive feature retention method for semi-supervised feature selection

Semi-supervised feature selection with minimal redundancy based on local adaptive

With the speedy development of network technology, diverse data increase by hundreds of millions per hour, causing increasing pressure on the acquisition of data labels. Semi-supervised feature selection has been among the forefront of dimensionality reduction research due to the outstanding achievements of “small labels” and “high efficiency”. Especially, the graph-based methods use data with missing labels completely and effectively, prompting it to become a research hotspots in semi-supervised feature selection. However, the existing graph-based methods do not take into account the effects of outliers, noise, and redundancy of selected features simultaneously. To solve those problems, a novel semi-supervised feature selection method based on local adaptive and minimal redundancy is proposed. The local structure is flexibly assigned weights according to the data conditions, thereby reducing the impact of outliers and noise; moreover, a high similarity penalty mechanism is introduced in the feature mapping matrix to promote discrimination and low redundancy of the selected feature subset. In addition, an iterative method is designed and its convergence is proved theoretically and experimentally. Finally, the proposed algorithm is verified to be stable and effective through experiments from five aspects on sixteen public datasets.

Adaptive discriminant analysis for semi-supervised feature selection

As semi-supervised feature selection is becoming much more popular among researchers, many related methods have been proposed in recent years. However, many of these methods first compute a similarity matrix prior to feature selection, and the matrix is then fixed during the subsequent feature selection process. Clearly, the similarity matrix generated from the original dataset is susceptible to the noise features. In this paper, we propose a novel adaptive discriminant analysis for semi-supervised feature selection, namely, SADA. Instead of computing a similarity matrix first, SADA simultaneously learns an adaptive similarity matrix S and a projection matrix W with an iterative process. Moreover. we introduce the ℓ2,p norm to control the sparsity of S by adjusting p. Experimental results show that S will become sparser with the decrease of p. The experimental results for synthetic datasets and nine benchmark datasets demonstrate the superiority of SADA, in comparison with 6 semi-supervised feature selection methods.

Binary Label Learning for Semi-supervised Feature Selection

Binary Label Learning for Semi-supervised Feature Selection

Prognostic outcome prediction by semi-supervised least squares classification

Although great progress has been made in prognostic outcome prediction, small sample size remains a challenge in obtaining accurate and robust classifiers. We proposed the Rescaled linear square Regression based Least Squares Learning (RRLSL), a jointly developed semi-supervised feature selection and classifier, for predicting prognostic outcome of cancer patients. RRLSL used the least square regression to identify the scale factors and then rank the features in available multiple types of molecular data. We applied the unlabeled multiple molecular data in conjunction with the labeled data to develop a similarity graph. RRLSL produced the constraint with kernel functions to bridge the gap between label information and geometry information from messenger RNA and microRNA expression profiling. Importantly, this semi-supervised model proposed the least squares learning with L2 regularization to develop a semi-supervised classifier. RRLSL suggested the performance improvement in the prognostic outcome prediction and successfully discriminated between the recurrent patients and non-recurrent ones. We also demonstrated that RRLSL improved the accuracy and Area Under the Precision Recall Curve (AUPRC) as compared to the baseline semi-supervised methods. RRLSL is available for a stand-alone software package (https://github.com/ShiMGLab/RRLSL). A short abstract We proposed the Rescaled linear square Regression based Least Squares Learning (RRLSL), a jointly developed semi-supervised feature selection and classifier, for predicting prognostic outcome of cancer patients. RRLSL used the least square regression to identify the scale factors to rank the features in available multiple types of molecular data. RRLSL produced the constraint with kernel functions to bridge the gap between label information and geometry information from messenger RNA and microRNA expression profiling. Importantly, this semi-supervised model proposed the least squares learning with L2 regularization to develop the semi-supervised classifier. RRLSL suggested the performance improvement in the prognostic outcome prediction and successfully discriminated between the recurrent patients and non-recurrent ones.

A novel method of constrained feature selection by the measurement of pairwise constraints uncertainty

In the past decades, the rapid growth of computer and database technologies has led to the rapid growth of large-scale datasets. On the other hand, data mining applications with high dimensional datasets that require high speed and accuracy are rapidly increasing. Semi-supervised learning is a class of machine learning in which unlabeled data and labeled data are used simultaneously to improve feature selection. The goal of feature selection over partially labeled data (semi-supervised feature selection) is to choose a subset of available features with the lowest redundancy with each other and the highest relevancy to the target class, which is the same objective as the feature selection over entirely labeled data. This method actually used the classification to reduce ambiguity in the range of values. First, the similarity values of each pair are collected, and then these values are divided into intervals, and the average of each interval is determined. In the next step, for each interval, the number of pairs in this range is counted. Finally, by using the strength and similarity matrices, a new constraint feature selection ranking is proposed. The performance of the presented method was compared to the performance of the state-of-the-art, and well-known semi-supervised feature selection approaches on eight datasets. The results indicate that the proposed approach improves previous related approaches with respect to the accuracy of the constrained score. In particular, the numerical results showed that the presented approach improved the classification accuracy by about 3% and reduced the number of selected features by 1%. Consequently, it can be said that the proposed method has reduced the computational complexity of the machine learning algorithm despite increasing the classification accuracy.

A Two-Phase Approach for Semi-Supervised Feature Selection

This paper proposes a novel approach for selecting a subset of features in semi-supervised datasets where only some of the patterns are labeled. The whole process is completed in two phases. In the first phase, i.e., Phase-I, the whole dataset is divided into two parts: The first part, which contains labeled patterns, and the second part, which contains unlabeled patterns. In the first part, a small number of features are identified using well-known maximum relevance (from first part) and minimum redundancy (whole dataset) based feature selection approaches using the correlation coefficient. The subset of features from the identified set of features, which produces a high classification accuracy using any supervised classifier from labeled patterns, is selected for later processing. In the second phase, i.e., Phase-II, the patterns belonging to the first and second part are clustered separately into the available number of classes of the dataset. In the clusters of the first part, take the majority of patterns belonging to a cluster as the class for that cluster, which is given already. Form the pairs of cluster centroids made in the first and second part. The centroid of the second part nearest to a centroid of the first part will be paired. As the class of the first centroid is known, the same class can be assigned to the centroid of the cluster of the second part, which is unknown. The actual class of the patterns if known for the second part of the dataset can be used to test the classification accuracy of patterns in the second part. The proposed two-phase approach performs well in terms of classification accuracy and number of features selected on the given benchmarked datasets.

Semi-supervised neighborhood discrimination index for feature selection

Neighborhood discriminant index (NDI) is an effective feature selection method for supervised learning. In reality, it is easy to obtain unlabeled data and is costly to tag them all. Thus, the given dataset commonly has only a small amount of tagged samples and a large amount of unlabeled ones, which cannot be handled by supervised learning methods. For this situation, we propose a semi-supervised feature selection method called semi-supervised neighborhood discriminant index (SSNDI) that combines NDI and the Laplacian score method to effectively deal with both labeled and unlabeled samples. The goal of SSNDI is to find an optimal feature subset that has a good ability to keep local geometrical structure and to distinguish samples belonging to different classes. In SSNDI, the classical Laplacian score method is modified to cooperate the iterative form of NDI. In each iteration, SSNDI picks up an important feature according to the new criterion that is a mixture of NDI and the modified Laplacian score. Extensive experiments are conducted on UCI and microarray gene datasets. Experimental results confirm that SSNDI can achieve a better performance than NDI and the other state-of-the-art semi-supervised methods.

A robust graph-based semi-supervised sparse feature selection method

Feature selection is used for excluding redundant features and enhancing learning performance. Abundant unlabeled data are existed in many applications which can be used in semi-supervised feature selection. Semi-supervised sparse feature selection methods have been presented to apply labeled and unlabeled data and consider the correlation among features. Most of the sparse methods utilize square-norm based loss function which is sensitive to outliers. In this paper, we propose a robust Graph-based Semi-Supervised Sparse Feature Selection (GS3FS) method based on the mixed convex and non-convex l2,p-norm (0 < p ≤ 1) minimization on regularization and loss function. The l2,p-norm based regularization ensures that the method selects relevant and sparse features, and the l2,p-norm based loss function makes the method robust to outliers. Proposed method takes the advantages of manifold learning and l2,p-norm minimization on regularization and loss function. The method includes the mixed l2,p-norm which is difficult to solve. To solve the proposed method for the convex (p = 1) and non-convex (0 < p <1) cases, we propose an efficient unified algorithm and prove its convergence. Extensive experiments are carried out on various classification and regression datasets for evaluating the proposed method. The results indicate that the proposed method selects the relevant features and improves the performance of classification and regression models.

Local preserving logistic I-Relief for semi-supervised feature selection

Logistic I-Relief (LIR), a supervised feature selection method, has been successfully extended for semi-supervised feature selection. However, exiting semi-supervised methods based on LIR are incapable of maintaining the local structure of data. In this paper, we propose a local preserving logistic I-Relief (LPLIR) algorithm for multi-class semi-supervised feature selection. The goal of LPLIR is to maximize the expected margin of the given labeled data and remain the local structural information of all given data, where the local structural information is represented by a local preserving regularization. Additionally, extensive experiments conducted on real-world datasets demonstrate that LPLIR is superior to the existing state-of-the-art semi-supervised feature selection methods.

Semi-Supervised Feature Selection via Sparse Rescaled Linear Square Regression

With the rapid increase of the data size, it has increasing demands for selecting features by exploiting both labeled and unlabeled data. In this paper, we propose a novel semi-supervised embedded feature selection method. The new method extends the least square regression model by rescaling the regression coefficients in the least square regression with a set of scale factors, which is used for evaluating the importance of features. An iterative algorithm is proposed to optimize the new model. It has been proved that solving the new model is equivalent to solving a sparse model with a flexible and adaptable $\ell _{2,p}$ l 2 , p norm regularization. Moreover, the optimal solution of scale factors provides a theoretical explanation for why we can use $\lbrace \left\Vert \mathbf {w}^{1} \right\Vert _{2},\ldots, \left\Vert \mathbf {w}^{d} \right\Vert _{2}\rbrace$ { w 1 2 , ... , w d 2 } to evaluate the importance of features. Experimental results on eight benchmark data sets show the superior performance of the proposed method.

Intrusion Detection System- Via Fuzzy Artmap in Addition with Advance Semi Supervised Feature Selection

Intrusion Detection System- Via Fuzzy Artmap in Addition with Advance Semi Supervised Feature Selection

Multi-view adaptive semi-supervised feature selection with the self-paced learning

In recent decades, semi-supervised feature selection (SSFS) has gained extensive research in the field of machine learning and computer vision. Most SSFS algorithms are based on graph-based semi-supervised learning (GSSL), and their performance depend heavily on the quality of the Laplacian weight graph. However, the Laplacian weight graph can't be changed once it is constructed, which greatly restricts the performance of SSFS. To address this defect, in this paper we propose a novel Multi-view Adaptive Semi-supervised Feature Selection (MASFS) algorithm, which introduces the self-paced learning (SPL) into SSFS to make the Laplacian weight graph adaptively change according to the current predicted information. Meanwhile, the MASFS algorithm utilizes the multi-view learning to effectively explore the complementary and related information contained in different views to enhance SSFS performance. We propose a valid iterative algorithm for optimizing the objective function, followed by the convergence analysis and the complexity analysis. To illustrate the effectiveness of the MASFS algorithm, some experiments are carried out on NUS-WIDE dataset and MSRA-MM2.0 dataset and the experimental results indicate that MASFS has better performance than other SSFS algorithms based on GSSL.

Joint Semi-Supervised Feature Selection and Classification through Bayesian Approach

With the increasing data dimensionality, feature selection has become a fundamental task to deal with high-dimensional data. Semi-supervised feature selection focuses on the problem of how to learn a relevant feature subset in the case of abundant unlabeled data with few labeled data. In recent years, many semi-supervised feature selection algorithms have been proposed. However, these algorithms are implemented by separating the processes of feature selection and classifier training, such that they cannot simultaneously select features and learn a classifier with the selected features. Moreover, they ignore the difference of reliability inside unlabeled samples and directly use them in the training stage, which might cause performance degradation. In this paper, we propose a joint semi-supervised feature selection and classification algorithm (JSFS) which adopts a Bayesian approach to automatically select the relevant features and simultaneously learn a classifier. Instead of using all unlabeled samples indiscriminately, JSFS associates each unlabeled sample with a self-adjusting weight to distinguish the difference between them, which can effectively eliminate the irrelevant unlabeled samples via introducing a left-truncated Gaussian prior. Experiments on various datasets demonstrate the effectiveness and superiority of JSFS.

Semi-Supervised Feature Selection with Adaptive Discriminant Analysis

In this paper, we propose a novel Adaptive Discriminant Analysis for semi-supervised feature selection, namely SADA. Instead of computing fixed similarities before performing feature selection, SADA simultaneously learns an adaptive similarity matrix S and a projection matrix W with an iterative method. In each iteration, S is computed from the projected distance with the learned W and W is computed with the learned S. Therefore, SADA can learn better projection matrix W by weakening the effect of noise features with the adaptive similarity matrix. Experimental results on 4 data sets show the superiority of SADA compared to 5 semisupervised feature selection methods.

Local adaptive learning for semi-supervised feature selection with group sparsity

Feature selection is often an important tool for many machine learning and data mining tasks. By largely removing the irrelevant features and reducing the complexity of the data processing, feature selection can significantly improve the performance of subsequent classification or clustering tasks. As a result of the rapid development of social networking, large amounts of high-dimensional data have been generated. Due to the high cost of collecting sufficient labels, graph-based semi-supervised feature selection algorithms have attracted the most research interest; however, these approaches neglect the local sparsity of data. Accordingly, motivated by the merits of adaptive learning and sparse learning, we propose a novel feature selection method with a local adaptive loss function and a global sparsity constraint in this paper. Our method can operate more flexibly to model data with different distributions. Moreover, when both the local and global sparsity of data is considered, our method is more capable of selecting the most discriminating features. Experimental results on various real-world applications demonstrate the effectiveness of the proposed feature selection method compared to several state-of-the-art methods.