Semi-supervised Feature Selection Research Articles

Semisupervised Feature Analysis by Mining Correlations Among Multiple Tasks

In this paper, we propose a novel semisupervised feature selection framework by mining correlations among multiple tasks and apply it to different multimedia applications. Instead of independently computing the importance of features for each task, our algorithm leverages shared knowledge from multiple related tasks, thus improving the performance of feature selection. Note that the proposed algorithm is built upon an assumption that different tasks share some common structures. The proposed algorithm selects features in a batch mode, by which the correlations between various features are taken into consideration. Besides, considering the fact that labeling a large amount of training data in real world is both time-consuming and tedious, we adopt manifold learning, which exploits both labeled and unlabeled training data for a feature space analysis. Since the objective function is nonsmooth and difficult to solve, we propose an iteractive algorithm with fast convergence. Extensive experiments on different applications demonstrate that our algorithm outperforms the other state-of-the-art feature selection algorithms.

A Semi-Supervised Feature Selection Approach for Fault Diagnostics in Evolving Environments

This paper introduces a Semi-Supervised Feature Selection (SSFS) approach for selecting the most suitable features for fault diagnostics in evolving environments. The effectiveness of the proposed SSFS approach is verified with respect to an application concerning the classification of the defect type of bearings in Fully Electric Vehicles operating at different loads. The results show that SSFS allows adapting the diagnostic model to the varying load by updating the set of features used for the classification and achieves more satisfactory diagnostic accuracy than the traditional diagnostic models. The proposed diagnostic approach can contribute significantly to the maintenance practice of components such as gearboxes, alternators, shafts and pumps, whose working conditions are usually characterized by evolving environment.

Unified discriminating feature analysis for visual category recognition

Visual category recognition (VCR) is one of the most important tasks in image and video indexing. To deal with high dimension image/video data, feature analysis algorithms have been widely used for visual category recognition. In this paper, to enhance the flexibility regarding the exploitation of labeled or unlabeled data, we propose a unified feature analysis framework that can be applied to both supervised and semi-supervised scenarios. Furthermore, by revealing intrinsic relationships of traditional feature analysis methods, our framework not only integrates traditional methods, but also introduces an ℓ2,1-norm regularization term for sparse learning. Extensive experiments report that the proposed method obtains advantageous performance in comparison with other state-of-the-art supervised and semi-supervised feature selection algorithms.

Semi-supervised class-specific feature selection for VHR remote sensing images

ABSTRACTFeatures relevant to a thematic class, that is, class-specific features are beneficial to thematic information extraction. However, existing class-specific feature selection methods require abundant labelled samples, while sample labelling is always labour intensive and time consuming. Therefore, it is necessary to select class-specific features with insufficient labelled objects. In this paper, we raise this problem as semi-supervised class-specific feature selection and propose a new two-stage method. First, a weight matrix fully integrates local geometrical structure and discriminative information. Second, the weight matrix is incorporated into a -norm minimization optimization problem of data reconstruction to objectively measure the effectiveness of features for a thematic class. Different from the explicit binarization in the label vector, the new method only implicitly employs binarization in the weight matrix. With area under receiver-operating characteristic curve, class-specific features result in an increase from 3% and 4% on average for Bayes and linear support vector machine, respectively.

Adaptive semisupervised feature selection without graph construction for very-high-resolution remote sensing images

Semisupervised feature selection methods can improve classification performance and enhance model comprehensibility with few labeled objects. However, most of the existing methods require graph construction beforehand, and the resulting heavy computational cost may bring about the failure to accurately capture the local geometry of data. To overcome the problem, adaptive semisupervised feature selection (ASFS) is proposed. In ASFS, the goodness of each feature is measured by linear objective functions based on loss functions and probability distribution matrices. By alternatively optimizing model parameters and automatically adjusting the probabilities of boundary objects, ASFS can measure the genuine characteristics of the data and then rank and select features. The experimental results attest to the effectiveness and practicality of the method in comparison with the latest and state-of-the-art methods on a Worldview II image and a Quickbird II image.

Semi-supervised feature selection for audio classification based on constraint compensated Laplacian score

Audio classification, classifying audio segments into broad categories such as speech, non-speech, and silence, is an important front-end problem in speech signal processing. Dozens of features have been proposed for audio classification. Unfortunately, these features are not directly complementary and combining them does not improve classification performance. Feature selection provides an effective mechanism for choosing the most relevant and least redundant features for classification. In this paper, we present a semi-supervised feature selection algorithm named Constraint Compensated Laplacian score (CCLS), which takes advantage of the local geometrical structure of unlabeled data as well as constraint information from labeled data. We apply this method to the audio classification task and compare it with other known feature selection methods. Experimental results demonstrate that CCLS gives substantial improvement.

Semisupervised Feature Selection with Universum

The Universum data, defined as a set of unlabeled examples that do not belong to any class of interest, have been shown to encode some prior knowledge by representing meaningful information in the same domain as the problem at hand. Universum data have been proved effective in improving learning performance in many tasks, such as classification and clustering. Inspired by its favorable performance, we address a novel semisupervised feature selection problem in this paper, called semisupervised feature selection with Universum, that can simultaneously exploit the unlabeled data and the Universum data. The experiments on several UCI data sets are presented to show that the proposed algorithms can achieve superior performances over conventional unsupervised and supervised methods.

Ensemble constrained Laplacian score for efficient and robust semi-supervised feature selection

In this paper, we propose an efficient and robust approach for semi-supervised feature selection, based on the constrained Laplacian score. The main drawback of this method is the choice of the sca...

Human Age Estimation Based on Locality and Ordinal Information

In this paper, we propose a novel feature selection-based method for facial age estimation. The face aging is a typical temporal process, and facial images should have certain ordinal patterns in the aging feature space. From the geometrical perspective, a facial image can be usually seen as sampled from a low-dimensional manifold embedded in the original high-dimensional feature space. Thus, we first measure the energy of each feature in preserving the underlying local structure information and the ordinal information of the facial images, respectively, and then we intend to learn a low-dimensional aging representation that can maximally preserve both kinds of information. To further improve the performance, we try to eliminate the redundant local information and ordinal information as much as possible by minimizing nonlinear correlation and rank correlation among features. Finally, we formulate all these issues into a unified optimization problem, which is similar to linear discriminant analysis in format. Since it is expensive to collect the labeled facial aging images in practice, we extend the proposed supervised method to a semi-supervised learning mode including the semi-supervised feature selection method and the semi-supervised age prediction algorithm. Extensive experiments are conducted on the FACES dataset, the Images of Groups dataset, and the FG-NET aging dataset to show the power of the proposed algorithms, compared to the state-of-the-arts.

Supervised, Unsupervised, and Semi-Supervised Feature Selection: A Review on Gene Selection

Recently, feature selection and dimensionality reduction have become fundamental tools for many data mining tasks, especially for processing high-dimensional data such as gene expression microarray data. Gene expression microarray data comprises up to hundreds of thousands of features with relatively small sample size. Because learning algorithms usually do not work well with this kind of data, a challenge to reduce the data dimensionality arises. A huge number of gene selection are applied to select a subset of relevant features for model construction and to seek for better cancer classification performance. This paper presents the basic taxonomy of feature selection, and also reviews the state-of-the-art gene selection methods by grouping the literatures into three categories: supervised, unsupervised, and semi-supervised. The comparison of experimental results on top 5 representative gene expression datasets indicates that the classification accuracy of unsupervised and semi-supervised feature selection is competitive with supervised feature selection.

Semi-supervised feature selection based on local discriminative information

Feature selection has been an effective way to reduce the dimensionality of the high dimensional data. In this paper, we propose a novel feature selection method which achieves batch feature selection using both supervised and unsupervised data samples. The objective function includes three parts: first, under the assumption that each data sample has been assigned a class label, the ratio of between class scatter matrix and total scatter matrix should be minimized, where the scatter matrices are formed by the selected features of these data samples; second, we use linear regression to model the correlations between the data samples with supervision information and their class labels; last, we use l2,1-norm to guarantee the sparsity of the feature selection matrix and exploit the sharing information between supervised and unsupervised data samples jointly. Different from existing methods, our approach exploits local discriminative information to construct the model, therefore we obtain better results from extensive experiments compared with the existing methods.

Compact representation for large-scale unconstrained video analysis

Recently, newly invented features (e.g. Fisher vector, VLAD) have achieved state-of-the-art performance in large-scale video analysis systems that aims to understand the contents in videos, such as concept recognition and event detection. However, these features are in high-dimensional representations, which remarkably increases computation costs and correspondingly deteriorates the performance of subsequent learning tasks. Notably, the situation becomes even worse when dealing with large-scale video data where the number of class labels are limited. To address this problem, we propose a novel algorithm to compactly represent huge amounts of unconstrained video data. Specifically, redundant feature dimensions are removed by using our proposed feature selection algorithm. Considering unlabeled videos that are easy to obtain on the web, we apply this feature selection algorithm in a semi-supervised framework coping with a shortage of class information. Different from most of the existing semi-supervised feature selection algorithms, our proposed algorithm does not rely on manifold approximation, i.e. graph Laplacian, which is quite expensive for a large number of data. Thus, it is possible to apply the proposed algorithm to a real large-scale video analysis system. Besides, due to the difficulty of solving the non-smooth objective function, we develop an efficient iterative approach to seeking the global optimum. Extensive experiments are conducted on several real-world video datasets, including KTH, CCV, and HMDB. The experimental results have demonstrated the effectiveness of the proposed algorithm.

Intrusion Detection System- Via Fuzzy Artmap in Addition with Advance Semi Supervised Feature Selection

Intrusion Detection System- Via Fuzzy Artmap in Addition with Advance Semi Supervised Feature Selection

Mutual-Information-Based Semi-Supervised Hyperspectral Band Selection With High Discrimination, High Information, and Low Redundancy

The large number of spectral bands in hyperspectral images provides abundant information to distinguish different land covers. However, these spectral bands have much redundancy and bring an extra computational burden. Thus, band selection is important for hyperspectral images. Since the labeled samples are difficult to obtain, a semi-supervised criterion based on maximum discrimination and information (MDI) is defined by using both limited labeled samples and sufficient unlabeled samples. This MDI criterion aims to select the most highly discriminative and informative bands, but it is hard to accurately calculate. Therefore, a novel criterion based on high discrimination, high information, and low redundancy (DIR) is proposed as its low-order approximation. Moreover, from an information theory perspective, a theoretical proof is given that many traditional semi-supervised feature selection criteria are the low-order approximations of this MDI criterion. Compared with them, the proposed criterion needs more relaxed approximation conditions. To search and optimize the proposed criterion, a novel clonal selection algorithm is proposed, where the adaptive clone and mutation operators are devised to speed up the convergence. Experimental results on hyperspectral images demonstrate the effectiveness of the proposed semi-supervised band selection method.

Emotion recognition using semi-supervised feature selection with speaker normalization

Feature selection methods are the mostly used dimensional reduction methods in speech emotion recognition. However, most methods cannot preserve the manifold of data and cannot use the information provided by unlabeled data, so that they cannot select a good sub feature set for speech emotion recognition. This paper presents a semi-supervised feature selection method that can preserve the manifold structure of data, preserve the category structure, and use the information provided by the unlabeled data. To further deal with the manifold of speech data influenced by factors such as emotion, speaker and sentence, a new speaker normalization method is also proposed, which can achieve a good speaker normalization result in the case of a small number of samples of a speaker available. This speaker normalization method can be used in most real application of speech emotion recognition. The conducted experiments validate the proposed semi-supervised feature selection method with the speaker normalization in terms of the effectiveness of the speech emotion recognition.

Semisupervised feature selection via spline regression for video semantic recognition.

To improve both the efficiency and accuracy of video semantic recognition, we can perform feature selection on the extracted video features to select a subset of features from the high-dimensional feature set for a compact and accurate video data representation. Provided the number of labeled videos is small, supervised feature selection could fail to identify the relevant features that are discriminative to target classes. In many applications, abundant unlabeled videos are easily accessible. This motivates us to develop semisupervised feature selection algorithms to better identify the relevant video features, which are discriminative to target classes by effectively exploiting the information underlying the huge amount of unlabeled video data. In this paper, we propose a framework of video semantic recognition by semisupervised feature selection via spline regression (S(2)FS(2)R) . Two scatter matrices are combined to capture both the discriminative information and the local geometry structure of labeled and unlabeled training videos: A within-class scatter matrix encoding discriminative information of labeled training videos and a spline scatter output from a local spline regression encoding data distribution. An l2,1 -norm is imposed as a regularization term on the transformation matrix to ensure it is sparse in rows, making it particularly suitable for feature selection. To efficiently solve S(2)FS(2)R , we develop an iterative algorithm and prove its convergency. In the experiments, three typical tasks of video semantic recognition, such as video concept detection, video classification, and human action recognition, are used to demonstrate that the proposed S(2)FS(2)R achieves better performance compared with the state-of-the-art methods.

Sketch-based human motion retrieval via selected 2D geometric posture descriptor

Sketch-based human motion retrieval is a hot topic in computer animation in recent years. In this paper, we present a novel sketch-based human motion retrieval method via selected 2-dimensional (2D) Geometric Posture Descriptor (2GPD). Specially, we firstly propose a rich 2D pose feature call 2D Geometric Posture Descriptor (2GPD), which is effective in encoding the 2D posture similarity by exploiting the geometric relationships among different human body parts. Since the original 2GPD is of high dimension and redundant, a semi-supervised feature selection algorithm derived from Laplacian Score is then adopted to select the most discriminative feature component of 2GPD as feature representation, and we call it as selected 2GPD. Finally, a posture-by-posture motion retrieval algorithm is used to retrieve a motion sequence by sketching several key postures. Experimental results on CMU human motion database demonstrate the effectiveness of our proposed approach.

Modified Constraint Scores for Semi-Supervised Feature Selection

Semi-supervised constraint scores, which utilize both pairwise constraints and the local property of the unlabeled data to select features, achieve comparable performance to the supervised feature selection methods. The local property is characterized without considering the pairwise constraints and these two conditions are introduced independently. However, the pairwise constraints and the local property may contain conflicting information. In this paper, we utilize the conflicting information to improve the local property. Instead of characterizing the local property by all neighbors, samples which do not appear in the cannot-link constraints can be used. A performance indicator, called neighborhood-cannotlink (NC) coefficient, is proposed to measure the improvement of the local property. We use the improved local property and the pairwise constraints to perform semi-supervised constraint scores algorithm. Experiments on several real world data sets demonstrate the effectiveness of the methods.

Feature selection and semi-supervised clustering using multiobjective optimization.

In this paper we have coupled feature selection problem with semi-supervised clustering. Semi-supervised clustering utilizes the information of unsupervised and supervised learning in order to overcome the problems related to them. But in general all the features present in the data set may not be important for clustering purpose. Thus appropriate selection of features from the set of all features is very much relevant from clustering point of view. In this paper we have solved the problem of automatic feature selection and semi-supervised clustering using multiobjective optimization. A recently created simulated annealing based multiobjective optimization technique titled archived multiobjective simulated annealing (AMOSA) is used as the underlying optimization technique. Here features and cluster centers are encoded in the form of a string. We assume that for each data set for 10% data points class level information are known to us. Two internal cluster validity indices reflecting different data properties, an external cluster validity index measuring the similarity between the obtained partitioning and the true labelling for 10% data points and a measure counting the number of features present in a particular string are optimized using the search capability of AMOSA. AMOSA is utilized to detect the appropriate subset of features, appropriate number of clusters as well as the appropriate partitioning from any given data set. The effectiveness of the proposed semi-supervised feature selection technique as compared to the existing techniques is shown for seven real-life data sets of varying complexities.

A Convex Formulation for Semi-Supervised Multi-Label Feature Selection

Explosive growth of multimedia data has brought challenge of how to efficiently browse, retrieve and organize these data. Under this circumstance, different approaches have been proposed to facilitate multimedia analysis. Several semi-supervised feature selection algorithms have been proposed to exploit both labeled and unlabeled data. However, they are implemented based on graphs, such that they cannot handle large-scale datasets. How to conduct semi-supervised feature selection on large-scale datasets has become a challenging research problem. Moreover, existing multi-label feature selection algorithms rely on eigen-decomposition with heavy computational burden, which further prevent current feature selection algorithms from being applied for big data. In this paper, we propose a novel convex semi-supervised multi-label feature selection algorithm, which can be applied to large-scale datasets. We evaluate performance of the proposed algorithm over five benchmark datasets and compare the results with state-of-the-art supervised and semi-supervised feature selection algorithms as well as baseline using all features. The experimental results demonstrate that our proposed algorithm consistently achieve superiors performances.