Conduct Feature Selection Research Articles

Online Group Feature Selection from Feature Streams

Standard feature selection algorithms deal with given candidate feature sets at the individual feature level. When features exhibit certain group structures, it is beneficial to conduct feature selection in a grouped manner. For high-dimensional features, it could be far more preferable to online generate and process features one at a time rather than wait for generating all features before learning begins. In this paper, we discuss a new and interesting problem of online group feature selection from feature streams at both the group and individual feature levels simultaneously from a feature stream. Extensive experiments on both real-world and synthetic datasets demonstrate the superiority of the proposed algorithm.

FSMRank: feature selection algorithm for learning to rank.

In recent years, there has been growing interest in learning to rank. The introduction of feature selection into different learning problems has been proven effective. These facts motivate us to investigate the problem of feature selection for learning to rank. We propose a joint convex optimization formulation which minimizes ranking errors while simultaneously conducting feature selection. This optimization formulation provides a flexible framework in which we can easily incorporate various importance measures and similarity measures of the features. To solve this optimization problem, we use the Nesterov's approach to derive an accelerated gradient algorithm with a fast convergence rate O(1/T(2)). We further develop a generalization bound for the proposed optimization problem using the Rademacher complexities. Extensive experimental evaluations are conducted on the public LETOR benchmark datasets. The results demonstrate that the proposed method shows: 1) significant ranking performance gain compared to several feature selection baselines for ranking, and 2) very competitive performance compared to several state-of-the-art learning-to-rank algorithms.

Unsupervised feature selection method based on latent Dirichlet allocation model and mutual information

To solve the category-deficiency and the tendency of selecting low-frequency words in feature selection process based on Mutual Information(MI),the method named LDA-σ was presented.Firstly,the latent topics were extracted by the Latent Dirichlet Allocation(LDA) model,and then the standard deviation of "Word-Topic" MI was calculated as the feature evaluation function.When conducting feature selection and categorization in Reuters-21578,the micro average F1 of LDA-σ reached up to 0.909 6,and the highest macro average F1 of LDA-σ was 0.782 3,which were higher than that of other algorithms.The experimental results indicate that LDA-σ can be applied to feature selection in text sets.

Discriminating Joint Feature Analysis for Multimedia Data Understanding

In this paper, we propose a novel semi-supervised feature analyzing framework for multimedia data understanding and apply it to three different applications: image annotation, video concept detection and 3-D motion data analysis. Our method is built upon two advancements of the state of the art: (1) <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2, 1</sub> -norm regularized feature selection which can jointly select the most relevant features from all the data points. This feature selection approach was shown to be robust and efficient in literature as it considers the correlation between different features jointly when conducting feature selection; (2) manifold learning which analyzes the feature space by exploiting both labeled and unlabeled data. It is a widely used technique to extend many algorithms to semi-supervised scenarios for its capability of leveraging the manifold structure of multimedia data. The proposed method is able to learn a classifier for different applications by selecting the discriminating features closely related to the semantic concepts. The objective function of our method is non-smooth and difficult to solve, so we design an efficient iterative algorithm with fast convergence, thus making it applicable to practical applications. Extensive experiments on image annotation, video concept detection and 3-D motion data analysis are performed on different real-world data sets to demonstrate the effectiveness of our algorithm.

On the discriminability of keystroke feature vectors used in fixed text keystroke authentication

Heterogeneous and aggregate vectors are the two widely used feature vectors in fixed text keystroke authentication. In this paper, we address the question “ Which vectors, heterogeneous, aggregate, or a combination of both, are more discriminative and why?” We accomplish this in three ways – (1) by providing an intuitive example to illustrate how aggregation of features inherently reduces discriminability; (2) by formulating “discriminability” as a non-parametric estimate of Bhattacharya distance, we show theoretically that the discriminability of a heterogeneous vector is higher than an aggregate vector; and (3) by conducting user recognition experiments using a dataset containing keystrokes from 33 users typing a 32-character reference text, we empirically validate our theoretical analysis. To compare the discriminability of heterogeneous and aggregate vectors with different combinations of keystroke features, we conduct feature selection analysis using three methods: (1) ReliefF, (2) correlation based feature selection, and (3) consistency based feature selection. Results of feature selection analysis reinforce the findings of our theoretical analysis.

Wavelet Feature Selection for Image Classification

Energy distribution over wavelet subbands is a widely used feature for wavelet packet based texture classification. Due to the overcomplete nature of the wavelet packet decomposition, feature selection is usually applied for a better classification accuracy and a compact feature representation. The majority of wavelet feature selection algorithms conduct feature selection based on the evaluation of each subband separately, which implicitly assumes that the wavelet features from different subbands are independent. In this paper, the dependence between features from different subbands is investigated theoretically and simulated for a given image model. Based on the analysis and simulation, a wavelet feature selection algorithm based on statistical dependence is proposed. This algorithm is further improved by combining the dependence between wavelet feature and the evaluation of individual feature component. Experimental results show the effectiveness of the proposed algorithms in incorporating dependence into wavelet feature selection.

Scoring and Selecting Terms for Text Categorization

We propose a set of (machine learning) ML-based scoring measures for conducting feature selection. We've tested these measures on documents from two well-known corpora, comparing them with other measures previously applied for this purpose. In particular, we've analyzed which measure obtains the best overall classification performance in terms of properties such as precision and recall, emphasizing to what extent some statistical properties of the corpus affects performance. The results show that some of our measures outperform the traditional measures in certain situations.

Fingertip force histories from multiple keys during typing

Heterogeneous and aggregate vectors are the two widely used feature vectors in fixed text keystroke authentication. In this paper, we address the question “Which vectors, heterogeneous, aggregate, or a combination of both, are more discriminative and why?” We accomplish this in three ways – (1) by providing an intuitive example to illustrate how aggregation of features inherently reduces discriminability; (2) by formulating “discriminability” as a non-parametric estimate of Bhattacharya distance, we show theoretically that the discriminability of a heterogeneous vector is higher than an aggregate vector; and (3) by conducting user recognition experiments using a dataset containing keystrokes from 33 users typing a 32-character reference text, we empirically validate our theoretical analysis. To compare the discriminability of heterogeneous and aggregate vectors with different combinations of keystroke features, we conduct feature selection analysis using three methods: (1) ReliefF, (2) correlation based feature selection, and (3) consistency based feature selection. Results of feature selection analysis reinforce the findings of our theoretical analysis.