Filter Feature Selection Algorithms Research Articles

A Study of Feature Selection Algorithms for Predicting Students Academic Performance

The main aim of all the educational organizations is to improve the quality of education and elevate the academic performance of students. Educational Data Mining (EDM) is a growing research field which helps academic institutions to improve the performance of their students. The academic institutions are most often judged by the grades achieved by the students in examination. EDM offers different practices to predict the academic performance of students. In EDM, Feature Selection (FS) plays a vital role in improving the quality of prediction models for educational datasets. FS algorithms eliminate unrelated data from the educational repositories and hence increase the performance of classifier accuracy used in different EDM practices to support decision making for educational settings. The good quality of educational dataset can produce better results and hence the decisions based on such quality dataset can increase the quality of education by predicting the performance of students. In the light of this mentioned fact, it is necessary to choose a feature selection algorithm carefully. This paper presents an analysis of the performance of filter feature selection algorithms and classification algorithms on two different student datasets. The results obtained from different FS algorithms and classifiers on two student datasets with different number of features will also help researchers to find the best combinations of filter feature selection algorithms and classifiers. It is very necessary to put light on the relevancy of feature selection for student performance prediction, as the constructive educational strategies can be derived through the relevant set of features. The results of our study depict that there is a 10% difference of prediction accuracies between the results of datasets with different number of features.

Differential evolution for filter feature selection based on information theory and feature ranking

Feature selection is an essential step in various tasks, where filter feature selection algorithms are increasingly attractive due to their simplicity and fast speed. A common filter is to use mutual information to estimate the relationships between each feature and the class labels (mutual relevancy), and between each pair of features (mutual redundancy). This strategy has gained popularity resulting a variety of criteria based on mutual information. Other well-known strategies are to order each feature based on the nearest neighbor distance as in ReliefF, and based on the between-class variance and the within-class variance as in Fisher Score. However, each strategy comes with its own advantages and disadvantages. This paper proposes a new filter criterion inspired by the concepts of mutual information, ReliefF and Fisher Score. Instead of using mutual redundancy, the proposed criterion tries to choose the highest ranked features determined by ReliefF and Fisher Score while providing the mutual relevance between features and the class labels. Based on the proposed criterion, two new differential evolution (DE) based filter approaches are developed. While the former uses the proposed criterion as a single objective problem in a weighted manner, the latter considers the proposed criterion in a multi-objective design. Moreover, a well known mutual information feature selection approach (MIFS) based on maximum-relevance and minimum-redundancy is also adopted in single-objective and multi-objective DE algorithms for feature selection. The results show that the proposed criterion outperforms MIFS in both single objective and multi-objective DE frameworks. The results also indicate that considering feature selection as a multi-objective problem can generally provide better performance in terms of the feature subset size and the classification accuracy.

Enhanced decision tree ensembles for land-cover mapping from fully polarimetric SAR data

ABSTRACTFully polarimetric synthetic aperture radar (PolSAR) Earth Observations showed great potential for mapping and monitoring agro-environmental systems. Numerous polarimetric features can be extracted from these complex observations which may lead to improve accuracy of land-cover classification and object characterization. This article employed two well-known decision tree ensembles, i.e. bagged tree (BT) and random forest (RF), for land-cover mapping from PolSAR imagery. Moreover, two fast modified decision tree ensembles were proposed in this article, namely balanced filter-based forest (BFF) and cost-sensitive filter-based forest (CFF). These algorithms, designed based on the idea of RF, use a fast filter feature selection algorithms and two extended majority voting. They are also able to embed some solutions of imbalanced data problem into their structures. Three different PolSAR datasets, with imbalanced data, were used for evaluating efficiency of the proposed algorithms. The results indicated that all the tree ensembles have higher efficiency and reliability than the individual DT. Moreover, both proposed tree ensembles obtained higher mean overall accuracy (0.5–14% higher), producer’s accuracy (0.5–10% higher), and user’s accuracy (0.5–9% higher) than the classical tree ensembles, i.e. BT and RF. They were also much faster (e.g. 2–10 times) and more stable than their competitors for classification of these three datasets. In addition, unlike BT and RF, which obtained higher accuracy in large ensembles (i.e. the high number of DT), BFF and CFF can also be more efficient and reliable in smaller ensembles. Furthermore, the extended majority voting techniques could outperform the classical majority voting for decision fusion.

Feature Analysis to Human Activity Recognition

Human activity recognition (HAR) is one of those research areas whose importance and popularity have notably increased in recent years. HAR can be seen as a general machine learning problem which requires feature extraction and feature selection. In previous articles different features were extracted from time, frequency and wavelet domains for HAR but it is not clear that, how to determine the best feature combination which maximizes the performance of a machine learning algorithm. The aim of this paper is to present the most relevant feature extraction methods in HAR and to compare them with widely-used filter and wrapper feature selection algorithms. This work is an extended version of [1]a where we tested the efficiency of filter and wrapper feature selection algorithms in combination with artificial neural networks. In this paper the efficiency of selected features has been investigated on more machine learning algorithms (feed-forward artificial neural network, k-nearest neighbor and decision tree) where an independent database was the data source. The result demonstrates that machine learning in combination with feature selection can overcome other classification approaches.

Exploring the potential role of feature selection in global land-cover mapping

ABSTRACTGlobal land cover has been acknowledged as a fundamental variable in several global-scale studies for environment and climate change. Recent developments in global land-cover mapping focused on spatial resolution improvement with more heterogeneous features to integrate the spatial, spectral, and temporal information. Although the high dimensional input features as a whole lead to discriminatory strengths to produce more accurate land-cover maps, it comes at the cost of an increased classification complexity. The feature selection method has become a necessity for dimensionality reduction in classification with large amounts of input features. In this study, the potential of feature selection in global land-cover mapping is explored. A total of 63 features derived from the Landsat Thematic Mapper (TM) spectral bands, Moderate Resolution Imaging Spectroradiometer (MODIS) time series enhanced vegetation index (EVI) data, digital elevation model (DEM), and many climate-ecological variables and global training samples are input to k-nearest neighbours (k-NN) and Random Forest (RF) classifiers. Two filter feature selection algorithms, i.e. Relieff and max-min-associated (MNA), were employed to select the optimal subsets of features for the whole world and different biomes. The mapping accuracies with/without feature selection were evaluated by a global validation sample set. Overall, the result indicates no significant accuracy improvement in global land-cover mapping after dimensionality reduction. Nevertheless, feature selection has the capability of identifying useful features in different biomes and improves the computational efficiency, which is valuable in global-scale computing.

Efficient feature selection based on correlation measure between continuous and discrete features

Feature selection is frequently used to reduce the number of features in many applications where data of high dimensionality are involved. Lots of the feature selection methods mainly focus on measuring the correlation (or similarity) between two features. However, most correlation measures are limited to handling only certain types of data. Feature space consisting of continuous/discrete feature or their combination presents a severe challenge to feature selection in terms of efficiency and effectiveness. This paper introduces a novel approach that can measure the correlation between a continuous and a discrete feature, and then proposes an efficient filter feature selection algorithm based on correlation analysis by removing weakly relevant and irrelevant features, as well as relevant but redundant features. Both theoretical and experimental comparisons with other representative filter approaches on UCI datasets show that the proposed algorithm is effective for selecting continuous and discrete features, as well as the mixture of continuous and discrete features. The performance of ECMBF is superior to other approaches in terms of dimensionality reduction and classification error rate.

Test on existence of histology subtype-specific prognostic signatures among early stage lung adenocarcinoma and squamous cell carcinoma patients using a Cox-model based filter.

BackgroundNon-small cell lung cancer (NSCLC) is the predominant histological type of lung cancer, accounting for up to 85% of cases. Disease stage is commonly used to determine adjuvant treatment eligibility of NSCLC patients, however, it is an imprecise predictor of the prognosis of an individual patient. Currently, many researchers resort to microarray technology for identifying relevant genetic prognostic markers, with particular attention on trimming or extending a Cox regression model.Adenocarcinoma (AC) and squamous cell carcinoma (SCC) are two major histology subtypes of NSCLC. It has been demonstrated that fundamental differences exist in their underlying mechanisms, which motivated us to postulate the existence of specific genes related to the prognosis of each histology subtype.ResultsIn this article, we propose a simple filter feature selection algorithm with a Cox regression model as the base. Applying this method to real-world microarray data identifies a histology-specific prognostic gene signature. Furthermore, the resulting 32-gene (32/12 for AC/SCC) prognostic signature for early-stage AC and SCC samples has superior predictive ability relative to two relevant prognostic signatures, and has comparable performance with signatures obtained by applying two state-of-the art algorithms separately to AC and SCC samples.ConclusionsOur proposal is conceptually simple, and straightforward to implement. Furthermore, it can be easily adapted and applied to a range of other research settings.ReviewersThis article was reviewed by Leonid Hanin (nominated by Dr. Lev Klebanov), Limsoon Wong and Jun Yu.Electronic supplementary materialThe online version of this article (doi:10.1186/s13062-015-0051-z) contains supplementary material, which is available to authorized users.

Improving the ranking quality of medical image retrieval using a genetic feature selection method

In this paper, we take advantage of single-valued functions that evaluate rankings to develop a family of feature selection methods based on the genetic algorithm approach, tailored to improve the accuracy of content-based image retrieval systems. Experiments on three image datasets, comprising images of breast and lung nodules, showed that developing functions to evaluate the ranking quality allows improving retrieval performance. This approach produces significantly better results than those of other fitness function approaches, such as the traditional wrapper and than filter feature selection algorithms.

Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework

This correspondence presents a novel hybrid wrapper and filter feature selection algorithm for a classification problem using a memetic framework. It incorporates a filter ranking method in the traditional genetic algorithm to improve classification performance and accelerate the search in identifying the core feature subsets. Particularly, the method adds or deletes a feature from a candidate feature subset based on the univariate feature ranking information. This empirical study on commonly used data sets from the University of California, Irvine repository and microarray data sets shows that the proposed method outperforms existing methods in terms of classification accuracy, number of selected features, and computational efficiency. Furthermore, we investigate several major issues of memetic algorithm (MA) to identify a good balance between local search and genetic search so as to maximize search quality and efficiency in the hybrid filter and wrapper MA.