Efficient Feature Selection Method Research Articles

Fast multi-label feature selection based on information-theoretic feature ranking

Multi-label feature selection involves selecting important features from multi-label data sets. This can be achieved by ranking features based on their importance and then selecting the top-ranked features. Many multi-label feature selection methods for finding a feature subset that can improve multi-label learning accuracy have been proposed. In contrast, computationally efficient multi-label feature selection methods have not been studied extensively. In this study, we propose a fast multi-label feature selection method based on information-theoretic feature ranking. Experimental results demonstrate that the proposed method generates a feature subset significantly faster than several other multi-label feature selection methods for large multi-label data sets.

Chi-square Statistics Feature Selection Based on Term Frequency and Distribution for Text Categorization

ABSTRACTText categorization (TC) becomes the key technology to find relevant and timely information from a volume of digital documents, and feature selection techniques are proposed to overcome the high dimensionality which causes the high computational complexity and low accuracy in TC tasks. Chi-square statistics (CHI) is one of the most efficient feature selection methods; however, it has two weaknesses. (1) It is document frequency based, and only counts whether the term occurs or not. Actually, high-frequency term occurring in few documents is often regarded as a discriminator in corpus. (2) It does not consider the term distribution. A term has more discriminating power for a specific category when its difference in degree of distribution is lower. In this paper, we propose a modified CHI feature selection approach which is called term frequency and distribution based CHI to overcome these weaknesses. We use sample variance to calculate the term distribution, and improve the classic CHI with maximum term frequency. Extensive and comparative experiments on three corpora show that the proposed approach is comparable to the classic feature selection methods in terms of macro-F1 and micro-F1.

Efficient sequential feature selection based on adaptive eigenspace model

Though Fisher score is a representative and effective feature selection method, it has an unsolved drawback: it either evaluates the features individually and selects the top features, or selects features using the sequential search strategies. The individual-method ignores the mutual relationship among the selected features while the sequential-methods always suffer from heavy computation. In this work, we present an efficient sequential feature selection method. In the proposed method, the generalized Fisher score is used as a robust measurement of the discriminative ability of the features, which can naturally deal with the Small Size Sample problem. Besides, each feature is considered as a pattern vector and an adaptive eigenspace model is applied to update the generalized Fisher score. In the proposed adaptive eigenspace model, the size of the eigen-decomposition problems does not increase with the number of selected features, but is determined by the dimension of the adaptive eignespace. If the dimension of the adaptive eigenspace model is fixed, the proposed algorithm approximately consumes constant time to evaluate a candidate feature. Therefore, the proposed method is computationally more efficient than the traditional sequential methods. Experiments on six widely used face databases are conducted to demonstrate the efficacy of the proposed approach.

Distance Variance Score: An Efficient Feature Selection Method in Text Classification

With the rapid development of web applications such as social network, a large amount of electric text data is accumulated and available on the Internet, which causes increasing interests in text mining. Text classification is one of the most important subfields of text mining. In fact, text documents are often represented as a high-dimensional sparse document term matrix (DTM) before classification. Feature selection is essential and vital for text classification due to high dimensionality and sparsity of DTM. An efficient feature selection method is capable of both reducing dimensions of DTM and selecting discriminative features for text classification. Laplacian Score (LS) is one of the unsupervised feature selection methods and it has been successfully used in areas such as face recognition. However, LS is unable to select discriminative features for text classification and to effectively reduce the sparsity of DTM. To improve it, this paper proposes an unsupervised feature selection method named Distance Variance Score (DVS). DVS uses feature distance contribution (a ratio) to rank the importance of features for text documents so as to select discriminative features. Experimental results indicate that DVS is able to select discriminative features and reduce the sparsity of DTM. Thus, it is much more efficient than LS.

Burned Area Mapping Using Support Vector Machines and the FuzCoC Feature Selection Method on VHR IKONOS Imagery

The ever increasing need for accurate burned area mapping has led to a number of studies that focus on improving the mapping accuracy and effectiveness. In this work, we investigate the influence of derivative spectral and spatial features on accurately mapping recently burned areas using VHR IKONOS imagery. Our analysis considers both pixel and object-based approaches, using two advanced image analysis techniques: (a) an efficient feature selection method based on the Fuzzy Complementary Criterion (FuzCoC) and (b) the Support Vector Machine (SVM) classifier. In both cases (pixel and object-based), a number of higher-order spectral and spatial features were produced from the original image. The proposed methodology was tested in areas of Greece recently affected by severe forest fires, namely, Parnitha and Rhodes. The extensive comparative analysis indicates that the SVM object-based scheme exhibits higher classification accuracy than the respective pixel-based one. Additionally, the accuracy increased with the addition of derivative features and subsequent implementation of the FuzCoC feature selection (FS) method. Apart from the positive effect in the classification accuracy, the application of the FuzCoC FS method significantly reduces the computational requirements and facilitates the manipulation of the large data volume. In both cases (pixel and objet) the results confirmed that the use of an efficient feature selection method is a prerequisite step when extra information through higher-order features is added to the classification process of VHR imagery for burned area mapping.

Mutual Information-based multi-label feature selection using interaction information

Multi-label feature selection is regarded as one of the most promising techniques that can be used to maximize the efficacy and efficiency of multi-label classification. However, because multi-label feature selection algorithms must consider multiple labels concurrently, the task is more difficult than single-label feature selection tasks. In this paper, we propose the Mutual Information-based multi-label feature selection method using interaction information. This method is naturally able to measure dependencies among multiple variables. To develop an efficient multi-label feature selection method, we derive theoretical bounds for the interaction information. Empirical studies indicate that our proposed multi-label feature selection method discovers effective feature subsets for multi-label classification problems.

HYBRID FEATURE SELECTION ALGORITHM FOR INTRUSION DETECTION SYSTEM

Network security is a serious global concern. Usefulness Intrusion Detection Systems (IDS) are increasing incredibly in Information Security research using Soft computing techniques. In the previous researches having irrelevant and redundant features are recognized causes of increasing the processing speed of evaluating the known intrusive patterns. In addition, an efficient feature selection method eliminates dimension of data and reduce redundancy and ambiguity caused by none important attributes. Therefore, feature selection methods are well-known methods to overcome this problem. There are various approaches being utilized in intrusion detections, they are able to perform their method and relatively they are achieved with some improvements. This work is based on the enhancement of the highest Detection Rate (DR) algorithm which is Linear Genetic Programming (LGP) reducing the False Alarm Rate (FAR) incorporates with Bees Algorithm. Finally, Support Vector Machine (SVM) is one of the best candidate solutions to settle IDSs problems. In this study four sample dataset containing 4000 random records are excluded randomly from this dataset for training and testing purposes. Experimental results show that the LGP_BA method improves the accuracy and efficiency compared with the previous related research and the feature subcategory offered by LGP_BA gives a superior representation of data.

Activity Recognition on Smartphones via Sensor-Fusion and KDA-Based SVMs

Although human activity recognition (HAR) has been studied extensively in the past decade, HAR on smartphones is a relatively new area. Smartphones are equipped with a variety of sensors. Fusing the data of these sensors could enable applications to recognize a large number of activities. Realizing this goal is challenging, however. Firstly, these devices are low on resources, which limits the number of sensors that can be utilized. Secondly, to achieve optimum performance efficient feature extraction, feature selection and classification methods are required. This work implements a smartphone-based HAR scheme in accordance with these requirements. Time domain features are extracted from only three smartphone sensors, and a nonlinear discriminatory approach is employed to recognize 15 activities with a high accuracy. This approach not only selects the most relevant features from each sensor for each activity but it also takes into account the differences resulting from carrying a phone at different positions. Evaluations are performed in both offline and online settings. Our comparison results show that the proposed system outperforms some previous mobile phone-based HAR systems.

A new hybrid pattern recognition scheme for automatic discrimination of power quality disturbances

In this paper an intelligent method for automatic detection of Power Quality Disturbances (PQDs) is presented. The proposed automatic scheme is noticed to retaining informative features or eliminating redundant features simultaneously. This paper presents an effective method, for extracting features, so-called “integrated approach”, using integration of Discrete Wavelet Transform and Hyperbolic S Transform. Moreover, a new efficient feature selection method namely Orthogonal Forward Selection by incorporating Gram Schmidt procedure and forward selection is applied for selection of the best subset features. Some different classifiers are empirically compared in order to determine the best classifier. In this automatic scheme, the variable parameters of classifiers are optimized using a powerful method namely Particle Swarm Optimization. The sensitivity of the proposed method under noisy conditions has been investigated. The average rate of correct classification using the proposed hybrid scheme for automatic discrimination of PQDs in various noise conditions 99.55% is obtained.

MutualCascade method for pedestrian detection

An effective and efficient feature selection method based on Gentle Adaboost (GAB) cascade and the Four Direction Feature (FDF), namely, MutualCascade, which can be applied to the pedestrian detection problem in a single image, is proposed in this paper. MutualCascade improves the classic method of cascade to remove irrelevant and redundant features. The mutual correlation coefficient is utilized as a criterion to determine whether a feature should be chosen or not. Experimental results show that the MutualCascade method is more efficient and effective than Voila and Jones’ cascade and some other Adaboost-based method, and is comparable with HOG-based methods. It also demonstrates a higher performance compared with the state-of-the-art methods.

Power system voltage stability monitoring using artificial neural networks with a reduced set of inputs

This paper presents an artificial neural network (ANN)-based approach for online monitoring of a voltage stability margin (VSM) in electric power systems. The VSM is calculated by estimating the distance from the current operation state to the maximum voltage stability limit point according to the system loading parameter. Using the Gram–Schmidt orthogonalization process along with an ANN-based sensitivity technique, an efficient feature selection method is proposed to find the fewest input variables required to approximate the VSM with sufficient accuracy and high execution speed. Many algorithms have already been proposed in the literature for voltage stability assessment (VSA) using neural networks; however, the main drawback of the previously published works is that they need to train a new neural network when a change in the power system topology (configuration) occurs. Therefore, the possibility of employing a single ANN for estimating the VSM for several system configurations is investigated in this paper. The effectiveness of the proposed method is tested on the dynamic models of the New England 39-bus and the southern/eastern (SE) Australian power systems. The results obtained indicate that the proposed scheme provides a compact and efficient ANN model that can successfully and accurately estimate the VSM considering different system configurations as well as operating conditions, employing the fewest possible input features.

An Efficient Feature Selection Method for Arabic Text Classification

This paper proposes an efficient, Chi-Square-based, feature selection method for Arabic text classification. In Data Mining, feature selection is a preprocessing step that can improve the classification performance. Although few works have studied the effect of feature selection methods on Arabic text classification, limited number of methods was compared. Furthermore, different datasets were used by different works. This paper improves the previous works in three aspects. First, it proposes a new efficient feature selection method for enhancing Arabic text classification. Second, it compares extended number of existing feature selection methods. Third, it adopts two publicly available datasets to encourage future works to adopt them in order to guarantee fair comparisons among the various works. Our experiments show that our proposed method outperformed the existing methods in term of accuracy.

Feature selection via maximizing global information gain for text classification

Feature selection is a vital preprocessing step for text classification task used to solve the curse of dimensionality problem. Most existing metrics (such as information gain) only evaluate features individually but completely ignore the redundancy between them. This can decrease the overall discriminative power because one feature’s predictive power is weakened by others. On the other hand, though all higher order algorithms (such as mRMR) take redundancy into account, the high computational complexity renders them improper in the text domain. This paper proposes a novel metric called global information gain (GIG) which can avoid redundancy naturally. An efficient feature selection method called maximizing global information gain (MGIG) is also given. We compare MGIG with four other algorithms on six datasets, the experimental results show that MGIG has better results than others methods in most cases. Moreover, MGIG runs significantly faster than the traditional higher order algorithms, which makes it a proper choice for feature selection in text domain.

Efficient Feature Selection Methods in Chinese Spam Filtering

Efficient Feature Selection Methods in Chinese Spam Filtering

Unsupervised Optimal Discriminant Vector Based Feature Selection Method

An efficient unsupervised feature selection method based on unsupervised optimal discriminant vector is developed to find the important features without using class labels. Features are ranked according to the feature importance measurement based on unsupervised optimal discriminant vector in the following steps. First, fuzzy Fisher criterion is adopted as objective function to derive the optimal discriminant vector in unsupervised pattern. Second, the feature importance measurement based on elements of unsupervised optimal discriminant vector is defined to determine the importance of each feature. The features with little importance measurement are removed from the feature subset. Experiments on UCI dataset and fault diagnosis are carried out to show that the proposed method is very efficient and able to deliver reliable results.

A combinatorial cooperative-tabu search feature reduction approach

Presenting an efficient general feature selection method for the problem of the curse of dimensionality is still an open problem in pattern recognition, and, considering the cooperation among features through search processes, it is the most important challenge. In this paper, a combinatorial approach has been proposed, which consists of three feature reduction algorithms that have been applied in a parallel manner to cooperate. We consider each of these algorithms as a component in a reduction framework. For each component, among all various attribute selection algorithms, the Tabu Search (TS) a useful and state of the art algorithm, is used. To take account of the interaction between features, more subsets should be examined. Hence, each component should explore individually through feature space in a local area which is different from other components. The proposed algorithm, called the Cooperative-Tabu-Search (CTS), and also a revised version of this new method, is introduced to accelerate the convergence. After sufficient iterations, which satisfy the objective function; the final subset has been selected by voting between three reduction phases, and the data is then transformed into the new space, where the data are classified with some commonly used classifiers, such as Nearest Neighbor (NN) and Support Vector Machine (SVM). The employed benchmark of this paper is chosen among the UCI datasets to evaluate the proposed method compared to others. The experimental results show the supremacy of the accuracy of the implemented combinatorial approach in comparison with traditional methods.

RFS: Efficient feature selection method based on R-value

Feature selection is one of the most important issues in classification. Many filter and wrapper methods have been proposed. Here, we propose a new efficient feature selection method based on the R-value, which is a measure that is used to capture the overlapped areas among classes in a feature. Our strategy was to select features that have low overlapping areas among classes. Proposed idea is simple, but powerful for feature selection. The experiment results showed that the proposed method is better than previous typical methods in many cases. Accordingly, the proposed method can be used in combination with other feature selection methods.

Modified particle swarm optimization method for variable selection in QSAR/QSPR studies

The selection of the most relevant variables is an important step in the QSAR/QSPR modeling process. In this work we apply modified particle swarm optimization (MPSO) based on multiple linear regression (MLR) for selecting a small subset of descriptors that has significant contribution to the Gibbs energy of formation for a diverse set of organic compounds. Nonlinear relationships between selected molecular descriptors and Gibbs energy of formation are achieved by radial basis function neural network (RBF NN), adaptive neuro-fuzzy inference system (ANFIS), and support vector machine (SVM) methods. The MLR, RBF NN, ANFIS, and SVM squared correlation coefficients are 0.928, 0.946, 0.945, and 0.947, respectively. The obtained results suggest that the proposed MPSO is an efficient and powerful method for feature selection (descriptor selection) in the QSAR/QSPR studies.

An efficient face recognition system based on embedded DCT pyramid

In this paper, an efficient feature selection method based on a combination of DCT pyramid for image decomposition and the concept of the set partitioning in hierarchal trees (SPIHT) for structuring of information for face recognition is presented. In the proposed method, the DCT pyramid decomposes each face image into an approximation subband and a set of reversed L-shape blocks containing the high frequency coefficients. The generalized parent-child relationships of SPIHT algorithm are then established among the DCT pyramids. This leads to efficient selection of the most important coefficients among the layers of the DCT pyramid. Experimental results on the standard ORL and FERET databases show that the proposed method achieves more accurate face recognition than the wavelet-based SPIHT feature selection. Moreover, it outperforms the other well-known methods such as the Eigenfaces and the block-based DCT with the zigzag scanning structure in terms of both accuracy and memory requirement.

An efficient feature selection method for mobile devices with application to activity recognition

This paper presents a feature selection method for data classification, which combines a model-based variable selection technique and a fast two-stage subset selection algorithm. The relationship between a specified (and complete) set of candidate features and the class label is modeled using a non-linear full regression model which is linear-in-the-parameters. The performance of a sub-model measured by the sum of the squared-errors (SSE) is used to score the informativeness of the subset of features involved in the sub-model. The two-stage subset selection algorithm approaches a solution sub-model with the SSE being locally minimized. The features involved in the solution sub-model are selected as inputs to support vector machines (SVMs) for classification. The memory requirement of this algorithm is independent of the number of training patterns. This property makes this method suitable for applications executed in mobile devices where physical RAM memory is very limited. An application was developed for activity recognition, which implements the proposed feature selection algorithm and an SVM training procedure. Experiments are carried out with the application running on a PDA for human activity recognition using accelerometer data. A comparison with an information gain-based feature selection method demonstrates the effectiveness and efficiency of the proposed algorithm.