Effective Feature Selection Method Research Articles

An exact feature selection algorithm based on rough set theory

Feature reduction based on rough set theory is an effective feature selection method in pattern recognition applications. Finding a minimal subset of the original features is inherent in rough set approach to feature selection. As feature reduction is a Nondeterministic Polynomial‐time‐hard problem, it is necessary to develop fast optimal or near‐optimal feature selection algorithms. This article aims to propose an exact feature selection algorithm in rough set that is efficient in terms of computation time. The proposed algorithm begins the examination of a solution tree by a breadth‐first strategy. The pruned nodes are held in a version of the trie data structure. Based on the monotonic property of dependency degree, all subsets of the pruned nodes cannot be optimal solutions. Thus, by detecting these subsets in trie, it is not necessary to calculate their dependency degree. The search on the tree continues until the optimal solution is found. This algorithm is improved by selecting an initial search level determined by the hill‐climbing method instead of searching the tree from the level below the root. The length of the minimal reduct and the size of data set can influence which starting search level is more efficient. The experimental results using some of the standard UCI data sets, demonstrate that the proposed algorithm is effective and efficient for data sets with more than 30 features. © 2014 Wiley Periodicals, Inc. Complexity 20: 50–62, 2015

MutualCascade method for pedestrian detection

An effective and efficient feature selection method based on Gentle Adaboost (GAB) cascade and the Four Direction Feature (FDF), namely, MutualCascade, which can be applied to the pedestrian detection problem in a single image, is proposed in this paper. MutualCascade improves the classic method of cascade to remove irrelevant and redundant features. The mutual correlation coefficient is utilized as a criterion to determine whether a feature should be chosen or not. Experimental results show that the MutualCascade method is more efficient and effective than Voila and Jones’ cascade and some other Adaboost-based method, and is comparable with HOG-based methods. It also demonstrates a higher performance compared with the state-of-the-art methods.

Derivative component analysis for mass spectral serum proteomic profiles.

BackgroundAs a promising way to transform medicine, mass spectrometry based proteomics technologies have seen a great progress in identifying disease biomarkers for clinical diagnosis and prognosis. However, there is a lack of effective feature selection methods that are able to capture essential data behaviors to achieve clinical level disease diagnosis. Moreover, it faces a challenge from data reproducibility, which means that no two independent studies have been found to produce same proteomic patterns. Such reproducibility issue causes the identified biomarker patterns to lose repeatability and prevents it from real clinical usage.MethodsIn this work, we propose a novel machine-learning algorithm: derivative component analysis (DCA) for high-dimensional mass spectral proteomic profiles. As an implicit feature selection algorithm, derivative component analysis examines input proteomics data in a multi-resolution approach by seeking its derivatives to capture latent data characteristics and conduct de-noising. We further demonstrate DCA's advantages in disease diagnosis by viewing input proteomics data as a profile biomarker via integrating it with support vector machines to tackle the reproducibility issue, besides comparing it with state-of-the-art peers.ResultsOur results show that high-dimensional proteomics data are actually linearly separable under proposed derivative component analysis (DCA). As a novel multi-resolution feature selection algorithm, DCA not only overcomes the weakness of the traditional methods in subtle data behavior discovery, but also suggests an effective resolution to overcoming proteomics data's reproducibility problem and provides new techniques and insights in translational bioinformatics and machine learning. The DCA-based profile biomarker diagnosis makes clinical level diagnostic performances reproducible across different proteomic data, which is more robust and systematic than the existing biomarker discovery based diagnosis.ConclusionsOur findings demonstrate the feasibility and power of the proposed DCA-based profile biomarker diagnosis in achieving high sensitivity and conquering the data reproducibility issue in serum proteomics. Furthermore, our proposed derivative component analysis suggests the subtle data characteristics gleaning and de-noising are essential in separating true signals from red herrings for high-dimensional proteomic profiles, which can be more important than the conventional feature selection or dimension reduction. In particular, our profile biomarker diagnosis can be generalized to other omics data for derivative component analysis (DCA)'s nature of generic data analysis.

Robust feature selection method in high-dimensional data mining

According to the feature of high-dimensional data,the number of variables is usually larger than the sample size and the data are often heterogeneous,a robust and effective feature selection method was proposed by using the dimensional reduction technique of variable selection and the modal regression based estimation method.The estimation algorithm was given by using Local Quadratic Algorithm(LQA) and Expectation-Maximum(EM) algorithm,and the selection method of the parameter adjustment was also discussed.Data analysis of the simulation shows that the proposed method is overall better than the least square and median regression based regularized method.Compared with the existing methods,the proposed method has higher prediction ability and stronger robustness especially for the non-normal error distribution.

Exploring the Contribution of Unlabeled Data in Financial Sentiment Analysis

With the proliferation of its applications in various industries, sentiment analysis by using publicly available web data has become an active research area in text classification during these years. It is argued by researchers that semi-supervised learning is an effective approach to this problem since it is capable to mitigate the manual labeling effort which is usually expensive and time-consuming. However, there was a long-term debate on the effectiveness of unlabeled data in text classification. This was partially caused by the fact that many assumptions in theoretic analysis often do not hold in practice. We argue that this problem may be further understood by adding an additional dimension in the experiment. This allows us to address this problem in the perspective of bias and variance in a broader view. We show that the well-known performance degradation issue caused by unlabeled data can be reproduced as a subset of the whole scenario. We argue that if the bias-variance trade-off is to be better balanced by a more effective feature selection method unlabeled data is very likely to boost the classification performance. We then propose a feature selection framework in which labeled and unlabeled training samples are both considered. We discuss its potential in achieving such a balance. Besides, the application in financial sentiment analysis is chosen because it not only exemplifies an important application, the data possesses better illustrative power as well. The implications of this study in text classification and financial sentiment analysis are both discussed.

A Novel Approach for Lie Detection Based on F-Score and Extreme Learning Machine

A new machine learning method referred to as F-score_ELM was proposed to classify the lying and truth-telling using the electroencephalogram (EEG) signals from 28 guilty and innocent subjects. Thirty-one features were extracted from the probe responses from these subjects. Then, a recently-developed classifier called extreme learning machine (ELM) was combined with F-score, a simple but effective feature selection method, to jointly optimize the number of the hidden nodes of ELM and the feature subset by a grid-searching training procedure. The method was compared to two classification models combining principal component analysis with back-propagation network and support vector machine classifiers. We thoroughly assessed the performance of these classification models including the training and testing time, sensitivity and specificity from the training and testing sets, as well as network size. The experimental results showed that the number of the hidden nodes can be effectively optimized by the proposed method. Also, F-score_ELM obtained the best classification accuracy and required the shortest training and testing time.

COMBINATION OF MULTIPLE FEATURE SELECTION METHODS FOR TEXT CATEGORIZATION BY USING COMBINATORIAL FUSION ANALYSIS AND RANK-SCORE CHARACTERISTIC

Effective feature selection methods are important for improving the efficiency and accuracy of text categorization algorithms by removing redundant and irrelevant terms from the corpus. Extensive research has been done to improve the performance of individual feature selection methods. However, it is always a challenge to come up with an individual feature selection method which would outperform other methods in most cases. In this paper, we explore the possibility of improving the overall performance by combining multiple individual feature selection methods. In particular, we propose a method of combining multiple feature selection methods by using an information fusion paradigm, called Combinatorial Fusion Analysis (CFA). A rank-score function and its associated graph, called rank-score graph, are adopted to measure the diversity of different feature selection methods. Our experimental results demonstrated that a combination of multiple feature selection methods can outperform a single method only if each individual feature selection method has unique scoring behavior and relatively high performance. Moreover, it is shown that the rank-score function and rank-score graph are useful for the selection of a combination of feature selection methods.

Multi sensor intelligent data fusion algorithm for principal component analysis

By combining Neural Networks (NN) with Principal Component Analysis (PCA), an algorithm for Multi-Sensor Intelligent Data Fusion (MSIDF) with an effective feature selection method is proposed. When the sensor vector dimension is high, it reduces the dimension by using PCA decomposition to is calculated the simplify data. Then the smallest error rate upper bound is tested by the iterative algorithm of neural networks data fusion. The proposed algorithm in sampling real time ship’s sensor data in experiments and gives a better performance. The proposed method can improve the fusion rate effectively and reduce the PCA computation time. The discussed algorithm is very useful in practice.

BFSSGA: Enhancing the Performance of Genetic Algorithm using Boosted Filtering Approach

Modern microarray chips can hold gene information from thousands of genes and hundreds of individuals and the main challenge of an effective feature selection method is to identify most useful genes from the whole dataset. Removal of less informative genes helps to alleviate the effects of noise and redundancy and simplifies the task of disease classification and prediction of medical conditions such as cancer. Genetic Algorithm (GA) based wrapper model performs well but suffers from over-fitting problem and the initial population is large and random. Traditional approaches use a filter based preprocessing step to reduce the dimension of the data on which GA operates and as filtering methods on its own has shown to introduce redundant features, in this paper Boosted Feature Subset Selection (BFSS) which is a boosted t-score filter method, is used as a preprocessing step. The gene subset provided by BFSS is fed to a Genetic Algorithm which reduces the feature subset in smaller numbers and helps to generate a better optimal subset of genes. The proposed hybrid approach is applied on leukemia, colon and lung cancer benchmarked datasets and have shown better results than other well-known approaches. General Terms Pattern Recognition, Feature Selection, Microarray Data Analysis, Evolutionary Algorithm.

Presenting a new search strategy to select synchronization values for classifying bipolar mood disorders from schizophrenic patients

There is a growing interest to employ synchronization methods to reveal natural connections among the brain lobes by measuring co-activations among EEG channels. Regarding high number of EEG channels, lots of synchronization indexes are determined between two by two channels leading to construct a high dimensional feature vector for each time frame. The objective of this paper is to propose an effective feature selection method to find discriminative synchronization indexes in order to classify patients with schizophrenia from those with bipolar mood disorder (BMD). The state-of-art synchronization methods from various domains such as phase-locking value (PLV), robust synchronization (RS), and synchronization likelihood (SL), were implemented to provide a rich feature set in order to classify the two groups. To increase the classification accuracy, a capable feature selection scheme is proposed entitled greedy overall relevancy (GOR) to select discriminative synchronization indexes. The elicited synchronization vectors of 53 subjects imposed to support vector machine (SVM) classifier and the classification result with and without applying GOR, provided 92.45% and 88.68% accuracy, respectively. Across-group variance (AGV) is chosen as a rival method to GOR; the selected features by AGV entered to the classifier resulting in 91.34% accuracy. Using pair T-test exhibits the significant superiority of GOR to AGV such that P-value determined less than 0.05. To the best of authors' knowledge, this is the first attempt to utilize the selected synchronization indexes as informative features applying to a classifier for diagnosing the psychiatric patients.

Prediction of Lysine Ubiquitylation with Ensemble Classifier and Feature Selection

Ubiquitylation is an important process of post-translational modification. Correct identification of protein lysine ubiquitylation sites is of fundamental importance to understand the molecular mechanism of lysine ubiquitylation in biological systems. This paper develops a novel computational method to effectively identify the lysine ubiquitylation sites based on the ensemble approach. In the proposed method, 468 ubiquitylation sites from 323 proteins retrieved from the Swiss-Prot database were encoded into feature vectors by using four kinds of protein sequences information. An effective feature selection method was then applied to extract informative feature subsets. After different feature subsets were obtained by setting different starting points in the search procedure, they were used to train multiple random forests classifiers and then aggregated into a consensus classifier by majority voting. Evaluated by jackknife tests and independent tests respectively, the accuracy of the proposed predictor reached 76.82% for the training dataset and 79.16% for the test dataset, indicating that this predictor is a useful tool to predict lysine ubiquitylation sites. Furthermore, site-specific feature analysis was performed and it was shown that ubiquitylation is intimately correlated with the features of its surrounding sites in addition to features derived from the lysine site itself. The feature selection method is available upon request.

Rough set and scatter search metaheuristic based feature selection for credit scoring

As the credit industry has been growing rapidly, credit scoring models have been widely used by the financial industry during this time to improve cash flow and credit collections. However, a large amount of redundant information and features are involved in the credit dataset, which leads to lower accuracy and higher complexity of the credit scoring model. So, effective feature selection methods are necessary for credit dataset with huge number of features. In this paper, a novel approach, called RSFS, to feature selection based on rough set and scatter search is proposed. In RSFS, conditional entropy is regarded as the heuristic to search the optimal solutions. Two credit datasets in UCI database are selected to demonstrate the competitive performance of RSFS consisted in three credit models including neural network model, J48 decision tree and Logistic regression. The experimental result shows that RSFS has a superior performance in saving the computational costs and improving classification accuracy compared with the base classification methods.

A feature selection method based on improved fisher’s discriminant ratio for text sentiment classification

Owing to its openness, virtualization and sharing criterion, the Internet has been rapidly becoming a platform for people to express their opinion, attitude, feeling and emotion. As the subjectivity texts are often too many for people to go through, how to automatically classify them into different sentiment orientation categories (e.g. positive/negative) has become an important research problem. In this paper, based on Fisher’s discriminant ratio, an effective feature selection method is proposed for subjectivity text sentiment classification. In order to validate the proposed method, we compared it with the method based on Information Gain while Support Vector Machine is adopted as the classifier. Two experiments are conducted by combining different feature selection methods with two kinds of candidate feature sets. Under 2739 subjectivity documents of COAE2008s and 1006 car-related subjectivity documents, the experimental results indicate that the Fisher’s discriminant ratio based on word frequency estimation has the best performance respectively with accuracy 86.61% and 82.80% under two corpus while the candidate features are the words which appear in both positive and negative texts.

An effective feature selection method for hyperspectral image classification based on genetic algorithm and support vector machine

With the development and popularization of the remote-sensing imaging technology, there are more and more applications of hyperspectral image classification tasks, such as target detection and land cover investigation. It is a very challenging issue of urgent importance to select a minimal and effective subset from those mass of bands. This paper proposed a hybrid feature selection strategy based on genetic algorithm and support vector machine (GA–SVM), which formed a wrapper to search for the best combination of bands with higher classification accuracy. In addition, band grouping based on conditional mutual information between adjacent bands was utilized to counter for the high correlation between the bands and further reduced the computational cost of the genetic algorithm. During the post-processing phase, the branch and bound algorithm was employed to filter out those irrelevant band groups. Experimental results on two benchmark data sets have shown that the proposed approach is very competitive and effective.

A novel approach to improving C-Tree for feature selection

Rough set approach is one of effective feature selection methods that can preserve the meaning of the features. So far, many feature selection (also called feature reduction) methods based on Rough set have been proposed. Of which, methods based on discernibility matrix are of considerable benefits for their conciseness and effectiveness, but have much higher space complexity. In order to reduce the storage space of the existing feature selection methods based on discernibility matrix, a novel condensing tree ( C-Tree ) structure was introduced, which is an extended order-tree, every nonempty element of a discernibility matrix is stored in one path in the C-Tree by given order of features and lots of nonempty elements share one path or sub-path, so the C-Tree has much lower space complexity as compared to discernibility matrix. However, the size of a C-Tree greatly depends on the order of features in most cases, hence how to set the proper order of features is of importance. To generate a higher compressed C-Tree , in this paper, after introducing an efficient trick for efficiently measuring the relative importance of every feature, we present a new feature ordering strategy according to the descending order of their importance. Further, based on the new feature ordering strategy, corresponding two heuristic algorithms for feature selection are introduced. Algorithms of this paper are experimented using six standard datasets and five synthetic datasets for testing both time and space complexities. Experimental results show that the newly improved feature selection algorithm can further efficiently reduce cost of storage in most cases.

Rough set and Tabu search based feature selection for credit scoring

Abstract As the credit industry has been growing rapidly, huge number of consumers’ credit data are collected by the credit department of the bank and credit scoring has become a very important issue. Usually, a large amount of redundant information and features are involved in the credit dataset, which leads to lower accuracy and higher complexity of the credit scoring model. So, effective feature selection methods are necessary for credit dataset with huge number of features. In this paper, a novel approach, called FSRT, to feature selection based on rough set and tabu search is proposed. In FSRT, conditional entropy is regarded as the heuristic to search the optimal solutions. The proposed method is introduced to credit scoring and Japan credit dataset in UCI database is selected to demonstrate the competitive performance of the proposed method. Moreover, FSRT shows a superior performance in saving the computational costs and improving classification accuracy.

Feature selection for SVM via optimization of kernel polarization with Gaussian ARD kernels

Feature selection aims at determining a subset of available features which is most discriminative and informative for data analysis. This paper presents an effective feature selection method for support vector machine (SVM). Unlike the traditional combinatorial searching method, feature selection is translated into the model selection of SVM which has been well studied. In more detail, the basic idea of this method is to tune the hyperparameters of the Gaussian Automatic Relevance Determination (ARD) kernels via optimization of kernel polarization, and then to rank all features in decreasing order of importance so that more relevant features can be identified. We test the proposed method with some UCI machine learning benchmark examples and show that it can dramatically reduce the number of features and outperforms SVM trained using the features selected according to correlation coefficient and using all features.

A novel condensing tree structure for rough set feature selection

Rough set approach is one of effective feature selection methods that can preserve the meaning of the features. The essence of feature selection based on rough set approach is to find a subset of the original features (attributes) using rough set theory. So far, many feature selection (also called feature reduction) methods based on rough set have been proposed, where numerous experimental results have demonstrated that these methods based on discernibility matrix are concise and efficient, but have much higher space complexity. In order to reduce the storage space of the existing feature selection methods based on discernibility matrix, in this paper, we introduce a novel condensing tree structure ( C-Tree), which is an extended order-tree, every non-empty element of a discernibility matrix is mapped to one path in the C-Tree and a lot of non-empty elements may share the same path or prefix, so the C-Tree has much lower space complexity as compared to discernibility matrix. Moreover, our feature selection algorithms employ the C-Tree structure and incorporate some heuristic strategies, hence efficiently reduce both space and computational complexities. Algorithms of this paper are experimented using some standard datasets and synthetic datasets for testing both time and space complexities. Experimental results show that the algorithms of this paper can efficiently reduce the cost of storage and be computationally inexpensive when compared to the existing algorithms based on discernibility matrix for feature reduction.

A NEURAL NETWORK MODEL WITH FEATURE SELECTION FOR KOREAN SPEECH ACT CLASSIFICATION

A speech act is a linguistic action intended by a speaker. Speech act classification is an essential part of a dialogue understanding system because the speech act of an utterance is closely tied with the user's intention in the utterance. We propose a neural network model for Korean speech act classification. In addition, we propose a method that extracts morphological features from surface utterances and selects effective ones among the morphological features. Using the feature selection method, the proposed neural network can partially increase precision and decrease training time. In the experiment, the proposed neural network showed better results than other models using comparatively high-level linguistic features. Based on the experimental result, we believe that the proposed neural network model is suitable for real field applications because it is easy to expand the neural network model into other domains. Moreover, we found that neural networks can be useful in speech act classification if we can convert surface sentences into vectors with fixed dimensions by using an effective feature selection method.