Feature Set Size Research Articles

Nature-inspired feature subset selection application to arabic speaker recognition system

Feature selection is an important task which can affect the performance of pattern classification and recognition system. This study uses two nature-inspired algorithms, namely genetic algorithms and particle swarm optimization, for this problem. The algorithms adopt classifier performance and the number of the selected features as heuristic information, and select the optimal feature subset in terms of feature set size and classification performance. From experimental results, the major contribution of this work are: the reduction of vector feature size without loosing performance which is crucial for real time application and low-resources devices as well as the time needed to find the optimal subset features compared to exhaustive search or other conventional methods (less than 50 iterations instead of billions of iterations to test all possible configurations).

A new strategy for parameter optimization to improve phase-dependent locomotion mode recognition

Phase-dependent recognition strategy is an effective approach for lower-limb locomotion mode recognition. However, in previous studies, classifiers, feature sets and other parameters for the classification are the same for all the phases. The potential of this method could therefore be limited, as movement characteristics of different phases are not the same. In this paper, we aim to further improve phase-dependent recognition by proposing a new parameter optimization strategy which optimizes classifier, feature set and window size individually for each phase. Seven able-bodied subjects and one transtibial amputee subject are recruited in this research and they are required to perform six kinds of locomotion tasks. Signals recorded from two inertial measurement units and one pressure insole of the measured side are used for feature set calculation. And phase-dependent recognition method with four phases defined is applied for locomotion mode identification. The proposed strategy for parameter optimization is proved to be more efficient than the conventional optimization strategy by providing better overall recognition performance and lower computation burden.

Feature selection methods and their combinations in high-dimensional classification of speaker likability, intelligibility and personality traits

This study focuses on feature selection in paralinguistic analysis and presents recently developed supervised and unsupervised methods for feature subset selection and feature ranking. Using the standard k-nearest-neighbors (kNN) rule as the classification algorithm, the feature selection methods are evaluated individually and in different combinations in seven paralinguistic speaker trait classification tasks. In each analyzed data set, the overall number of features highly exceeds the number of data points available for training and evaluation, making a well-generalizing feature selection process extremely difficult. The performance of feature sets on the feature selection data is observed to be a poor indicator of their performance on unseen data. The studied feature selection methods clearly outperform a standard greedy hill-climbing selection algorithm by being more robust against overfitting. When the selection methods are suitably combined with each other, the performance in the classification task can be further improved. In general, it is shown that the use of automatic feature selection in paralinguistic analysis can be used to reduce the overall number of features to a fraction of the original feature set size while still achieving a comparable or even better performance than baseline support vector machine or random forest classifiers using the full feature set. The most typically selected features for recognition of speaker likability, intelligibility and five personality traits are also reported.

Feature Extraction for Classification of Proteomic Profile

This work studies on feature extraction for classification of proteomic profile. We evaluated four methods, including principal component analysis (PCA), independent component analysis (ICA), locally linear embedding (LLE) and weighted maximum margin criterion (WMMC). PCA, ICA and LLE extract features based on traditional low-dimension map technique. Comparatively, WMMC extracts features according to classification goal. To study classification performance of PCA, ICA, LLE and WMMC in detail, we used two well known classification methods, support vector machine (SVM) and Fisher discriminant analysis (FDA), to classify profiles. The results show WMMC having relatively good performance due to its prediction accuracy, sensitivity and specificity for diagnosis; it can correctly identify features with high discrimination ability from high-dimensional proteomic profile. When feature set size was reduced less than 10, PCA, ICA and LLE lose a lot of classification information, and the prediction accuracies are less than 90%. However, WMMC can extract most classification information. Its prediction accuracies, sensitivities and specificities are more than 95%. Obviously, WMMC is more suitable to proteomic profile classification. For classifier, FDA is sensible to feature extraction.

Using statistical text classification to identify health information technology incidents

To examine the feasibility of using statistical text classification to automatically identify health information technology (HIT) incidents in the USA Food and Drug Administration (FDA) Manufacturer and User Facility Device Experience (MAUDE) database. We used a subset of 570 272 incidents including 1534 HIT incidents reported to MAUDE between 1 January 2008 and 1 July 2010. Text classifiers using regularized logistic regression were evaluated with both 'balanced' (50% HIT) and 'stratified' (0.297% HIT) datasets for training, validation, and testing. Dataset preparation, feature extraction, feature selection, cross-validation, classification, performance evaluation, and error analysis were performed iteratively to further improve the classifiers. Feature-selection techniques such as removing short words and stop words, stemming, lemmatization, and principal component analysis were examined. κ statistic, F1 score, precision and recall. Classification performance was similar on both the stratified (0.954 F1 score) and balanced (0.995 F1 score) datasets. Stemming was the most effective technique, reducing the feature set size to 79% while maintaining comparable performance. Training with balanced datasets improved recall (0.989) but reduced precision (0.165). Statistical text classification appears to be a feasible method for identifying HIT reports within large databases of incidents. Automated identification should enable more HIT problems to be detected, analyzed, and addressed in a timely manner. Semi-supervised learning may be necessary when applying machine learning to big data analysis of patient safety incidents and requires further investigation.

Resampling methods for quality assessment of classifier performance and optimal number of features

We address two fundamental design issues of a classification system: the choice of the classifier and the dimensionality of the optimal feature subset. Resampling techniques are applied to estimate both the probability distribution of the misclassification rate (or any other figure of merit of a classifier) subject to the size of the feature set, and the probability distribution of the optimal dimensionality given a classification system and a misclassification rate. The latter allows for the estimation of confidence intervals for the optimal feature set size. Based on the former, a quality assessment for the classifier performance is proposed. Traditionally, the comparison of classification systems is accomplished for a fixed feature set. However, a different set may provide different results. The proposed method compares the classifiers independently of any pre-selected feature set. The algorithms are tested on 80 sets of synthetic examples and six standard databases of real data. The simulated data results are verified by an exhaustive search of the optimum and by two feature selection algorithms for the real data sets.

An Improved Minimum Redundancy Maximum Relevance Approach for Feature Selection in Gene Expression Data

Abstract In this article, an improved feature selection technique has been proposed. Mutual Information is taken as the basic criterion to find the feature relevance and redundancy. The mutual information between a feature and class labels defines the relevance of that feature. Again, the mutual information among different features defines the correlation i.e., the redundancy among those features. Now our objective is to find such a feature set for which the mutual information among the features and the class labels are maximized and the mutual information among the features are minimized. Therefore, the goal of the proposed method is to find the most relevant and least redundant feature set. The number of output features is provided by the user. First the most relevant feature is added to the empty final feature set. Then in each iteration a non-dominated feature set with respect to relevance and redundancy is generated and from this set of features, the most relevant and non-redundant feature is included in the final feature set. Thereafter, in an incremental way a feature is added in every iteration and this step is repeated while the size of the final feature set is equal to the user given number of features. The features contained by the final feature set have maximum relevance and least correlation. The proposed method is applied on microarray gene expression data to find the most relevant and non-redundant genes and the performance of the proposed method is compared with that of the popular mRMR (MIQ) and mRMR (MID) schemes on several real-life data sets.

Comparison of Machine Learning Methods for Evaluating Pavement Roughness Based on Vehicle Response

The roughness of a road pavement affects safety, ride comfort, and road durability. A useful indicator for evaluating roughness is the weighted longitudinal profile (wLP). In this paper, three machine learning models are compared for estimating the wLP when vehicle response information, i.e., accelerometer and wheel speed data is collected from common in-vehicle sensors. A multilayer perceptron, support vector machine (SVM) and random forest were applied for testing their effectiveness in estimating the key indices of wLP, namely, range and standard deviation. These models were trained from a set of features extracted from vehicle response simulations on accurate replications of roads with various roughness problems. In contrast to other research, the authors validated models with measurements collected with a probe vehicle. The results show that roughness phenomena can be accurately detected. The SVM produced the best results, although the models achieved rather similar performance. However, differences were found regarding the model robustness when reducing the size of the training feature set. The proposed method enables road network monitoring to be achieved by conventional passenger cars, which can be seen as a practical supplement to the prevalent road measurements with cost-intensive mobile devices.

Efficient ant colony optimization for image feature selection

Feature selection (FS) is an important task which can significantly affect the performance of image classification and recognition. In this paper, we present a feature selection algorithm based on ant colony optimization (ACO). For n features, existing ACO-based feature selection methods need to traverse a complete graph with O(n2) edges. However, we propose a novel algorithm in which the artificial ants traverse on a directed graph with only O(2n) arcs. The algorithm incorporates the classification performance and feature set size into the heuristic guidance, and selects a feature set with small size and high classification accuracy. We perform extensive experiments on two large image databases and 15 non-image datasets to show that our proposed algorithm can obtain higher processing speed as well as better classification accuracy using a smaller feature set than other existing methods.

Mutual information based input feature selection for classification problems

The elimination process aims to reduce the size of the input feature set and at the same time to retain the class discriminatory information for classification problems. This paper investigates the approaches to solve classification problems of the feature selection and proposes a new feature selection algorithm using the mutual information (MI) concept in information theory for the classification problems. The proposed algorithm calculates the MI between the combinations of input features and the class instead of the MI between a single input feature and the class for both continuous-valued and discrete-valued features. Three experimental tests are conducted to evaluate the proposed algorithm. Comparison studies of the proposed algorithm with the previously published classification algorithms indicate that the proposed algorithm is robust, stable and efficient.

Feature Selection Based on Class-Dependent Densities for High-Dimensional Binary Data

Data and knowledge management systems employ feature selection algorithms for removing irrelevant, redundant, and noisy information from the data. There are two well-known approaches to feature selection, feature ranking (FR) and feature subset selection (FSS). In this paper, we propose a new FR algorithm, termed as class-dependent density-based feature elimination (CDFE), for binary data sets. Our theoretical analysis shows that CDFE computes the weights, used for feature ranking, more efficiently as compared to the mutual information measure. Effectively, rankings obtained from both the two criteria approximate each other. CDFE uses a filtrapper approach to select a final subset. For data sets having hundreds of thousands of features, feature selection with FR algorithms is simple and computationally efficient but redundant information may not be removed. On the other hand, FSS algorithms analyze the data for redundancies but may become computationally impractical on high-dimensional data sets. We address these problems by combining FR and FSS methods in the form of a two-stage feature selection algorithm. When introduced as a preprocessing step to the FSS algorithms, CDFE not only presents them with a feature subset, good in terms of classification, but also relieves them from heavy computations. Two FSS algorithms are employed in the second stage to test the two-stage feature selection idea. We carry out experiments with two different classifiers (naive Bayes' and kernel ridge regression) on three different real-life data sets (NOVA, HIVA, and GINA) of the”Agnostic Learning versus Prior Knowledge” challenge. As a stand-alone method, CDFE shows up to about 92 percent reduction in the feature set size. When combined with the FSS algorithms in two-stages, CDFE significantly improves their classification accuracy and exhibits up to 97 percent reduction in the feature set size. We also compared CDFE against the winning entries of the challenge and found that it outperforms the best results on NOVA and HIVA while obtaining a third position in case of GINA.

Hybrid feature selection for text classification

Feature selection is vital in the field of pattern classification due to accuracy and processing time considerations. The selection of proper features is of greater importance when the initial feature set is considerably large. Text classification is a typical example of this situation, where the size of the initial feature set may reach to hundreds or even thousands. There are numerous research studies in the literature offering different feature selection strategies for text classification, mostly focused on filters. In spite of the extensive number of these studies, there is no significant work investigating the efficacy of a combination of features, which are selected by different selection methods, under different conditions. In this study, a hybrid feature selection strategy, which consists of both filter and wrapper feature selection steps, is proposed to comprehensively analyze the redundancy or relevancy of the text features selected by different methods in the case of different feature set sizes, dataset characteristics, classifiers, and success measures. The results of the experimental study reveal that a combination of the features selected by various methods is more effective than the features selected by the single selection method. The profile of the combination is, however, influenced by characteristics of the dataset, choice of the classification algorithm, and the success measure.

특징 선택에서 선택적 평가를 사용하는 개미 군집 최적화의 수렴 특성

최근 특징 선택에서 개미군집 최적화를 위한 선택적 평가 기법이 제안되었다. 이 기법은 불필요하거나 가능성이 적은 후보 해를 실제 평가 과정에서 제외함으로써 계산량을 줄인다. 실험을 통해 이 기법의 우수성을 보였으나, 하나의 데이터만을 사용하였으므로 통계적으로 충분한 신뢰성을 보여주지 못한다. 이 논문의 목적은 선택적 평가 기법의 수렴 특성을 분석하고 결론의 신뢰성을 높이는 것이다. 실험을 위해 UCI 데이터베이스에서 필기, 의료, 음성에 관련된 세가지 데이터를 선택하였다. 이들의 특징 집합 크기는 256부터 617까지 분포한다. 통계적으로 안정된 데이터를 얻기 위해, 이들 각각에 대해 프로그램을 독립적으로 12번 실행하였다. 긴 시간에 걸친 수렴을 관찰하기 위해, 각각의 프로그램 실행은 72시간 동안 이루어졌다. 실험 데이터의 분석을 바탕으로, 선택적 평가 기법의 우수성에 대한 이유와 이 기법의 적용 범위에 대해 기술한다. In feature selection, the selective evaluation scheme for Ant Colony Optimization(ACO) has recently been proposed, which reduces computational load by excluding unnecessary or less promising candidate solutions from the actual evaluation. Its superiority was supported by experimental results. However the experiment seems to be not statistically sufficient since it used only one dataset. The aim of this paper is to analyze convergence characteristics of the selective evaluation scheme and to make the conclusion more convincing. We chose three datasets related to handwriting, medical, and speech domains from UCI repository whose feature set size ranges from 256 to 617. For each of them, we executed 12 independent runs in order to obtain statistically stable data. Each run was given 72 hours to observe the long-time convergence. Based on analysis of experimental data, we describe a reason for the superiority and where the scheme can be applied.

Improved contourlet-based steganalysis using binary particle swarm optimization and radial basis neural networks

Steganography is the science of hiding information in a media such as video, image or audio files. On the other hand, the aim of steganalysis is to detect the presence of embedded data in a given media. In this paper, a steganalysis method is presented for the colored joint photographic experts group images in which the statistical moments of contourlet transform coefficients are used as the features. In this way, binary particle swarm optimization algorithm is also employed as a closed-loop feature selection method to select the efficient features in tandem with improvement of the detection rate. Nonlinear support vector machine and two variants of radial basis neural networks, i.e., radial basis function and probabilistic neural network, are used as the classification tools and their performance is compared in detecting the stego and clean images. Experimental results show that even for low embedding rates, the detection accuracy of the proposed method is more than 80% along with 30% reduction in the size of feature set.

Information-theoretic approaches to SVM feature selection for metagenome read classification

AbstractAnalysis of DNA sequences isolated directly from the environment, known as metagenomics, produces a large quantity of genome fragments that need to be classified into specific taxa. Most composition-based classification methods use all features instead of a subset of features that may maximize classifier accuracy. We show that feature selection methods can boost performance of taxonomic classifiers. This work proposes three different filter-based feature selection methods that stem from information theory: (1) a technique that combines Kullback–Leibler, Mutual Information, and distance information, (2) a text mining technique, TF-IDF, and (3) minimum redundancy-maximum-relevance (mRMR). The feature selection methods are compared by how well they improve support vector machine classification of genomic reads. Overall, the 6mer mRMR method performs well, especially on the phyla-level. If the number of total features is very large, feature selection becomes difficult because a small subset of features that captures a majority of the data variance is less likely to exist. Therefore, we conclude that there is a trade-off between feature set size and feature selection method to optimize classification performance. For larger feature set sizes, TF-IDF works better for finer-resolutions while mRMR performs the best out of any method for N=6 for all taxonomic levels.

On Scale Invariance Texture Image Retrieval using Fuzzy Logic and Wavelet Co-occurrence based Features

this paper, analysis of the feature selection for scale invariance texture image retrieval using fuzzy logic classifier and wavelet and co-occurrence matrix based feature is carried out. Two types of texture features are extracted one using Discrete Wavelet Transform (DWT) and other using Co- occurrence matrix. Energy and Standard Deviation are obtained from each sub-band of DWT coefficients up to fifth level of decomposition and eight features are extracted from co-occurrence matrix of whole image and each sub-band of first level DWT decomposition. The different size samples of texture image are undertaken. The suitability of features extracted is analyzed using a fuzzy logic classifier. The performance is measured in terms of Success Rate. Best and Worst case analysis is done for each of the feature set and texture image size. Also the minimum number of features required for maximum average success rate is obtained. This study shows that for samples taken from 256x256 texture size, excellent success rate is achieved for Wavelet Statistical Features (WSF) as well as Wavelet Co-occurrence Features (WCF). Also WSF perform better for 128x128 and 256x256 texture image. For both the types of features performance degrades in case of 512x512 texture image. Worst case analysis shows that energy feature WSF and 8-features group WCF performs excellently.

Hierarchical genetic algorithm with new evaluation function and bi-coded representation for the selection of features considering their confidence rate

In this paper, we propose a new feature selection method based on a hierarchical genetic algorithm (GA) with a new evaluation function and a bi-coded representation. The hierarchical GA with homogeneous and heterogeneous population is used to minimize the computational load and to accelerate convergence speed. The fitness function is designed to find the solution that both maximizes the recognition rate and minimizes the feature set size. Each solution candidate is represented by two chromosomes which lengths are identical to the number of available features. The first binary chromosome represents the presence of features in the solution candidate; the second represents the confidence rates of features, which are used to assign different weights to features during the classification procedure and to achieve more accurate classifier. The proposed method is tested using five databases and is shown to outperform many commonly used feature selection algorithms.

Efficient mining for structurally diverse subgraph patterns in large molecular databases

We present a new approach to large-scale graph mining based on so-called backbone refinement classes. The method efficiently mines tree-shaped subgraph descriptors under minimum frequency and significance constraints, using classes of fragments to reduce feature set size and running times. The classes are defined in terms of fragments sharing a common backbone. The method is able to optimize structural inter-feature entropy as opposed to purely occurrence-based criteria, which is characteristic for open or closed fragment mining. We first give an intuitive explanation why backbone refinement class features lead to a set of relevant features that are suitable for classification, in particular in the area of structure-activity relationships (SARs). We then show that backbone refinement classes yield a high compression in the search space of rooted perfect binary trees. We conduct several experiments to evaluate our theoretical insights in practice: A visualization suggests low co-occurrence and high entropy of backbone refinement class features. By comparison to a class of patterns sampled from the maximal patterns previously introduced by Al Hasan et al., we find a favorable tradeoff between the structural similarity and the resources needed to compute the descriptors. Cross-validation shows that classification accuracy is similar to the complete set of trees but significantly better than that of open trees, while feature set size is reduced by >90% and >30% compared to complete tree mining and open tree mining, respectively. Furthermore, compared to open or closed pattern mining, a large part of the search space can be pruned due to an improved statistical constraint (dynamic upper bound adjustment). This is confirmed experimentally by running times reduced by more than 60% compared to ordinary (static) upper bound pruning. The application of our method to the largest datasets that have been used in correlated graph mining so far indicates robustness against the minimum frequency parameter, and a cross-validation run on this data confirms that the novel descriptors render large training sets feasible, which previously might have been intractable. A C++ implementation of the mining algorithm is available at http://www.maunz.de/libfminer-doc . Animated figures, links to datasets, and further resources are available at http://www.maunz.de/mlj-res .

Evaluation of annual forest disturbance monitoring using a static decision tree approach and 250 m MODIS data

Abstract Research on change detected has largely focused on method development and evaluation in a temporally dependent manner where training and validation data are from the same temporal period. Monitoring over several change periods needs to account for increased variability resulting from possible combinations of atmosphere, sensor, and surface conditions. Training a change method for each monitoring period (i.e. a dynamic approach) is an option, but can be costly to develop the needed training datasets and may not be warranted if sufficient accuracy can be obtained without retraining (i.e. a static approach). In this research the potential of change detection using a static approach suitable for near-real time annual monitoring was evaluated. The research assessed the influence of feature set size, radiometric normalization, incorporation of temporal information, and change object size and sub-pixel fraction on accuracy. The static approach was based on a decision tree developed using 250 m MODIS data from 2005 to 2006 and applied annually for the period 2001–2005. Change results between years were combined and compared to reference data representing change from 2001 to 2005 to evaluate monitoring performance. Results revealed high accuracy for the decision tree change model development from 2005 to 2006 (bootstrap cross-validation KAPPA = 0.91), with lower accuracy (KAPPA = 0.80) when applied for monitoring from 2001 to 2005. Radiometric normalization increased monitoring accuracy (KAPPA = 0.86). Further improvement was achieved with the incorporation of temporal contextual tests to combine the 2001–2005 inter-annual change maps (KAPPA = 0.90), but required a time lag of 1 year. An alternative temporal test that was not restricted by the 1 year time lag produced slightly lower accuracy (KAPPA = 0.88). Evaluation of the effect of object size on detection accuracy showed that accuracy for objects less than 7 pixels was strongly related to object size, with objects less than 3 pixels having low detection rates. The effect of sub-pixel change fraction was found to be dependent on object size with larger objects reducing detection error across the range of fractions evaluated.

Application of complex discrete wavelet transform in classification of Doppler signals using complex-valued artificial neural network

In biomedical signal classification, due to the huge amount of data, to compress the biomedical waveform data is vital. This paper presents two different structures formed using feature extraction algorithms to decrease size of feature set in training and test data. The proposed structures, named as wavelet transform-complex-valued artificial neural network (WT-CVANN) and complex wavelet transform-complex-valued artificial neural network (CWT-CVANN), use real and complex discrete wavelet transform for feature extraction. The aim of using wavelet transform is to compress data and to reduce training time of network without decreasing accuracy rate. In this study, the presented structures were applied to the problem of classification in carotid arterial Doppler ultrasound signals. Carotid arterial Doppler ultrasound signals were acquired from left carotid arteries of 38 patients and 40 healthy volunteers. The patient group included 22 males and 16 females with an established diagnosis of the early phase of atherosclerosis through coronary or aortofemoropopliteal (lower extremity) angiographies (mean age, 59 years; range, 48-72 years). Healthy volunteers were young non-smokers who seem to not bear any risk of atherosclerosis, including 28 males and 12 females (mean age, 23 years; range, 19-27 years). Sensitivity, specificity and average detection rate were calculated for comparison, after training and test phases of all structures finished. These parameters have demonstrated that training times of CVANN and real-valued artificial neural network (RVANN) were reduced using feature extraction algorithms without decreasing accuracy rate in accordance to our aim.