Issue Of Feature Selection Research Articles

ARABIC SCRIPT WEB PAGE LANGUAGE IDENTIFICATION USING HYBRID-KNN METHOD

In this paper, we proposed hybrid-KNN methods on the Arabic script web page language identification. One of the crucial tasks in the text-based language identification that utilizes the same script is how to produce reliable features and how to deal with the huge number of languages in the world. Specifically, it has involved the issue of feature representation, feature selection, identification performance, retrieval performance, and noise tolerance performance. Therefore, there are a number of methods that have been evaluated in this work; k-nearest neighbor (KNN), support vector machine (SVM), backpropagation neural networks (BPNN), hybrid KNN-SVM, and KNN-BPNN, in order to justify the capability of the state-of-the-art methods. KNN is prominent in data clustering or data filtering, SVM and BPNN are well known in supervised classification, and we have proposed hybrid-KNN for noise removal on web page language identification. We have used the standard measurements which are accuracy, precision, recall and F1 measurements to evaluate the effectiveness of the proposed hybrid-KNN. From the experiment, we have observed that BPNN is able to produce precise identification if the data set given is clean. However, when increasing the level of noise in the training data, KNN-SVM performs better than KNN-BPNN against the misclassification data, even on the level of 50% noise. Therefore, it is proven that KNN-SVM produce promising identification performance, in which KNN is able to reduce the noise in the data set and SVM is reliable in the language identification.

Distance metric learning and support vector machines for classification of mass spectrometry proteomics data

Mass spectrometry has become a widely used measurement in proteomics research. High dimensionality of features and small dataset are two major limitations hindering the accuracy of classification in mass spectrum data analysis; consequently, to obtain good results, the issues of feature extraction and feature selection are especially important. The quality of the feature set determines the reliability of the prediction of disease status. A well-known approach is to detect peak values and then apply support vector machine recursive feature elimination (SVMRFE) to choose feature sets for classification. In this paper, we apply a distance metric learning to classify proteomics mass spectrometry data. Experimental results show that distance metric learning can successfully be applied to the classification of proteomics data and the results are comparable to or better than, the best results by applying SVM to the feature sets chosen with the use of SVMRFE. We also perform feature reduction using manifold learning and experimental results indicate its promising potential in this application.

Population-level inferences for distributed MEG source localization under multiple constraints: Application to face-evoked fields

We address some key issues entailed by population inference about responses evoked in distributed brain systems using magnetoencephalography (MEG). In particular, we look at model selection issues at the within-subject level and feature selection issues at the between-subject level, using responses evoked by intact and scrambled faces around 170 ms (M170). We compared the face validity of subject-specific forward models and their summary statistics in terms of how estimated responses reproduced over subjects. At the within-subject level, we focused on the use of multiple constraints, or priors, for inverting distributed source models. We used restricted maximum likelihood (ReML) estimates of prior covariance components (in both sensor and source space) and show that their relative importance is conserved over subjects. At the between-subject level, we used standard anatomical normalization methods to create posterior probability maps that furnish inference about regionally specific population responses. We used these to compare different summary statistics, namely; (i) whether to test for differences between condition-specific source estimates, or whether to test the source estimate of differences between conditions, and (ii) whether to accommodate differences in source orientation by using signed or unsigned (absolute) estimates of source activity.

Feature mining and pattern classification for steganalysis of LSB matching steganography in grayscale images

In this paper, we present a scheme based on feature mining and pattern classification to detect LSB matching steganography in grayscale images, which is a very challenging problem in steganalysis. Five types of features are proposed. In comparison with other well-known feature sets, the set of proposed features performs the best. We compare different learning classifiers and deal with the issue of feature selection that is rarely mentioned in steganalysis. In our experiments, the combination of a dynamic evolving neural fuzzy inference system (DENFIS) with a feature selection of support vector machine recursive feature elimination (SVMRFE) achieves the best detection performance. Results also show that image complexity is an important reference to evaluation of steganalysis performance.

Personal recognition using hand shape and texture

This paper proposes a new bimodal biometric system using feature-level fusion of hand shape and palm texture. The proposed combination is of significance since both the palmprint and hand-shape images are proposed to be extracted from the single hand image acquired from a digital camera. Several new hand-shape features that can be used to represent the hand shape and improve the performance are investigated. The new approach for palmprint recognition using discrete cosine transform coefficients, which can be directly obtained from the camera hardware, is demonstrated. None of the prior work on hand-shape or palmprint recognition has given any attention on the critical issue of feature selection. Our experimental results demonstrate that while majority of palmprint or hand-shape features are useful in predicting the subjects identity, only a small subset of these features are necessary in practice for building an accurate model for identification. The comparison and combination of proposed features is evaluated on the diverse classification schemes; naive Bayes (normal, estimated, multinomial), decision trees (C4.5, LMT), k-NN, SVM, and FFN. Although more work remains to be done, our results to date indicate that the combination of selected hand-shape and palmprint features constitutes a promising addition to the biometrics-based personal recognition systems.

Selecting features for nuclear transients classification by means of genetic algorithms

The issue of feature selection is particularly critical for the application of monitoring and "on condition" diagnostic techniques to complex plants, like the nuclear power plants, where hundreds of parameters are measured. Indeed, irrelevant and noisy features unnecessarily increase the complexity of the problem and can degrade the diagnostic performance. In this paper, the problem of choosing among the several measured plant parameters those to be used for efficient, early transient diagnosis is tackled by means of genetic algorithms. Three different schemes for simultaneously performing the feature selection and the training of an empirical diagnostic classifier are presented. The first approach employs a single objective genetic algorithm to search the vector of features optimal with respect to the classification performance of a Fuzzy K-Nearest Neighbors classifier. With reference to the same classifier, the second and third approaches embrace a multi-objective perspective to find feature sets that achieve high classification performances with low numbers of features. In all cases, validation of the performance of the classifiers based on the optimal feature subsets identified by the genetic algorithm is successively carried out with respect to transients never used during the feature selection phase. The effectiveness of the proposed approaches is tested on a diagnostic problem regarding the classification of simulated transients in the feedwater system of a Boiling Water Reactor.

Second Order Cone Programming Formulations for Feature Selection

This paper addresses the issue of feature selection for linear classifiers given the moments of the class conditional densities. The problem is posed as finding a minimal set of features such that ...

Object detection using feature subset selection

Past work on object detection has emphasized the issues of feature extraction and classification, however, relatively less attention has been given to the critical issue of feature selection. The m...

Simultaneous feature selection and clustering using mixture models

Clustering is a common unsupervised learning technique used to discover group structure in a set of data. While there exist many algorithms for clustering, the important issue of feature selection, that is, what attributes of the data should be used by the clustering algorithms, is rarely touched upon. Feature selection for clustering is difficult because, unlike in supervised learning, there are no class labels for the data and, thus, no obvious criteria to guide the search. Another important problem in clustering is the determination of the number of clusters, which clearly impacts and is influenced by the feature selection issue. In this paper, we propose the concept of feature saliency and introduce an expectation-maximization (EM) algorithm to estimate it, in the context of mixture-based clustering. Due to the introduction of a minimum message length model selection criterion, the saliency of irrelevant features is driven toward zero, which corresponds to performing feature selection. The criterion and algorithm are then extended to simultaneously estimate the feature saliencies and the number of clusters.

Object detection using feature subset selection

Past work on object detection has emphasized the issues of feature extraction and classification, however, relatively less attention has been given to the critical issue of feature selection. The main trend in feature extraction has been representing the data in a lower dimensional space, for example, using principal component analysis (PCA). Without using an effective scheme to select an appropriate set of features in this space, however, these methods rely mostly on powerful classification algorithms to deal with redundant and irrelevant features. In this paper, we argue that feature selection is an important problem in object detection and demonstrate that genetic algorithms (GAs) provide a simple, general, and powerful framework for selecting good subsets of features, leading to improved detection rates. As a case study, we have considered PCA for feature extraction and support vector machines (SVMs) for classification. The goal is searching the PCA space using GAs to select a subset of eigenvectors encoding important information about the target concept of interest. This is in contrast to traditional methods selecting some percentage of the top eigenvectors to represent the target concept, independently of the classification task. We have tested the proposed framework on two challenging applications: vehicle detection and face detection. Our experimental results illustrate significant performance improvements in both cases.

Estimating breast cancer risks using neural networks

Breast cancer is one of the most important medical problems. In this paper, we report the results of using neural networks for breast cancer diagnosis. The theoretical advantage is that posterior probabilities of malignancy can be estimated directly, and coupled with resampling techniques such as the bootstrap, distributions of the probabilities can also be obtained. These allow a researcher much more insight into the variability of estimated probabilities. Another contribution is that we present an integrative approach to building neural network models. The issues of model selection, feature selection, and function approximation are discussed in some detail and illustrated with the application to breast cancer diagnosis.

Estimating breast cancer risks using neural networks

Breast cancer is one of the most important medical problems. In this paper, we report the results of using neural networks for breast cancer diagnosis. The theoretical advantage is that posterior probabilities of malignancy can be estimated directly, and coupled with resampling techniques such as the bootstrap, distributions of the probabilities can also be obtained. These allow a researcher much more insight into the variability of estimated probabilities. Another contribution is that we present an integrative approach to building neural network models. The issues of model selection, feature selection, and function approximation are discussed in some detail and illustrated with the application to breast cancer diagnosis.

Feature analysis through information granulation and fuzzy sets

Feature analysis and feature selection are fundamental pursuits in pattern recognition. We revisit and generalize an issue of feature selection by introducing a mechanism of soft (fuzzy) feature selection. The underlying idea is to consider features to be granular rather than numeric. By varying the level of granularity, we modify the level of contribution of the specific feature to the overall feature space. We admit an interval model of the features meaning that their values assume a form of numeric intervals. The intervalization of the features exhibits a clear-cut interpretation. Moreover a contribution of the features to the formation of the feature space can be easily controlled: the broader the interval, the less essential contribution of the feature to the entire feature space. In limit, when the intervals get broad enough, one may view the feature to be completely eliminated (dropped) from the feature space. The quantification of the features in terms of their importance is realized in the setting of the clustering FCM model (namely, a process of the binary or fuzzy feature selection is carried out and numerically quantified in the space of membership values generated by fuzzy clusters). As the focal point of this study concerns an interval-like form of information granules, we reveal how such feature intervalization helps approximate fuzzy sets described by any type of membership function. Detailed computations give rise to a detailed quantification of such granular features. Numerical experiments provide a comprehensive numerical illustration of the problem.

Metaphors, Logic, and Type Theory

Metaphorical use of language is often thought to be at odds with compositional, truth-conditional approaches to semantics: After all, most metaphors are literally false. In this article we sketch an approach to metaphors based on standard type theory. Our approach is classical: We do not invent a new logic. The approach models sense extension in a simple and elegant way: The properties (supertypes) shared between tenor and vehicle include the extensions of at least both. The original predicates remain unchanged. Our approach captures an asymmetry between metaphor and simile. The literal interpretation of a metaphor comes out as (mostly) false, whereas its nonliteral interpretation is that of a corresponding reduced simile. A compositional syntax-semantics interface is provided and a deductive account of metaphor resolution is outlined. The approach readily translates into a simple computational implementation in Prolog. We discuss how our approach addresses issues of generalization, feature selection, asymmetry, tension, trivialization, prototypicality, truth conditions, comprehension, and generativeness.

A neural network and fuzzy rule base hybrid for pattern classification

This paper presents a novel hybrid of the two complimentary technologies of soft computing viz. neural networks and fuzzy logic to design a fuzzy rule based pattern classifier for problems with higher dimensional feature spaces. The neural network component of the hybrid, which acts as a pre-processor, is designed to take care of the all-important issue of feature selection. To circumvent the disadvantages of the popular back propagation algorithm to train the neural network, a meta-heuristic viz. threshold accepting (TA) has been used instead. Then, a fuzzy rule based classifier takes over the classification task with a reduced feature set. A combinatorial optimisation problem is formulated to minimise the number of rules in the classifier while guaranteeing high classification power. A modified threshold accepting algorithm proposed elsewhere by the authors (Ravi V, Zimmermann H.-J. (2000) Eur J Oper Res 123: 16–28) has been employed to solve this optimization problem. The proposed methodology has been demonstrated for (1) the wine classification problem having 13 features and (2) the Wisconsin breast cancer determination problem having 9 features. On the basis of these examples the results seem to be very interesting, as there is no reduction in the classification power in either of the problems, despite the fact that some of the original features have been completely eliminated from the study. On the contrary, the chosen features in both the problems yielded 100% classification power in some cases.