New Approach For Feature Selection Research Articles

Automatically Diagnosing Skin Cancers From Multimodality Images Using Two-Stage Genetic Programming.

Developing a computer-aided diagnostic system for detecting various skin malignancies from images has attracted many researchers. Unlike many machine-learning approaches, such as artificial neural networks, genetic programming (GP) automatically evolves models with flexible representation. GP successfully provides effective solutions using its intrinsic ability to select prominent features (i.e., feature selection) and build new features (i.e., feature construction). Existing approaches have utilized GP to construct new features from the complete set of original features and the set of operators. However, the complete set of features may contain redundant or irrelevant features that do not provide useful information for classification. This study aims to develop a two-stage GP method, where the first stage selects prominent features, and the second stage constructs new features from these selected features and operators, such as multiplication in a wrapper approach to improve the classification performance. To include local, global, texture, color, and multiscale image properties of skin images, GP selects and constructs features extracted from local binary patterns and pyramid-structured wavelet decomposition. The accuracy of this GP method is assessed using two real-world skin image datasets captured from the standard camera and specialized instruments, and compared with commonly used classification algorithms, three state of the art, and an existing embedded GP method. The results reveal that this new approach of feature selection and feature construction effectively helps improve the performance of the machine-learning classification algorithms. Unlike other black-box models, the evolved models by GP are interpretable; therefore, the proposed method can assist dermatologists to identify prominent features, which has been shown by further analysis on the evolved models.

Generalizing Gain Penalization for Feature Selection in Tree-Based Models

We develop a new approach for feature selection via gain penalization in tree-based models. First, we show that previous methods do not perform sufficient regularization and often exhibit sub-optimal out-of-sample performance, especially when correlated features are present. Instead, we develop a new gain penalization idea that exhibits a general local-global regularization for tree-based models. The new method allows for full flexibility in the choice of feature-specific importance weights, while also applying a global penalization. We validate our method on both simulated and real data, exploring how the hyperparameters interact and we provide the implementation as an extension of the popular R package ranger.

Feature Selection for Classification through Population Random Search with Memory

We propose a new approach for feature selection. The proposed approach is based on a combination of random and heuristic search strategies. The solution is represented as a binary vector whose dimension is determined by the number of features in the dataset. New solutions are generated at random using a normal and uniform distribution. The heuristic underlying the proposed approach can be formulated as follows: the chance of a feature to get into the next generation is proportional to the frequency of this feature appearing in previous best solutions. The proposed approach has been tested on several datasets from the KEEL repository. We also show an experimental comparison with other methods.

A New Approach for Feature Selection from Microarray Data Based on Mutual Information.

Mutual information (MI) is a powerful concept for correlation-centric applications. It has been used for feature selection from microarray gene expression data in many works. One of the merits of MI is that, unlike many other heuristic methods, it is based on a mature theoretic foundation. When applied to microarray data, however, it faces some challenges. First, due to the large number of features (i.e., genes) present in microarray data, the true distributions for the expression values of some genes may be distorted by noise. Second, evaluating inter-group mutual information requires estimating multi-variate distributions, which is quite difficult if not impossible. To address these problems, in this paper, we propose a new MI-based feature selection approach for microarray data. Our approach relies on two strategies: one is relevance boosting, which requires a desirable feature to show substantially additional relevance with class labeling beyond the already selected features, the other is feature interaction enhancing, which probabilistically compensates for feature interaction missing from simple aggregation-based evaluation. We justify our approach from both theoretical perspective and experimental results. We use a synthetic dataset to show the statistical significance of the proposed strategies, and real-life datasets to show the improved performance of our approach over the existing methods.

New Approach for Feature Selection of Thermomechanically Processed HSLA Steel using Pruned-Modular Neural Networks

A new approach has been used in modeling of strength and ductility of high strength low alloy (HSLA) steel, where a comparative study among fully-connected neural network, modular network and pruned-module architecture has been performed. The important features for modeling such a complex steel processing system have been worked out. Performance evaluation and feature selection in the soft computing domain are the two important activities for modeling input–output relationship. The need arises specially when the system is complex in terms of type of network architecture, number of features involved, number of inter-connections, application domain etc. In this paper, an attempt is made to develop a new metric of performance evaluation, using mean squared error and the total number of inter-connections of a network to improve the understanding about a complex system of thermomechanically controlled processed HSLA steels. The methodology for feature selection is developed next based on the functional form of output in terms of input variables where gradient of the function can be computed in the network.

Feature selection based on linear discriminant analysis

The paper proposed a new approach of feature selection based on Constrained Linear Discriminant Analysis(CLDA),which modeled feature selection as a search problem in subspace and made optimal solution subject to some restrictions.Furthermore,CLDA optimization problem was transformed into a process of scoring and sorting features.Experiments on UCI machine learning repository and Reuters-21578 dataset show that the proposed approach can consistently obtain better results with fewer features than that with all features.

A new approach of feature selection for text categorization

This paper proposes a new approach of feature selection based on the independent measure between features for text categorization. A fundamental hypothesis that occurrence of the terms in documents is independent of each other, widely used in the probabilistic models for text categorization (TC), is discussed. However, the basic hypothesis is incomplete for independence of feature set. From the view of feature selection, a new independent measure between features is designed, by which a feature selection algorithm is given to obtain a feature subset. The selected subset is high in relevance with category and strong in independence between features, satisfies the basic hypothesis at maximum degree. Compared with other traditional feature selection method in TC (which is only taken into the relevance account), the performance of feature subset selected by our method is prior to others with experiments on the benchmark dataset of 20 Newsgroups.

Feature selection by separability assessment of input spaces for transient stability classification based on neural networks

Power system transient-stability assessment based on neural networks can usually be treated as a two-pattern classification problem separating the stable class from the unstable class. In such a classification problem, the feature extraction and selection is the first important task to be carried out. A new approach of feature selection is presented using a new separability measure in this paper. Through finding the ‘inconsistent cases’ in a sample set, a separability index of input spaces is defined. Using the defined separability index as criterion, the breadth-first searching technique is employed to find the minimal or optimal subsets of the initial feature set. The numerical results based on extensive data obtained for the 10-unit 39-bus New England power system demonstrate the effectiveness of the proposed approach in extracting the ‘best combination’ of features for improving the quality of transient-stability classification.