Representative Feature Subset Research Articles

Feature Selection Using Submodular Approach forFinancial Big Data

As the world is moving towards digitization, data is generated from various sources at a faster rate. It is getting humungous and is termed as big data. The financial sector is one domain which needs to leverage the big data being generated to identify financial risks, fraudulent activities, and so on. The design of predictive models for such financial big data is imperative for maintaining the health of the country’s economics. Financial data has many features such as transaction history, repayment data, purchase data, investment data, and so on. The main problem in predictive algorithm is finding the right subset of representative features from which the predictive model can be constructed for a particular task. This paper proposes a correlation-based method using submodular optimization for selecting the optimum number of features and thereby, reducing the dimensions of the data for faster and better prediction. The important proposition is that the optimal feature subset should contain features having high correlation with the class label, but should not correlate with each other in the subset. Experiments are conducted to understand the effect of the various subsets on different classification algorithms for loan data. The IBM Bluemix Big Data platform is used for experimentation along with the Spark notebook. The results indicate that the proposed approach achieves considerable accuracy with optimal subsets in significantly less execution time. The algorithm is also compared with the existing feature selection and extraction algorithms.

An Empirical Study on the Effectiveness of Feature Selection for Cross-Project Defect Prediction

Software defect prediction has attracted much attention of researchers in software engineering. At present, feature selection approaches have been introduced into software defect prediction, which can improve the performance of traditional defect prediction (known as within-project defect prediction, WPDP) effectively. However, the studies on feature selection are not sufficient for cross-project defect prediction (CPDP). In this paper, we use the feature subset selection and feature ranking approaches to explore the effectiveness of feature selection for CPDP. An empirical study is conducted on NASA and PROMISE datasets. The results show that both the feature subset selection and feature ranking approaches can improve the performance of CPDP. Therefore, we should select the representative feature subset or set a reasonable proportion of selected features to improve the performance of CPDP in future studies.

Graph autoencoder-based unsupervised feature selection with broad and local data structure preservation

Abstract Feature selection is a dimensionality reduction technique that selects a subset of representative features from high-dimensional data by eliminating irrelevant and redundant features. Recently, feature selection combined with sparse learning has attracted significant attention due to its outstanding performance compared with traditional feature selection methods that ignores correlation between features. These works first map data onto a low-dimensional subspace and then select features by posing a sparsity constraint on the transformation matrix. However, they are restricted by design to linear data transformation, a potential drawback given that the underlying correlation structures of data are often non-linear. To leverage a more sophisticated embedding, we propose an autoencoder-based unsupervised feature selection approach that leverages a single-layer autoencoder for a joint framework of feature selection and manifold learning. More specifically, we enforce column sparsity on the weight matrix connecting the input layer and the hidden layer, as in previous work. Additionally, we include spectral graph analysis on the projected data into the learning process to achieve local data geometry preservation from the original data space to the low-dimensional feature space. Extensive experiments are conducted on image, audio, text, and biological data. The promising experimental results validate the superiority of the proposed method.

An improved wrapper-based feature selection method for machinery fault diagnosis.

A major issue of machinery fault diagnosis using vibration signals is that it is over-reliant on personnel knowledge and experience in interpreting the signal. Thus, machine learning has been adapted for machinery fault diagnosis. The quantity and quality of the input features, however, influence the fault classification performance. Feature selection plays a vital role in selecting the most representative feature subset for the machine learning algorithm. In contrast, the trade-off relationship between capability when selecting the best feature subset and computational effort is inevitable in the wrapper-based feature selection (WFS) method. This paper proposes an improved WFS technique before integration with a support vector machine (SVM) model classifier as a complete fault diagnosis system for a rolling element bearing case study. The bearing vibration dataset made available by the Case Western Reserve University Bearing Data Centre was executed using the proposed WFS and its performance has been analysed and discussed. The results reveal that the proposed WFS secures the best feature subset with a lower computational effort by eliminating the redundancy of re-evaluation. The proposed WFS has therefore been found to be capable and efficient to carry out feature selection tasks.

Self-representation dimensionality reduction for multi-model classification

Feature selection remove the noisy/irrelevant samples and select the subset of representative features, in general, from the high-dimensional space of data has been a fatal significant technique in computer vision and machine learning. Afterwards, motivated by the interpretable ability of feature selection patterns, beside, and the successful use of low-rank constraint in static and sparse learning in the field of machine learning. We present a novel feature selection model with unsupervised learning by using low-rank regression on loss function, and a sparsity term plus K-means clustering method on regularization term during this article. In order to distinguish from those existing state-of-the-art attribute selection measures, the propose method have described as follows: (1) represent the every feature by other features (including itself) via utilize the corresponding loss function with a feature-level self-express way; (2) embed K-means to generate pseudo class label information for the attribute selection as an pseudo supervised method, because of the supervised learning usually have the better recognition results than unsupervised learning; (3) also use the low-rank constraint to feature selection which considers two aspects of information inherent in data. The low-rank constraint takes the correlation of response variables into account, while an ℓ2, p-norm regularizer considers the correlation between feature vectors and their corresponding response variables. The extensive relevant results of experiment on three multi-model comparison data demonstrated that our new unsupervised feature selection pattern outperforms the related approaches.

MO-G-201-03: Deep-Learning Based Prediction of Achievable Dose for Personalizing Inverse Treatment Planning

Purpose: Inverse planning starts with prescribed dose or DVHs. The time/effort spent on obtaining a clinically acceptable plan and, more importantly, the quality of the resultant treatment plan depends critically on the prescription. The present work aims to facilitate the treatment planning workflow by predicting the achievable dose distribution based on a dataset of quality plans designed for past patients. Methods: A novel learning-empowered approach is developed that predicts achievable 3D dose volume for a new patient based on the geometrical features of the contoured anatomical-scan. Two major steps are involved: (1) Feature extraction: To extract a rich set of features resilient to patient and tumor-shape variations, multi-layer convolutional auto-encoders (CAE) are applied separately to structural contours and datasets of isodose distribution. (2) Learning the relation map: From the historical dose and contour features, a predictive model based on multi-task regression is trained, where dose features are linearly correlated with a subset of contour features selected via sparsity regularization. IMRT plans for 115 prostate patients were used to train the models and the final method is tested on additional 10 prostate plans. Dose and contour volumes are rearranged to form a 3D polar grid with (r,ϕ,z) coordinates. Results: A deep-learning based dose prediction model for VMAT/IMRT treatment is established. Our preliminary results using a two-layer linear auto-encoder achieve more than %70 overlap volume between the actual and predicted isodose surfaces associated with V60%, V75%, and V90. Analysis of dose and contour data reveal nonlinear behaviors which we further deploy enhance prediction accuracy. Conclusion: The present work puts forth a novel predictive model for dose volume based on anatomical scan. Deep CAEs extract a subset of representative features that are subsequently used to derive a contour-to-dose relation map. The resultant dose map serves as a prior to facilitate treatment planning.

Prediction of recombinant protein overexpression in Escherichia coli using a machine learning based model (RPOLP)

Recombinant protein overexpression, an important biotechnological process, is ruled by complex biological rules which are mostly unknown, is in need of an intelligent algorithm so as to avoid resource-intensive lab-based trial and error experiments in order to determine the expression level of the recombinant protein. The purpose of this study is to propose a predictive model to estimate the level of recombinant protein overexpression for the first time in the literature using a machine learning approach based on the sequence, expression vector, and expression host. The expression host was confined to Escherichia coli which is the most popular bacterial host to overexpress recombinant proteins. To provide a handle to the problem, the overexpression level was categorized as low, medium and high. A set of features which were likely to affect the overexpression level was generated based on the known facts (e.g. gene length) and knowledge gathered from related literature. Then, a representative sub-set of features generated in the previous objective was determined using feature selection techniques. Finally a predictive model was developed using random forest classifier which was able to adequately classify the multi-class imbalanced small dataset constructed. The result showed that the predictive model provided a promising accuracy of 80% on average, in estimating the overexpression level of a recombinant protein.

A graph theoretic approach for unsupervised feature selection

Feature subset selection is a major problem in data mining which can help to reduce computation time, improve prediction performance, and build understandable models. Specifically, feature selection realized in the absence of class labels, namely unsupervised feature selection, is challenging and interesting. In this paper a novel graph-theoretic approach for unsupervised feature selection has been proposed. The proposed method works in three steps. In the first step, the entire feature set is represented as a weighted graph. In the second step, the features are divided into several clusters using a community detection algorithm and finally in the third step, a novel iterative search strategy based on node centrality is developed to select the final subset of features. The proposed feature selection method offers two major advantages: first, our method groups features into different clusters based on their similarities, in which the features in the same cluster are similar to each other, and to obtain the reduced redundancy set, the final subset of features is selected from different clusters. Second, the node centrality measure and term variance are used to identify the most representative and informative feature subset; hence, the optimal size of the feature subset can be automatically determined. The performance of the proposed method has been compared to those of the state-of-the-art unsupervised and supervised feature selection methods on eight benchmark classification problems. The results show that our method has produced consistently better classification accuracies.

Drug activity prediction using multiple-instance learning via joint instance and feature selection

BackgroundIn drug discovery and development, it is crucial to determine which conformers (instances) of a given molecule are responsible for its observed biological activity and at the same time to recognize the most representative subset of features (molecular descriptors). Due to experimental difficulty in obtaining the bioactive conformers, computational approaches such as machine learning techniques are much needed. Multiple Instance Learning (MIL) is a machine learning method capable of tackling this type of problem. In the MIL framework, each instance is represented as a feature vector, which usually resides in a high-dimensional feature space. The high dimensionality may provide significant information for learning tasks, but at the same time it may also include a large number of irrelevant or redundant features that might negatively affect learning performance. Reducing the dimensionality of data will hence facilitate the classification task and improve the interpretability of the model.ResultsIn this work we propose a novel approach, named multiple instance learning via joint instance and feature selection. The iterative joint instance and feature selection is achieved using an instance-based feature mapping and 1-norm regularized optimization. The proposed approach was tested on four biological activity datasets.ConclusionsThe empirical results demonstrate that the selected instances (prototype conformers) and features (pharmacophore fingerprints) have competitive discriminative power and the convergence of the selection process is also fast.

Selecting feature subset for high dimensional data via the propositional FOIL rules

Feature interaction is an important issue in feature subset selection. However, most of the existing algorithms only focus on dealing with irrelevant and redundant features. In this paper, a propositional FOIL rule based algorithm FRFS, which not only retains relevant features and excludes irrelevant and redundant ones but also considers feature interaction, is proposed for selecting feature subset for high dimensional data. FRFS first merges the features appeared in the antecedents of all FOIL rules, achieving a candidate feature subset which excludes redundant features and reserves interactive ones. Then, it identifies and removes irrelevant features by evaluating features in the candidate feature subset with a new metric CoverRatio, and obtains the final feature subset. The efficiency and effectiveness of FRFS are extensively tested upon both synthetic and real world data sets, and it is compared with other six representative feature subset selection algorithms, including CFS, FCBF, Consistency, Relief-F, INTERACT, and the rule-based FSBAR, in terms of the number of selected features, runtime and the classification accuracies of the four well-known classifiers including Naive Bayes, C4.5, PART and IB1 before and after feature selection. The results on the five synthetic data sets show that FRFS can effectively identify irrelevant and redundant features while reserving interactive ones. The results on the 35 real world high dimensional data sets demonstrate that compared with other six feature selection algorithms, FRFS cannot only efficiently reduce the feature space, but also can significantly improve the performance of the four well-known classifiers.

On the development of an automatic voice pleasantness classification and intensity estimation system

In the last few years, the number of systems and devices that use voice based interaction has grown significantly. For a continued use of these systems, the interface must be reliable and pleasant in order to provide an optimal user experience. However there are currently very few studies that try to evaluate how pleasant is a voice from a perceptual point of view when the final application is a speech based interface. In this paper we present an objective definition for voice pleasantness based on the composition of a representative feature subset and a new automatic voice pleasantness classification and intensity estimation system. Our study is based on a database composed by European Portuguese female voices but the methodology can be extended to male voices or to other languages. In the objective performance evaluation the system achieved a 9.1% error rate for voice pleasantness classification and a 15.7% error rate for voice pleasantness intensity estimation.

Genetic algorithm-based efficient feature selection for classification of pre-miRNAs

In order to classify the real/pseudo human precursor microRNA (pre-miRNAs) hairpins with ab initio methods, numerous features are extracted from the primary sequence and second structure of pre-miRNAs. However, they include some redundant and useless features. It is essential to select the most representative feature subset; this contributes to improving the classification accuracy. We propose a novel feature selection method based on a genetic algorithm, according to the characteristics of human pre-miRNAs. The information gain of a feature, the feature conservation relative to stem parts of pre-miRNA, and the redundancy among features are all considered. Feature conservation was introduced for the first time. Experimental results were validated by cross-validation using datasets composed of human real/pseudo pre-miRNAs. Compared with microPred, our classifier miPredGA, achieved more reliable sensitivity and specificity. The accuracy was improved nearly 12%. The feature selection algorithm is useful for constructing more efficient classifiers for identification of real human pre-miRNAs from pseudo hairpins.

An Entropy-based gene selection method for cancer classification using microarray data

BackgroundAccurate diagnosis of cancer subtypes remains a challenging problem. Building classifiers based on gene expression data is a promising approach; yet the selection of non-redundant but relevant genes is difficult.The selected gene set should be small enough to allow diagnosis even in regular clinical laboratories and ideally identify genes involved in cancer-specific regulatory pathways. Here an entropy-based method is proposed that selects genes related to the different cancer classes while at the same time reducing the redundancy among the genes.ResultsThe present study identifies a subset of features by maximizing the relevance and minimizing the redundancy of the selected genes. A merit called normalized mutual information is employed to measure the relevance and the redundancy of the genes. In order to find a more representative subset of features, an iterative procedure is adopted that incorporates an initial clustering followed by data partitioning and the application of the algorithm to each of the partitions. A leave-one-out approach then selects the most commonly selected genes across all the different runs and the gene selection algorithm is applied again to pare down the list of selected genes until a minimal subset is obtained that gives a satisfactory accuracy of classification.The algorithm was applied to three different data sets and the results obtained were compared to work done by others using the same data setsConclusionThis study presents an entropy-based iterative algorithm for selecting genes from microarray data that are able to classify various cancer sub-types with high accuracy. In addition, the feature set obtained is very compact, that is, the redundancy between genes is reduced to a large extent. This implies that classifiers can be built with a smaller subset of genes.

A design scheme for a hierarchical fuzzy pattern matching classifier and its application to the tire tread pattern recognition

Abstract We propose a design scheme for a hierarchical fuzzy pattern matching classifier (HFPMC) and apply it to the tire tread pattern recognition problem. In this design scheme, a binary decision tree is constructed at first by using fuzzy C-means (FCM) algorithm. At each node, a representative subset of features which can split best the labelled data into two dissimilar groups is selected from all the available features on the base of cluster validity. The cluster validity is evaluated under the two criteria. The one is the polarization degree, and the other is whether all the samples of a class belong to the same cluster or not. Then, a hierarchical cluster structure for the HFPMC is reconstructed by combining the successive nodes formed by the same representative subset of features. As the hierarchical classifier, is used a fuzzy pattern matching classifier in which the designer's intuitive knowledges about the pattern recognition problem can be easily incorporated. At each subhierarchy, the reference fuzzy sets and prototypes for the HFPMC are defined based on the cluster centers of the corresponding subhierarchy. The proposed design scheme is applied to the design of a HFPMC for the tire tread pattern recognition. The design procedure including feature extraction is described in detail. Experimental results show the usefulness of the proposed design scheme.