Combination Of Feature Selection Methods Research Articles

PCM-SABRE: a platform for benchmarking and comparing outcome prediction methods in precision cancer medicine

BackgroundNumerous publications attempt to predict cancer survival outcome from gene expression data using machine-learning methods. A direct comparison of these works is challenging for the following reasons: (1) inconsistent measures used to evaluate the performance of different models, and (2) incomplete specification of critical stages in the process of knowledge discovery. There is a need for a platform that would allow researchers to replicate previous works and to test the impact of changes in the knowledge discovery process on the accuracy of the induced models.ResultsWe developed the PCM-SABRE platform, which supports the entire knowledge discovery process for cancer outcome analysis. PCM-SABRE was developed using KNIME. By using PCM-SABRE to reproduce the results of previously published works on breast cancer survival, we define a baseline for evaluating future attempts to predict cancer outcome with machine learning. We used PCM-SABRE to replicate previous work that describe predictive models of breast cancer recurrence, and tested the performance of all possible combinations of feature selection methods and data mining algorithms that was used in either of the works. We reconstructed the work of Chou et al. observing similar trends – superior performance of Probabilistic Neural Network (PNN) and logistic regression (LR) algorithms and inconclusive impact of feature pre-selection with the decision tree algorithm on subsequent analysis.ConclusionsPCM-SABRE is a software tool that provides an intuitive environment for rapid development of predictive models in cancer precision medicine.

Using an ensemble classifier based on sequential floating forward selection for financial distress prediction problem

Financial distress prediction (FDP) is a significant issue investigated by researchers, credit institutions and banks. Although extensive research has been conducted in this area, applications of combined feature selection (FS) methods and classification models are subjects that have been addressed intensely in recent years. One of the most important issues in the FDP problem is to employ an effective FS algorithm, leading to an acceptable level of performance accuracy in the implementation stage. Hence, this study primarily attempted to introduce a precise FS model and compared the obtained results with those of other conventional models tackling FDP in terms of accuracy. The proposed method involved the sequential floating forward selection (SFFS) algorithm applied as a wrapper FS technique to determine the best subset of features. At the classification stage, the support vector machine (SVM), owing to its good performance, demonstrated in numerous studies, in solving classification problems, was deployed. The performance of the proposed method was compared with those of other current well-known FS methods including artificial bee colony (ABC), genetic algorithm (GA) and sequential forward selection (SFS) (all of which are categorized under wrapper methods), and principal component analysis (PCA), relief and information gain (IG) (best known as filter techniques) for our given datasets. The results indicated that a combined model of SVM based on the SFFS approach can yield greater accuracy than the other methods applied for our defined domestic and foreign datasets. Therefore, the SFFS-SVM ensemble classifier can be considered a promising addition to existent models when confronting the FDP issue.

Feature Selection as a Time and Cost-Saving Approach for Land Suitability Classification (Case Study of Shavur Plain, Iran)

Land suitability classification is important in planning and managing sustainable land use. Most approaches to land suitability analysis combine a large number of land and soil parameters, and are time-consuming and costly. In this study, a potentially useful technique (combined feature selection and fuzzy-AHP method) to increase the efficiency of land suitability analysis was presented. To this end, three different feature selection algorithms—random search, best search and genetic methods—were used to determine the most effective parameters for land suitability classification for the cultivation of barely in the Shavur Plain, southwest Iran. Next, land suitability classes were calculated for all methods by using the fuzzy-AHP approach. Salinity (electrical conductivity (EC)), alkalinity (exchangeable sodium percentage (ESP)), wetness and soil texture were selected using the random search method. Gypsum, EC, ESP, and soil texture were selected using both the best search and genetic methods. The result shows a strong agreement between the standard fuzzy-AHP methods and methods presented in this study. The values of Kappa coefficients were 0.82, 0.79 and 0.79 for the random search, best search and genetic methods, respectively, compared with the standard fuzzy-AHP method. Our results indicate that EC, ESP, soil texture and wetness are the most effective features for evaluating land suitability classification for the cultivation of barely in the study area, and uses of these parameters, together with their appropriate weights as obtained from fuzzy-AHP, can perform good results for land suitability classification. So, the combined feature selection presented and the fuzzy-AHP approach has the potential to save time and money for land suitability classification.

EEG-based mild depressive detection using feature selection methods and classifiers

Background and objectiveDepression has become a major health burden worldwide, and effectively detection of such disorder is a great challenge which requires latest technological tool, such as Electroencephalography (EEG). This EEG-based research seeks to find prominent frequency band and brain regions that are most related to mild depression, as well as an optimal combination of classification algorithms and feature selection methods which can be used in future mild depression detection. MethodsAn experiment based on facial expression viewing task (Emo_block and Neu_block) was conducted, and EEG data of 37 university students were collected using a 128 channel HydroCel Geodesic Sensor Net (HCGSN). For discriminating mild depressive patients and normal controls, BayesNet (BN), Support Vector Machine (SVM), Logistic Regression (LR), k-nearest neighbor (KNN) and RandomForest (RF) classifiers were used. And BestFirst (BF), GreedyStepwise (GSW), GeneticSearch (GS), LinearForwordSelection (LFS) and RankSearch (RS) based on Correlation Features Selection (CFS) were applied for linear and non-linear EEG features selection. Independent Samples T-test with Bonferroni correction was used to find the significantly discriminant electrodes and features. ResultsData mining results indicate that optimal performance is achieved using a combination of feature selection method GSW based on CFS and classifier KNN for beta frequency band. Accuracies achieved 92.00% and 98.00%, and AUC achieved 0.957 and 0.997, for Emo_block and Neu_block beta band data respectively. T-test results validate the effectiveness of selected features by search method GSW. Simplified EEG system with only FP1, FP2, F3, O2, T3 electrodes was also explored with linear features, which yielded accuracies of 91.70% and 96.00%, AUC of 0.952 and 0.972, for Emo_block and Neu_block respectively. ConclusionsClassification results obtained by GSW + KNN are encouraging and better than previously published results. In the spatial distribution of features, we find that left parietotemporal lobe in beta EEG frequency band has greater effect on mild depression detection. And fewer EEG channels (FP1, FP2, F3, O2 and T3) combined with linear features may be good candidates for usage in portable systems for mild depression detection.

Analysis of network traffic features for anomaly detection

Anomaly detection in communication networks provides the basis for the uncovering of novel attacks, misconfigurations and network failures. Resource constraints for data storage, transmission and processing make it beneficial to restrict input data to features that are (a) highly relevant for the detection task and (b) easily derivable from network observations without expensive operations. Removing strong correlated, redundant and irrelevant features also improves the detection quality for many algorithms that are based on learning techniques. In this paper we address the feature selection problem for network traffic based anomaly detection. We propose a multi-stage feature selection method using filters and stepwise regression wrappers. Our analysis is based on 41 widely-adopted traffic features that are presented in several commonly used traffic data sets. With our combined feature selection method we could reduce the original feature vectors from 41 to only 16 features. We tested our results with five fundamentally different classifiers, observing no significant reduction of the detection performance. In order to quantify the practical benefits of our results, we analyzed the costs for generating individual features from standard IP Flow Information Export records, available at many routers. We show that we can eliminate 13 very costly features and thus reducing the computational effort for on-line feature generation from live traffic observations at network nodes.

Novel feature selection method based on harmony search for email classification

Feature selection is often used in email classification to reduce the dimensionality of the feature space. In this study, a new document frequency and term frequency combined feature selection method (DTFS) is proposed to improve the performance of email classification. Firstly, an existing optimal document frequency based feature selection method (ODFFS) and a predetermined threshold are applied to select the most discriminative features. Secondly, an existing optimal term frequency based feature selection (OTFFS) method and another predetermined threshold are applied to select more discriminative features. Finally, ODFFS and OTFFS are combined to select the remaining features. In order to improve the convergence rate of parameter optimization, a metaheuristic method, namely global best harmony oriented harmony search (GBHS), is proposed to search these optimal predetermined thresholds. Experiments with fuzzy Support Vector Machine (FSVM) and Naïve Bayesian (NB) classifiers are applied on six corpuses: PU2, CSDMC2010, PU3, Lingspam, Enron-spam and Trec2007. Experimental results show that, DTFS outperforms other methods: such as Chi-squre, comprehensively measure feature selection, t-test based feature selection, term frequency based information gain, two-step based hybrid feature selection method and improved term frequency inverse document frequency method on six corpuses.

Abstract 1485: Biomarker development for lung cancer diagnosis using integrative microRNA and gene expression networks

Abstract Introduction: We have previously shown that smoking creates a “molecular field of injury” throughout the epithelial cells that line the respiratory tract and that gene-expression alterations in the cytologically-normal mainstem bronchus epithelium can serve as an early detection biomarker for lung cancer. We hypothesize that microRNAs (miRNAs) regulate these airway gene expression changes and that miRNA expression differences in this tissue can be used as biomarkers for lung cancer diagnosis. Methods: We profiled miRNA expression via small RNA sequencing in bronchial epithelial brushes collected from the mainstem bronchus of 230 subjects undergoing bronchoscopy for suspect lung cancer and gene expression (mRNA) via microarray for 201 matched samples. Bilal et al. (2013) have shown that incorporating biological knowledge into model building improves prediction. Therefore, we sought to test the hypothesis that including information about the expression levels of the predicted mRNA targets of miRNA may improve miRNA feature selection and aid in interpretation of signatures. We used mirConnX which combines miRNA with mRNA data to create disease-specific, genome-wide regulatory networks. Results: First, we show that there is a miRNA expression signal for cancer: across many combinations of feature selection methods, predictive models and different parameters within 100 bootstraps to predict cancer phenotype, we find the AUC values obtained are consistently higher compared to the random control procedure where we randomly shuffle the class labels (p < 0.001). Second, using a training set of 106 samples, we built networks separately for cancer and non-cancer samples, using 10-fold cross validation in order to determine robust cancer and non-cancer specific features. The disease-state specific networks are then aggregated by taking the overlapping features across the ten folds. Next, we select the non-overlapping features (miRNAs and genes) between cancer and non-cancer as those that capture the difference between the two phenotypes. The selected genes are enriched for relevant cancer related pathways, including KEGG pathways in cancer (p = 0.0003), cell cycle (p = 0.008), WNT signaling (p = 0.0001), basal cell carcinoma (p = 0.023), MAPK signaling pathway (p = 0.024), TGF-beta signaling pathway (p = 0.00014), p53 signaling pathway (p = 0.087) etc. Most importantly, the miRNA features from these disease specific networks are found to have higher predictive power (highest AUC 0.72) compared to all miRNA features (highest AUC 0.57), on a second set of 68 samples in cross-validation. Conclusion: Using novel integrative analysis, we improved miRNA biomarker prediction. This is the first report of cancer-associated miRNA expression differences in cytologically normal bronchial epithelium, for lung cancer diagnosis, and it extends our previous work focused on mRNA biomarkers from this tissue. Citation Format: Ana Pavel, Joshua Campbell, Gang Liu, Sherry Zhang, Hanqiao Liu, Steven Dubinett, David Elashoff, Kate Porta, Duncan Whitney, Marc Lenburg, Avrum Spira. Biomarker development for lung cancer diagnosis using integrative microRNA and gene expression networks. [abstract]. In: Proceedings of the 105th Annual Meeting of the American Association for Cancer Research; 2014 Apr 5-9; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2014;74(19 Suppl):Abstract nr 1485. doi:10.1158/1538-7445.AM2014-1485

Term frequency combined hybrid feature selection method for spam filtering

Feature selection is an important technology on improving the efficiency and accuracy of spam filtering. Among the numerous methods, document frequency-based feature selections ignore the effect of term frequency information, thus always deduce unsatisfactory results. In this paper, a hybrid method (called HBM), which combines the document frequency information and term frequency information is proposed. To maintain the category distinguishing ability of the selected features, an optimal document frequency-based feature selection (called ODFFS) is chosen; terms which are indeed discriminative will be selected by ODFFS. For the remaining features, term frequency information is considered and the terms with the highest HBM values are selected. Further, a novel method called feature subset evaluating parameter optimization (FSEPO) is proposed for parameter optimization. Experiments with support vector machine (SVM) and Naive Bayesian (NB) classifiers are applied on four corpora: PU1, LingSpam, SpamAssian and Trec2007. Six feature selections: information gain, Chi square, improved Gini-index, multi-class odds ratio, normalized term frequency-based discriminative power measure and comprehensively measure feature selection are compared with HBM. Experimental results show that, HBM is significantly superior to other feature selection methods on four corpora when SVM and NB are applied, respectively.

Hybrid methodologies for summarisation of Kannada language text documents

The problem of information explosion is becoming a serious concern. In this regard, any new methodology developed to solve the issues related to information retrieval (a.k.a. Information Retrieval or IR) draws wide attention. Text summarisation is a predominant field of NLP which may provide promising solution to the issues stated earlier. Text summarisation or text document summarisation provides a quick and concise meaning of the document without even reading the whole document. In this work, we have developed hybrid methodologies for providing summary of a given document in the Kannada language. The approach is new as we have used combination of feature selection methods as a pre-processing step for summarisation. In this work, we have devised four different methodologies for text document summarisation, which focus on text extraction, which is an open approach as stated earlier: (a) summarisation based on keywords, (b) summarisation based on sentence ranking, (c) summarisation based on Jaccards' similarity score and (d) summarisation based on neural network approach.

Microarray gene-expression data classification using less gene expressions by combining feature selection methods and classifiers

Microarray Data, often characterised by high-dimensions and small samples, is used for cancer classification problems that classify the given (tissue) samples as deceased or healthy on the basis of analysis of gene expression profile. The goal of feature selection is to search the most relevant features from thousands of related features of a particular problem domain. The focus of this study is a method that relaxes the maximum accuracy criterion for feature selection and selects the combination of feature selection method and classifier that using small subset of features obtains accuracy not statistically indicatively different than the maximum accuracy. By selecting the classifier employing small number of features along with a good accuracy, the risk of over fitting (bias) is reduced. This has been corroborated empirically using some common attribute selection methods (ReliefF, SVM-RFE, FCBF, and Gain Ratio) and classifiers (3 Nearest Neighbour, Naive Bayes and SVM) applied to 6 different microarray cancer data sets. We use hypothesis testing to compare several configurations and select particular configurations that perform well with small genes on these data sets.

COMBINATION OF MULTIPLE FEATURE SELECTION METHODS FOR TEXT CATEGORIZATION BY USING COMBINATORIAL FUSION ANALYSIS AND RANK-SCORE CHARACTERISTIC

Effective feature selection methods are important for improving the efficiency and accuracy of text categorization algorithms by removing redundant and irrelevant terms from the corpus. Extensive research has been done to improve the performance of individual feature selection methods. However, it is always a challenge to come up with an individual feature selection method which would outperform other methods in most cases. In this paper, we explore the possibility of improving the overall performance by combining multiple individual feature selection methods. In particular, we propose a method of combining multiple feature selection methods by using an information fusion paradigm, called Combinatorial Fusion Analysis (CFA). A rank-score function and its associated graph, called rank-score graph, are adopted to measure the diversity of different feature selection methods. Our experimental results demonstrated that a combination of multiple feature selection methods can outperform a single method only if each individual feature selection method has unique scoring behavior and relatively high performance. Moreover, it is shown that the rank-score function and rank-score graph are useful for the selection of a combination of feature selection methods.

Comparative Study on Feature Selection in Uighur Text Categorization

In this paper, the methods of classifying the Uighur language feature extraction have been studied. According to the feature that the Uighur language belongs to adhesion, three experiments were designed to inspect the influence on the accuracy of text classification by using different methods of feature extraction, the first experiment was designed to inspect the accuracy of text classification in case of stem segmentation by using the traditional methods of feature extraction, such as DF,IG,MI,CHI. The results show that the best classification accuracy rate is 91.34% by the method of DF feature extraction, while the best accuracy rate is 88.03% in the second experiment by the method of CHI feature extraction in the case of stem that are not segmented. The third experiment uses combination of feature selection methods, such as DF+IG,DF+MI,DF+CHI, and the result show that the accuracy rate of classification is 93.57% by the method of DF+CHI feature selection, which shows that it is the best method in all experiments.

Combining multiple feature selection methods for stock prediction: Union, intersection, and multi-intersection approaches

To effectively predict stock price for investors is a very important research problem. In literature, data mining techniques have been applied to stock (market) prediction. Feature selection, a pre-processing step of data mining, aims at filtering out unrepresentative variables from a given dataset for effective prediction. As using different feature selection methods will lead to different features selected and thus affect the prediction performance, the purpose of this paper is to combine multiple feature selection methods to identify more representative variables for better prediction. In particular, three well-known feature selection methods, which are Principal Component Analysis (PCA), Genetic Algorithms (GA) and decision trees (CART), are used. The combination methods to filter out unrepresentative variables are based on union, intersection, and multi-intersection strategies. For the prediction model, the back-propagation neural network is developed. Experimental results show that the intersection between PCA and GA and the multi-intersection of PCA, GA, and CART perform the best, which are of 79% and 78.98% accuracy respectively. In addition, these two combined feature selection methods filter out near 80% unrepresentative features from 85 original variables, resulting in 14 and 17 important features respectively. These variables are the important factors for stock prediction and can be used for future investment decisions.

P.2.d.019 Prefrontal oscillation associated with randomised, placebo-controlled treatment with agomelatine and psychotherapy

Depression has become a major health burden worldwide, and effectively detection of such disorder is a great challenge which requires latest technological tool, such as Electroencephalography (EEG). This EEG-based research seeks to find prominent frequency band and brain regions that are most related to mild depression, as well as an optimal combination of classification algorithms and feature selection methods which can be used in future mild depression detection.An experiment based on facial expression viewing task (Emo_block and Neu_block) was conducted, and EEG data of 37 university students were collected using a 128 channel HydroCel Geodesic Sensor Net (HCGSN). For discriminating mild depressive patients and normal controls, BayesNet (BN), Support Vector Machine (SVM), Logistic Regression (LR), k-nearest neighbor (KNN) and RandomForest (RF) classifiers were used. And BestFirst (BF), GreedyStepwise (GSW), GeneticSearch (GS), LinearForwordSelection (LFS) and RankSearch (RS) based on Correlation Features Selection (CFS) were applied for linear and non-linear EEG features selection. Independent Samples T-test with Bonferroni correction was used to find the significantly discriminant electrodes and features.Data mining results indicate that optimal performance is achieved using a combination of feature selection method GSW based on CFS and classifier KNN for beta frequency band. Accuracies achieved 92.00% and 98.00%, and AUC achieved 0.957 and 0.997, for Emo_block and Neu_block beta band data respectively. T-test results validate the effectiveness of selected features by search method GSW. Simplified EEG system with only FP1, FP2, F3, O2, T3 electrodes was also explored with linear features, which yielded accuracies of 91.70% and 96.00%, AUC of 0.952 and 0.972, for Emo_block and Neu_block respectively.Classification results obtained by GSW + KNN are encouraging and better than previously published results. In the spatial distribution of features, we find that left parietotemporal lobe in beta EEG frequency band has greater effect on mild depression detection. And fewer EEG channels (FP1, FP2, F3, O2 and T3) combined with linear features may be good candidates for usage in portable systems for mild depression detection.

A high-performance extraction method for public opinion on internet

Aiming at the importance of the analysis for public opinion on Internet, the authors propose a high-performance extraction method for public opinion. In this method, the space model for classification is adopted to describe the relationship between words and categories. The combined feature selection method is used to remove noisy words from the original feature space effectively. Then the category weight of words is calculated by the improved formula combining the frequency of words and distribution of words. Finally, the class weights of the not-categorized documents based on the category weight of words are obtained for realizing opinion extraction. Experiment results show that the method has comparatively high classification and good stability.

특징 추출과 분석 기법에 기반한 단백질 상호작용 데이터 신뢰도 향상 시스템

대용량 실험으로부터 산출된 단백질 상호작용 데이터는 위양성(false positive) 데이터의 비율이 높다는 단점을 가지고 있다. 본 논문에서는 오류가 섞여있는 단백질 상호작용 데이터를 입력으로 받아 각 단백질 상호작용의 신뢰도를 검증하는 시스템을 제안하고 구현하였다. 제안 시스템은 단백질 상호작용 데이터에 상호작용의 근거로서 사용될 수 있는 다양한 생물학적 특징들에 관한 데이터를 통합하고 특징 선택 방법을 사용하여 통합된 속성들 중 위양성 여부를 판별하는데 가장 적합한 특징들을 선택한 후 데이터 마이닝 분류 알고리즘을 적용하여 대용량 실험으로부터 산출된 단백질 상호작용 데이터의 신뢰도를 평가한다. 특징 선택의 결과와 분류 기법의 성능은 데이터 특성에 매우 의존하므로, 제안시스템에 가장 적합한 속성 부분집합과 가장 좋은 성능을 내는 분류 알고리즘을 찾기 위해 다양한 특징 선택 방법과 데이터 마이닝 분류 알고리즘들을 적용하고 그 성능을 다각적으로 비교분석 하였다. 실험 결과, 특징 선택 방법과 분류 알고리즘을 결합시킨 제안 시스템은 오류 데이터가 섞여있는 단백질 상호작용 데이터에서 실제로 상호작용하는 단백질 쌍을 골라내는 작업에 있어 기존 연구들에 비해 매우 뛰어난 성능을 보여줬다. 또한 본 연구를 통해 단백질 상호작용 데이터의 신뢰도를 검증함에 있어서 다양한 특징 선택 방법들과 분류 알고리즘들이 성능에 미치는 영향에 관해서도 정리할 수 있었다. Protein-protein interaction data obtained from high-throughput experiments includes high false positives. In this paper, we introduce a new protein-protein interaction reliability verification system. The proposed system integrates various biological features related with protein-protein interactions, and then selects the most relevant and informative features among them using a feature selection method. To assess the reliability of each protein-protein interaction data, the system construct a classifier that can distinguish true interacting protein pairs from noisy protein-protein interaction data based on the selected biological evidences using a classification technique. Since the performance of feature selection methods and classification techniques depends heavily upon characteristics of data, we performed rigorous comparative analysis of various feature selection methods and classification techniques to obtain optimal performance of our system. Experimental results show that the combination of feature selection method and classification algorithms provide very powerful tools in distinguishing true interacting protein pairs from noisy protein-protein interaction dataset. Also, we investigated the effects on performances of feature selection methods and classification techniques in the proposed protein interaction verification system.