Fast Correlation Based Feature Selection Research Articles

Augmentation in performance and security of WSNs for IoT applications using feature selection and classification techniques

Energy consumption, secure connection of Sensor Nodes (SNs), and their performance analysis play a major role in Wireless Sensor Networks (WSNs) research. Increased case use of various Internet of Things (IoT) applications has led to the development of complicated networks. To address the threats and security issues faced in complicated WSNs, a novel combined feature selection method known as the Fast Correlation-based Feature Selection (FCBFS) with XG-Boost has been proposed for the NSL-KDD intrusion detection benchmark dataset. It helps in the selection of the best features in a cluster-based WSN before the classification step. Five popular machine learning-based classifiers, namely decision tree, random forest, Naïve Bayes, extra tree, and XG-Boost, are used for the development of a robust intrusion detection system in WSN and its IoT applications. The effectiveness and robustness of the developed ensemble methods have been checked using classification accuracy, precision, recall, and F-score. XG-Boost classifier along with the FCBFS process has performed best with a classic accuracy, precision, recall, and F-score up to 99.84%, 99.83%, 99.84%, and 99.82% on multiple runs. Upon comparison with the existing state-of-the-art work in this field, the proposed work has outperformed. Results show that, in contrast to the previous filter approaches, our proposed process can successfully minimize the number of features while maintaining a high classification precision and recognition rate. It further tends to lower the overall energy required by sensor nodes during attack detection, hence extending the network lifetime and usefulness to a sufficient time frame.

Optimal Combination of Multivariate Filter Feature Selection and Classifier for Speech-Based Depression Detection

In this paper, we have focused to improve the performance of a speech-based uni-modal depression detection system, which is non-invasive, involves low cost and computation time in comparison to multi-modal systems. The performance of a decision system mainly depends on the choice of feature selection method and the classifier. We have investigated the combination of four well-known multivariate filter methods (minimum Redundancy Maximum Relevance, Scatter Ratio, Mahalanobis Distance, Fast Correlation Based feature selection) and four well-known classifiers (k-Nearest Neighbour, Linear Discriminant classifier, Decision Tree, Support Vector Machine) to obtain a minimal set of relevant and non-redundant features to improve the performance. This will speed up the acquisition of features from speech and build the decision system with low cost and complexity. Experimental results on the high and low-level features of recent work on the DAICWOZ dataset demonstrate the superior performance of the combination of Scatter Ratio and LDC as well as that of Mahalanobis Distance and LDC, in comparison to other combinations and existing speech-based depression results, for both gender independent and gender-based studies. Further, these combinations have also outperformed a few multimodal systems. It was noted that low-level features are more discriminatory and provide a better f1 score.

Differentiation of paediatric posterior fossa tumours by the multiregional and multiparametric MRI radiomics approach: a study on the selection of optimal multiple sequences and multiregions.

To evaluate the diagnostic performance of a radiomics model based on multiregional and multiparametric MRI to classify paediatric posterior fossa tumours (PPFTs), explore the contribution of different MR sequences and tumour subregions in tumour classification, and examine whether contrast-enhanced T1 weighted (T1C) images have irreplaceable added value. This retrospective study of 136 PPFTs extracted 11,958 multiregional (enhanced, non-enhanced, and total tumour) features from multiparametric MRI (T1- and T2 weighted, T1C, fluid-attenuated inversion recovery, and diffusion-weighted images). These features were subjected to fast correlation-based feature selection and classified by a support vector machine based on different tasks. Diagnostic performances of multiregional and multiparametric MRI features, different sequences, and different tumoral regions were evaluated using multiclass and one-vs-rest strategies. The established model achieved an overall area under the curve (AUC) of 0.977 in the validation cohort. The performance of PPFTs significantly improved after replacing T1C with apparent diffusion coefficient maps added into the plain scan sequences (AUC from 0.812 to 0.917). When oedema features were added to contrast-enhancing tumour volume, the performance did not significantly improve. The radiomics model built by multiregional and multiparametric MRI features allows for the excellent distinction of different PPFTs and provides valuable references for the rational adoption of MR sequences. This study emphasized that T1C has limited added value in predicting PPFTs and should be cautiously adopted. Selecting optimal MR sequences may help guide clinicians to better allocate acquisition sequences and reduce medical costs.

Cancer Classification with a Cost-Sensitive Naive Bayes Stacking Ensemble.

Ensemble learning combines multiple learners to perform combinatorial learning, which has advantages of good flexibility and higher generalization performance. To achieve higher quality cancer classification, in this study, the fast correlation-based feature selection (FCBF) method was used to preprocess the data to eliminate irrelevant and redundant features. Then, the classification was carried out in the stacking ensemble learner. A library for support vector machine (LIBSVM), K-nearest neighbor (KNN), decision tree C4.5 (C4.5), and random forest (RF) were used as the primary learners of the stacking ensemble. Given the imbalanced characteristics of cancer gene expression data, the embedding cost-sensitive naive Bayes was used as the metalearner of the stacking ensemble, which was represented as CSNB stacking. The proposed CSNB stacking method was applied to nine cancer datasets to further verify the classification performance of the model. Compared with other classification methods, such as single classifier algorithms and ensemble algorithms, the experimental results showed the effectiveness and robustness of the proposed method in processing different types of cancer data. This method may therefore help guide cancer diagnosis and research.

Extracting salient information from discarded features via attribute selection and pruning

This paper introduces a novel two-step proposal for attribute subset selection. The first step is able to extract handy information from non-selected features after an initial feature subset selection to be added to the preliminary subset selected; for this step the feature subset selector relies on the Correlation-based Feature Selection (CFS). The second step tries to prune the solution given that in the first step some extra attributes out of initially non-selected features have been added to the relevant features; the ultimate goal of this pruning is to optimise the attribute subset prior to conduct a classification task. In accordance with the experimental results, the simplification of solutions is effective since an improvement takes place in most of the cases where the new method does determine some attributes to be removed. Some performance measures are reported within a test bed composed of a good number of binary and multi-class classification problems. Comparisons with other attribute selection procedures such as CFS, Fast Correlation-based Feature Selection and Gain Ratio revealed that the new approach is very competitive and may be taken into account by the data preparation community.

Heart Disease Prediction and Classification Using Machine Learning Algorithms Optimized by Particle Swarm Optimization and Ant Colony Optimization

The prediction of heart disease is one of the areas where machine learning can be implemented. Optimization algorithms have the advantage of dealing with complex non-linear problems with a good flexibility and adaptability. In this paper, we exploited the Fast Correlation-Based Feature Selection (FCBF) method to filter redundant features in order to improve the quality of heart disease classification. Then, we perform a classification based on different classification algorithms such as K-Nearest Neighbour, Support Vector Machine, Naïve Bayes, Random Forest and a Multilayer Perception | Artificial Neural Network optimized by Particle Swarm Optimization (PSO) combined with Ant Colony Optimization (ACO) approaches. The proposed mixed approach is applied to heart disease dataset; the results demonstrate the efficacy and robustness of the proposed hybrid method in processing various types of data for heart disease classification. Therefore, this study examines the different machine learning algorithms and compares the results using different performance measures, i.e. accuracy, precision, recall, f1-score, etc. A maximum classification accuracy of 99.65% using the optimized model proposed by FCBF, PSO and ACO. The results show that the performance of the proposed system is superior to that of the classification technique presented above.

Hybrid adapted fast correlation FCBF-support vector machine recursive feature elimination for feature selection

This paper investigates feature selection methods based on hybrid architecture using feature selection algorithm called Adapted Fast Correlation Based Feature selection and Support Vector Machine Recursive Feature Elimination (AFCBF-SVMRFE). The AFCBF-SVMRFE has three stages and composed of SVMRFE embedded method with Correlation based Features Selection. The first stage is the relevance analysis, the second one is a redundancy analysis, and the third stage is a performance evaluation and features restoration stage. Experiments show that the proposed method tested on different classifiers: Support Vector Machine SVM and K nearest neighbors KNN provide a best accuracy on various dataset. The SVM classifier outperforms KNN classifier on these data. The AFCBF-SVMRFE outperforms FCBF multivariate filter, SVMRFE, Particle swarm optimization PSO and Artificial bees colony ABC.

Managing Student Performance: A Predictive Analytics using Imbalanced Data

Big data has revolutionized every field of life, which accumulates human learning as well. The field of education has progressed in past couple of decades, and addition to that, rapid growth in the number of educational institutions has created a tough competition. The massive accumulation of data in the educational sector has created a great scope of EDM (Educational Data Mining) with the support of robust predictive models. It is quite necessary to regularly examine the performance of the students to make them perform better, thus helps to maintain the reputation of the institution. This study proposed a predictive model through which the performance of the student can be forecasted depending upon various characteristics. The KDD(Knowledge Discovery in Databases) methodology was followed stepwise in this study for developing predictive models to predict student performance. The data balancing techniques such as SMOTE (Synthetic Minority Over-sampling Technique) and ADASYN (Adaptive Synthetic Sampling) were employed to handle the unbalanced effect of data which causes bias predictions. Also, for the selection of significant features techniques, FCBF (Fast Correlation Based Feature selection) and RFE (Recursive Feature Elimination) were used. The EDM algorithms Random Forest (RF), Support Vector Machine (SVM) and Artificial Neural Network (ANN) were utilized for predicting student performance with suitable hyper-parameter tuning using random search to enhance the performance of the model. The results obtained were cross-validated using Ensemble Method and benchmarked with previous studies. The random forest model achieved the highest accuracy of 86% after data balancing and careful selection of significant features.

Correlation Based Feature Selection Algorithms for Varying Datasets of Different Dimensionality

Curse of dimensionality problem needs to be addressed carefully when designing a classifier. Given a huge dimensional dataset, one interesting problem is the choice of optimal selection of features for classification. Feature selection is an interesting and most optimal solution to the curse of dimensionality problem. Numerous feature selection algorithms have been proposed in the recent past to solve the curse of dimensionality problem but no one stop solution prevails. This paper proposes two novel algorithms for feature selection namely Reverse Piece-wise Correlation Based Feature Selection (RPwCBFS) and Shuffled Piece-wise Correlation Based Feature Selection (SPwCBFS) that divides the feature space into pieces and computes the similarity of feature subsets in reverse order and in random shuffled manner respectively. The proposed algorithms are compared with Fast Correlation Based Feature selection (FCBF), Fast Correlation Based Feature selection # (FCBF#) and Fast Correlation Based Feature selection In Piece (FCBFiP). Standard medium and huge dimensional datasets are used for experimentation purpose. Experimental results prove that the Reverse Piece-wise Correlation Based Feature Selection algorithm (RPwCBFS) and Shuffled Piece-wise Correlation Based Feature Selection algorithm (SPwCBFS) are prominent solution for feature selection when the underlying dataset is medium sized. For huge dimensional datasets, Shuffled Piece-wise Correlation Based Feature Selection algorithm (SPwCBFS) proves to be an optimal choice.

A Hybrid Feature Selection Method for Complex Diseases SNPs

Machine learning techniques have the potential to revolutionize medical diagnosis. Single Nucleotide Polymorphisms (SNPs) are one of the most important sources of human genome variability; thus, they have been implicated in several human diseases. To separate the affected samples from the normal ones, various techniques have been applied on SNPs. Achieving high classification accuracy in such a high-dimensional space is crucial for successful diagnosis and treatment. In this work, we propose an accurate hybrid feature selection method for detecting the most informative SNPs and selecting an optimal SNP subset. The proposed method is based on the fusion of a filter and a wrapper method, i.e., the Conditional Mutual Information Maximization (CMIM) method and the support vector machine-recursive feature elimination, respectively. The performance of the proposed method was evaluated against four state-of-the-art feature selection methods, minimum redundancy maximum relevancy, fast correlation-based feature selection, CMIM, and ReliefF, using four classifiers, support vector machine, naive Bayes, linear discriminant analysis, and $k$ nearest neighbors on five different SNP data sets obtained from the National Center for Biotechnology Information gene expression omnibus genomics data repository. The experimental results demonstrate the efficiency of the adopted feature selection approach outperforming all of the compared feature selection algorithms and achieving up to 96% classification accuracy for the used data set. In general, from these results we conclude that SNPs of the whole genome can be efficiently employed to distinguish affected individuals with complex diseases from the healthy ones.

Cancer Classification Based on Support Vector Machine Optimized by Particle Swarm Optimization and Artificial Bee Colony

Intelligent optimization algorithms have advantages in dealing with complex nonlinear problems accompanied by good flexibility and adaptability. In this paper, the FCBF (Fast Correlation-Based Feature selection) method is used to filter irrelevant and redundant features in order to improve the quality of cancer classification. Then, we perform classification based on SVM (Support Vector Machine) optimized by PSO (Particle Swarm Optimization) combined with ABC (Artificial Bee Colony) approaches, which is represented as PA-SVM. The proposed PA-SVM method is applied to nine cancer datasets, including five datasets of outcome prediction and a protein dataset of ovarian cancer. By comparison with other classification methods, the results demonstrate the effectiveness and the robustness of the proposed PA-SVM method in handling various types of data for cancer classification.

Classification models for clear cell renal carcinoma stage progression, based on tumor RNAseq expression trained supervised machine learning algorithms.

BackgroundClear-cell Renal Cell Carcinoma (ccRCC) is the most- prevalent, chemotherapy resistant and lethal adult kidney cancer. There is a need for novel diagnostic and prognostic biomarkers for ccRCC, due to its heterogeneous molecular profiles and asymptomatic early stage. This study aims to develop classification models to distinguish early stage and late stage of ccRCC based on gene expression profiles. We employed supervised learning algorithms- J48, Random Forest, SMO and Naïve Bayes; with enriched model learning by fast correlation based feature selection to develop classification models trained on sequencing based gene expression data of RNAseq experiments, obtained from The Cancer Genome Atlas.ResultsDifferent models developed in the study were evaluated on the basis of 10 fold cross validations and independent dataset testing. Random Forest based prediction model performed best amongst the models developed in the study, with a sensitivity of 89%, accuracy of 77% and area under Receivers Operating Curve of 0.8.ConclusionsWe anticipate that the prioritized subset of 62 genes and prediction models developed in this study will aid experimental oncologists to expedite understanding of the molecular mechanisms of stage progression and discovery of prognostic factors for ccRCC tumors.

A Comparative Study on Classification of Sleep Stage Based on EEG Signals Using Feature Selection and Classification Algorithms

Sleep scoring is one of the most important diagnostic methods in psychiatry and neurology. Sleep staging is a time consuming and difficult task undertaken by sleep experts. This study aims to identify a method which would classify sleep stages automatically and with a high degree of accuracy and, in this manner, will assist sleep experts. This study consists of three stages: feature extraction, feature selection from EEG signals, and classification of these signals. In the feature extraction stage, it is used 20 attribute algorithms in four categories. 41 feature parameters were obtained from these algorithms. Feature selection is important in the elimination of irrelevant and redundant features and in this manner prediction accuracy is improved and computational overhead in classification is reduced. Effective feature selection algorithms such as minimum redundancy maximum relevance (mRMR); fast correlation based feature selection (FCBF); ReliefF; t-test; and Fisher score algorithms are preferred at the feature selection stage in selecting a set of features which best represent EEG signals. The features obtained are used as input parameters for the classification algorithms. At the classification stage, five different classification algorithms (random forest (RF); feed-forward neural network (FFNN); decision tree (DT); support vector machine (SVM); and radial basis function neural network (RBF)) classify the problem. The results, obtained from different classification algorithms, are provided so that a comparison can be made between computation times and accuracy rates. Finally, it is obtained 97.03 % classification accuracy using the proposed method. The results show that the proposed method indicate the ability to design a new intelligent assistance sleep scoring system.

Novel approaches for automated epileptic diagnosis using FCBF selection and classification algorithms

This paper presents a new application for automated epileptic detection using the fast correlation-based feature (FCBF) selection and classification algorithms. This study consists of 3 stages: feature extraction, feature selection from electroencephalography (EEG) signals, and the classification of these signals. In the feature extraction phase, 16 attribute algorithms are used in 5 categories, and 36 feature parameters are obtained from these algorithms. In the feature selection phase, the FCBF algorithm is chosen to select a set of attributes that best represent the EEG signals. The resulting attributes are used as input parameters for the classification algorithms. In the classification phase, the problem is classified with 6 different classification algorithms. The results obtained with the different classification algorithms are provided in order to compare the calculation times and the accuracy rates. The evolution of the proposed system is conducted using k-fold cross-validation, classification accuracy, sensitivity and specificity values, and a confusion matrix. The proposed approach enables 100% classification accuracy with the use of the multilayer perceptron neural network and naive Bayes algorithm. The stated results show that the proposed method is capable of designing a new intelligent assistance diagnostic system.

An efficient ensemble learning method for gene microarray classification.

The gene microarray analysis and classification have demonstrated an effective way for the effective diagnosis of diseases and cancers. However, it has been also revealed that the basic classification techniques have intrinsic drawbacks in achieving accurate gene classification and cancer diagnosis. On the other hand, classifier ensembles have received increasing attention in various applications. Here, we address the gene classification issue using RotBoost ensemble methodology. This method is a combination of Rotation Forest and AdaBoost techniques which in turn preserve both desirable features of an ensemble architecture, that is, accuracy and diversity. To select a concise subset of informative genes, 5 different feature selection algorithms are considered. To assess the efficiency of the RotBoost, other nonensemble/ensemble techniques including Decision Trees, Support Vector Machines, Rotation Forest, AdaBoost, and Bagging are also deployed. Experimental results have revealed that the combination of the fast correlation-based feature selection method with ICA-based RotBoost ensemble is highly effective for gene classification. In fact, the proposed method can create ensemble classifiers which outperform not only the classifiers produced by the conventional machine learning but also the classifiers generated by two widely used conventional ensemble learning methods, that is, Bagging and AdaBoost.

The elusive short gene – an ensemble method for recognition for prokaryotic genome

Accurate prediction of short protein coding DNA from genome sequence information remains an unsolved problem in DNA sequence analysis. Popular gene finding tools show drastic reduction in accuracy while attempting to predict genes of length less than 400nt, a length we define as short. This study performs a quantitative evaluation of a set of selected coding measures in terms of their discriminative power in recognizing short genes in prokaryotic genomes. By performing Fast Correlation Based Feature Selection (FCBF) technique, we identified a subset of coding measures with high discriminative power. Using the measures identified thus, we present a novel approach for short genes recognition. A short-gene predictor employing AdaBoost.M1 in conjunction with random forests as the base classifier gives 92.74% accuracy, 94.77% sensitivity and 90.06% specificity on short genes.