Proposed Feature Selection Method Research Articles

An implementation of normal distribution based segmentation and entropy controlled features selection for skin lesion detection and classification

BackgroundMelanoma is the deadliest type of skin cancer with highest mortality rate. However, the annihilation in its early stage implies a high survival rate therefore, it demands early diagnosis. The accustomed diagnosis methods are costly and cumbersome due to the involvement of experienced experts as well as the requirements for the highly equipped environment. The recent advancements in computerized solutions for this diagnosis are highly promising with improved accuracy and efficiency.MethodsIn this article, a method for the identification and classification of the lesion based on probabilistic distribution and best features selection is proposed. The probabilistic distribution such as normal distribution and uniform distribution are implemented for segmentation of lesion in the dermoscopic images. Then multi-level features are extracted and parallel strategy is performed for fusion. A novel entropy-based method with the combination of Bhattacharyya distance and variance are calculated for the selection of best features. Only selected features are classified using multi-class support vector machine, which is selected as a base classifier.ResultsThe proposed method is validated on three publicly available datasets such as PH2, ISIC (i.e. ISIC MSK-2 and ISIC UDA), and Combined (ISBI 2016 and ISBI 2017), including multi-resolution RGB images and achieved accuracy of 97.5%, 97.75%, and 93.2%, respectively.ConclusionThe base classifier performs significantly better on proposed features fusion and selection method as compared to other methods in terms of sensitivity, specificity, and accuracy. Furthermore, the presented method achieved satisfactory segmentation results on selected datasets.

A Distributed Feature Selection Algorithm Based on Distance Correlation with an Application to Microarrays.

DNA microarray datasets are characterized by a large number of features with very few samples, which is a typical cause of overfitting and poor generalization in the classification task. Here, we introduce a novel feature selection (FS) approach which employs the distance correlation (dCor) as a criterion for evaluating the dependence of the class on a given feature subset. The dCor index provides a reliable dependence measure among random vectors of arbitrary dimension, without any assumption on their distribution. Moreover, it is sensitive to the presence of redundant terms. The proposed FS method is based on a probabilistic representation of the feature subset model, which is progressively refined by a repeated process of model extraction and evaluation. A key element of the approach is a distributed optimization scheme based on a vertical partitioning of the dataset, which alleviates the negative effects of its unbalanced dimensions. The proposed method has been tested on several microarray datasets, resulting in quite compact and accurate models obtained at a reasonable computational cost.

Transductive Feature Selection Using Clustering-Based Sample Entropy for Temperature Prediction in Weather Forecasting.

Entropy measures have been a major interest of researchers to measure the information content of a dynamical system. One of the well-known methodologies is sample entropy, which is a model-free approach and can be deployed to measure the information transfer in time series. Sample entropy is based on the conditional entropy where a major concern is the number of past delays in the conditional term. In this study, we deploy a lag-specific conditional entropy to identify the informative past values. Moreover, considering the seasonality structure of data, we propose a clustering-based sample entropy to exploit the temporal information. Clustering-based sample entropy is based on the sample entropy definition while considering the clustering information of the training data and the membership of the test point to the clusters. In this study, we utilize the proposed method for transductive feature selection in black-box weather forecasting and conduct the experiments on minimum and maximum temperature prediction in Brussels for 1–6 days ahead. The results reveal that considering the local structure of the data can improve the feature selection performance. In addition, despite the large reduction in the number of features, the performance is competitive with the case of using all features.

Social spider optimisation algorithm for dimension reduction of electroencephalogram signals in human emotion recognition

Due to some limitations of current heuristics and evolutionary algorithms, this paper proposed a new swarm based algorithm for feature selection method called Social Spider Optimization (SSO-FS). In this research, SSO-FS is used in the EEG-based emotion recognition model as searching method to find optimal feature set to maximize classification performance and mimics the cooperative behaviour and mechanism of social spiders in nature. This proposed feature selection method has been tested on DEAP EEG dataset with six subjects and compared with the most popular heuristic algorithms such as GA, PSO and ABC. The results show that the SSO-FS provides a remarkable and comparable performance compared to other existing methods. Whereby, the max accuracy obtained is 66.66% and 70.83%, the mean accuracy obtained is 55.51 7.17 and 60.97 8.38 for 3-level of valence emotions and 3-level of arousal emotions classification respectively.

Exploring diagnosis and imaging biomarkers of Parkinson’s disease via iterative canonical correlation analysis based feature selection

Parkinson’s disease (PD) is a neurodegenerative disorder that progressively hampers the brain functions and leads to various movement and non-motor symptoms. However, it is difficult to attain early-stage PD diagnosis based on the subjective judgment of physicians in clinical routines. Therefore, automatic and accurate diagnosis of PD is highly demanded, so that the corresponding treatment can be implemented more appropriately. In this paper, we focus on finding the most discriminative features from different brain regions in PD through T1-weighted MR images, which can help the subsequent PD diagnosis. Specifically, we proposed a novel iterative canonical correlation analysis (ICCA) feature selection method, aiming at exploiting MR images in a more comprehensive manner and fusing features of different types into a common space. To state succinctly, we first extract the feature vectors from the gray matter and the white matter tissues separately, represented as insights of two different anatomical feature spaces for the subject’s brain. The ICCA feature selection method aims at iteratively finding the optimal feature subset from two sets of features that have inherent high correlation with each other. In experiments we have conducted thorough investigations on the optimal feature set extracted by our ICCA method. We also demonstrate that using the proposed feature selection method, the PD diagnosis performance is further improved, and also outperforms many state-of-the-art methods.

A fuzzy rough set-based feature selection method using representative instances

The fuzzy rough set theory has been widely used to deal with uncertainty in real-valued or even complex data, in which one of the most concerned issues is feature selection. Since a real-world data set generally contains redundant data objects (or instances) and errors which lead to the fact that not all the instances are of equal importance, focusing on the representative instances for feature selection can not only acquire more convincing analysis results but also alleviate computational complexity in mining knowledge. At present, however, little attention has been paid on using representative instances to select features. In this paper, the issue of selecting features by using representative instances is investigated based on fuzzy rough sets and a representative instance-based feature selection approach is proposed. First, the fuzzy granular rule is employed to describe the discriminating information of an instance. Then, the representative instances are selected according to the coverage ability of the fuzzy granular rules induced by all of the instances. Furthermore, an implication relationship-preserved reduction is presented to maintain the discriminating information of the selected instances, and then a heuristic algorithm is presented to search for such a feature subset. Finally, a filter-wrapper approach is suggested to select the best subset of the features. Some numerical experiments are further conducted to show the performance of the proposed feature selection method and the results are satisfactory in terms of both efficiency and effectiveness.

Multidimensional BSS/WSS Criterion for Modified SBFS Feature Selection Method in Tumor Classification

To avoid the defect of BSS/WSS criterion, we propose a multidimensional BSS/WSS feature selection criterion and modify the sequential backward floating selection (SBFS) algorithm to deal with the case where the covariance matrix is singular in this study. Then, we use support vector machine (SVM) to classify the gene expression data based on the proposed feature selection algorithm. The performance of the proposed approach is compared with BSS/WSS criterion and some other popular methods in feature selection and classification via the wellknown colon cancer and prostate datasets in microarray literature, which demonstrates that the proposed criterion can take into account genes' joint discriminatory power, and the proposed feature selection method can obtain correct and informative gene subset for tumor classification.

Mid-Price Prediction Based on Machine Learning Methods with Technical and Quantitative Indicators

Stock price prediction is a challenging task, but machine learning methods have recently been used successfully for this purpose. In this paper, we extract over 270 hand-crafted features (factors) inspired by technical and quantitative analysis and tested their validity on short-term mid-price movement prediction. We focus on a wrapper feature selection method using entropy, least-mean squares, and linear discriminant analysis. We also build a quantitative feature based on adaptive logistic regression for online learning, which is constantly selected first among the majority of the proposed feature selection methods. This study examines the best combination of features using high frequency limit order book data from Nasdaq Nordic. Our results suggest that sorting methods and classifiers can be used in such a way that one can reach the best performance with a combination of only very few informative features. This paper opens avenues for developing more advanced features combined with more sophisticated feature selection methods. It also provides helpful insight to market makers and traders in general by providing useful results that can be used to gain an information edge in trading.

Feature selection based on Bayesian network for chiller fault diagnosis from the perspective of field applications

From the perspective of field applications, a feature selection (FS) method is proposed in this study for chiller fault diagnosis (FD). First, the candidate existing features that can be retained (CRE-features) are nominated through the following criteria: high existent frequency of sensors installed on the field chillers, high sensitivity to faults and small amount of calculation. Then these features are evaluated by using an FD method based on a Bayesian network merged distance rejection (DR-BN) technique to remove redundant features. Second, when the expected performance cannot be obtained by only using these specifically retained existing features (RE-features), additional features need be included. Supplemental features (S-features) are nominated through the following criteria: low cost of measurement and high sensitivity to faults. Then these S-features are evaluated together with RE-features by using the same method to determine the specific S-features. The experimental data from ASHRAE RP-1043 are used to validate the FS method. Results show that the proposed FS method is effective for chiller FD, and can make the matched FD method perform well by selecting the commonly available features in the field and by supplementing a few features with low cost of measurement to indicate faults.

Feature selection for continuous aggregate response and its application to auto insurance data

This paper presents new feature selection algorithms for aggregate data analysis. Data aggregation is commonly used when it is not appropriate to model the relationship between a response and explanatory variables at an individual-level. We investigate substantial challenges in analysis for aggregate data. Then, we propose a groupwise feature selection method that addresses (i) the change in dataset depending on the selection of predictor variables, (ii) the presence of potential missing responses, and (iii) the suitability of model selection criteria when comparing models using different datasets. In application to real auto insurance data, we find a set of important predictors to classify the policyholders into some homogeneous risk groups. Our results clearly demonstrate the potential of the proposed feature selection method for aggregate data analysis in terms of flexibility and computational complexity. We expect that the proposed algorithms would be further applied into a wide range of decision-making tasks using aggregate data as they are applicable to any type of data.

A discriminative multi-class feature selection method via weighted l2,1-norm and Extended Elastic Net

Abstract Feature selection has playing an important role in many pattern recognition and machine learning applications, where meaningful features are desired to be extracted from high dimensional raw data and noisy ones are expected to be eliminated. l 2,1 -norm regularization based Robust Feature Selection (RFS) has extracted a lot of attention due to its efficiency and high performance of joint sparsity. In this paper, we propose a more general framework for robust and discriminative multi-class feature selection. Four types of weighting, which are based on correlation information between features and labels, are adopted to strengthen the discriminative performance of l 2,1 -norm joint sparsity. F-norm regularization, which is extended from multi-class Elastic Net, is added to improve the stability of the method. An efficient algorithm and its corresponding convergence proof are provided. Experimental results on several two-class and multi-class datasets are performed to verify the effectiveness of the proposed feature selection method.

Feature selection in multiword expression recognition

In multiword expression (MWE) recognition, there exist many studies where different learning methods are employed to decide whether given word combination is a multiword expression. The recognition methods commonly utilize a number of features that are extracted from a data source, frequently from the given text. Though the recognition methods and the features are well studied, we believe that to achieve the best possible performance with a learning method, different subsets of features should also be considered and the best performing subset must be selected.In this paper, we propose a procedure that covers the performance comparison of well-known feature selection methods to obtain the best feature subset in MWE recognition. The evaluation tests are performed on a Turkish MWE data set and the performance is measured by precision, recall and F1 values. The highest F1 value =0.731 is obtained by C4.5 classifier employing either wrapper or filtering method in feature selection. In the regarding setting(s), it is examined that the performance is increased by 1.11% compared to the setting where all features are employed in classification.Based on the experimental results, it may be stated that feature selection improves the performance of MWE recognition by eliminating the noisy/non-effective features. Moreover, it is obvious that proposed feature selection method contributes to the overall MWE recognition system by reducing the measurement and storage requirements due to the lower number of features in classification, providing a faster and more-cost effective learning model.

A Hybrid Feature Selection Method for Improved Detection of Wired/Wireless Network Intrusions

Internet has become an essential aspect of communication in the day to day life of everyone around the world. With the increased usage of Internet, attacks have also increased and the need for various levels of security is on the rise, both in wired and wireless environments. Intrusion detection system (IDS) has become a mandatory level of security for organizations to protect themselves from intruders. Improving the accuracy of IDS is crucial and it is the present focus of researchers. Feature selection has its role in enhancing accuracy by extracting the most relevant features. This study proposes a hybrid method for feature selection that picks and combines the best features from different feature selection methods. This method can be applied for feature reduction in any application domain. In this work, the proposed hybrid method is employed for intrusion detection and six predominant features are picked from NSL-KDD dataset. An exhaustive performance investigation has proved that the proposed feature selection method increases the detection rate by 5% thereby improving the accuracy of intrusion detection system by 3%.

Self-Weighted Supervised Discriminative Feature Selection.

In this brief, a novel self-weighted orthogonal linear discriminant analysis (SOLDA) problem is proposed, and a self-weighted supervised discriminative feature selection (SSD-FS) method is derived by introducing sparsity-inducing regularization to the proposed SOLDA problem. By using the row-sparse projection, the proposed SSD-FS method is superior to multiple sparse feature selection approaches, which can overly suppress the nonzero rows such that the associated features are insufficient for selection. More specifically, the orthogonal constraint ensures the minimal number of selectable features for the proposed SSD-FS method. In addition, the proposed feature selection method is able to harness the discriminant power such that the discriminative features are selected. Consequently, the effectiveness of the proposed SSD-FS method is validated theoretically and experimentally.

Multivariate correlation coefficient and mutual information-based feature selection in intrusion detection

ABSTRACTFeature selection is one of the major problems in an intrusion detection system (IDS) since there are additional and irrelevant features. This problem causes incorrect classification and low detection rate in those systems. In this article, four feature selection algorithms, named multivariate linear correlation coefficient (MLCFS), feature grouping based on multivariate mutual information (FGMMI), feature grouping based on linear correlation coefficient (FGLCC), and feature grouping based on pairwise MI, are proposed to solve this problem. These algorithms are implementable in any IDS. Both linear and nonlinear measures are used in the sense that the correlation coefficient and the multivariate correlation coefficient are linear, whereas the MI and the multivariate MI are nonlinear. Least Square Support Vector Machine (LS-SVM) as an intrusion classifier is used to evaluate the selected features. Experimental results on the KDDcup99 and Network Security Laboratory-Knowledge Discovery and Data Mining (NSL) datasets showed that the proposed feature selection methods have a higher detection and accuracy and lower false-positive rate compared with the pairwise linear correlation coefficient and the pairwise MI employed in several previous algorithms.

Predicting the listing statuses of Chinese-listed companies using decision trees combined with an improved filter feature selection method

Predicting the listing statuses of Chinese-listed companies (PLSCLC) is an important and complex problem for investors in China. There is a large quantity of information related to each company’s listing status. We propose an improved filter feature selection method to select effective features for predicting the listing statuses of Chinese-listed companies. Due to the practical concerns of analysts in finance about the performance and interpretability of the prediction models, models based on decision trees C4.5 and C5.0 are employed and are compared with several other widely used models. To evaluate the models’ robustness with time, the models are also tested under rolling time windows. The empirical results demonstrate the efficacy of the proposed feature selection method and decision tree C5.0 model.

An improved scheme for rice phenology estimation based on time-series multispectral HJ-1A/B and polarimetric RADARSAT-2 data

Rice phenology information is critical for farm management and productivity evaluation. Synthetic aperture radar (SAR) and optical remote sensing data are very useful for monitoring rice growth. Therefore, this paper focuses on building a robust, accurate and operational scheme for phenology estimation based on SAR and multispectral remote sensing data. The proposed scheme has three main improvements. First, two main types of rice field - transplanted indica rice field (TRF) and direct-sown japonica rice field (DRF) - were considered individually. Second, 149 SAR signatures and optical Vegetation Indices (VIs) were extracted from time-series of optical and full/compact-pol SAR data and a novel feature selection method based on Monte Carlo experiments and the correlation limitation (MCCL) was proposed. Third, six different phenological labeling cases were considered, with the multiclass relevant vector machine (mRVM) being used to identify different stages. Based on the improvements above, we generated the optimal feature matrix based on the MCCL; this matrix consisted of the optimal feature subset (OFS) used for the identification of each stage. The overall phenological estimation accuracy (OPEA) based on the optical VIs and SAR signatures together (86.59%), was shown to be better than that based on either the optical VIs or the SAR signatures only. In addition, it was significant to consider the differences between the DRF and TRF for rice phenology estimation. The OPEA was higher than 85% when the DRF and TRF were considered separately, much higher than the accuracy obtained (68.89%) when the differences between the DRF and TRF were ignored. Lastly, the availability of the proposed feature selection method MCCL was discussed. It was found that the best OPEA was generated using the OFS selected by the proposed MCCL when the dimensions of the OFS were kept small.

A discrete bacterial algorithm for feature selection in classification of microarray gene expression cancer data

When mining in high dimensional data, the curse of dimensionality is one of the major difficulty to overcome. In this paper, a weighted feature selection strategy is developed and embedded in bacterial based algorithms to reduce the feature dimension in classification. The proposed weighted feature selection strategy distinguishes the features by their classification performances as well as the occurrence frequency in population according to the two matrices. The objectives of minimizing the number of features, maximizing the performance, and minimizing the computational cost are all considered. Regarding the drawback of bacterial based algorithms, Bacterial Colony Optimization based feature selection algorithm is proposed to decrease the computational complexity as well as improve the search ability even in discrete optimization problems. To test the effectiveness of the proposed feature selection method, four bacterial based methods with the weighted strategy embedded have been compared with four classical feature selection methods and three well-known population based algorithms using 15 cancer micro-array datasets with different numbers of features and classes. The results show that the weighted feature selection strategies embedded have improved the feature selection capability of bacterial algorithms. The new proposed mechanisms embedded in Bacterial Colony Optimization method can overcome the limitation of the traditional bacterial based algorithms using premature termination to decrease the computational time, and provide comparable or in most cases better solutions than other feature selection methods considered in the comparison.

Features extraction and multi-classification of sEMG using a GPU-Accelerated GA/MLP hybrid algorithm.

Surface electromyography (sEMG) signal is the combined effect of superficial muscle EMG and neural electrical activity. In recent years, researchers did large amount of human-machine system studies by using the physiological signals as control signals. To develop and test a new multi-classification method to improve performance of analyzing sEMG signals based on public sEMG dataset. First, ten features were selected as candidate features. Second, a genetic algorithm (GA) was applied to select representative features from the initial ten candidates. Third, a multi-layer perceptron (MLP) classifier was trained by the selected optimal features. Last, the trained classifier was used to predict the classes of sEMG signals. A special graphics processing unit (GPU) was used to speed up the learning process. Experimental results show that the classification accuracy of the new method reached higher than 90%. Comparing to other previously reported results, using the new method yielded higher performance. The proposed features selection method is effective and the classification result is accurate. In addition, our method could have practical application value in medical prosthetics and the potential to improve robustness of myoelectric pattern recognition.

A novel low-rank hypergraph feature selection for multi-view classification

In order to select informative features from a high-dimensional multi-view dataset, we have proposed a feature selection method that simultaneously embedding the low-rank constraint, sparse representation, global and local structure learning into a unified framework. Firstly, we utilize the conventional regression function (i.e. the least square loss function) to form a novel regression framework by introducing a low-rank constraint and a relaxation term. And then we employ an l21-norm regularization term to filter out the redundant and irrelative features. Furthermore, we utilize a hypergraph based regularization term rather than the simple graph to construct a Laplacian matrix that will be used in enhancing the inherent association of data. Besides, we proposed a novel optimization algorithm to solve the objective function. Finally, we feed the reduced data got by the proposed feature selection method into Support Vector Machines (SVM) in term of classification accuracy. The experimental results showed that the proposed method achieved the best classification performance, compared with the state-of-the-art feature selection methods on real multi-view dataset.