Proposed Feature Selection Method Research Articles

Feature Selection Based on Multi Resolution Analysis of Text Documents for Effective Clustering

Users of digital era are overwhelmed with large volumes of text collections. Most of the text collections are without class labels. Clustering is the only feasible solution to extract valuable insights from the data. Clustering of text collections in high dimensional space is inefficient. Several unsupervised dimensionality reduction methods have been proposed in the literature. Feature selection methods are easy to interpret. Filter feature selection methods have been proved to be scalable and efficient for high dimensional datasets. The aim of this work is to propose an unsupervised univariate filter feature selection method for efficient clustering of very high dimensional text datasets in low dimensional feature space. Wavelets are mathematical functions that can transform a signal into space or time, frequency domain to analyze the signal at different resolutions of transformed domain. Wavelet transforms are efficient in identifying transients of a signal. Stationary wavelet transform using Symlet of order 2 is used to identify the most discriminant features of text documents for efficient clustering in low dimensional feature space. The proposed feature selection method is compared with nine other relevant methods by their quality of clustering solution on seven real text document collections. The proposed method has been able to identify the most discriminative features that have resulted in the best peak clustering performance better than clustering performance with all the features, on four out of seven datasets, with at most 1.5% of top-rated features.

Fuzzy information decomposition incorporated and weighted Relief-F feature selection: When imbalanced data meet incompletion

Data classification is an important computer task in data analysis, which suffers seriously unknown features, imbalanced class, and incomplete data. However, despite their vital yet practical significance, few results have been made on such three distinct issues. To address this problem, we propose a novel feature selection method for the data subject to incomplete data and imbalanced class, namely, improved fuzzy information decomposition (IFID) incorporated and weighted Relief-F (WRelief-F) feature selection. The main idea of the proposed feature selection method is threefold. (1) The proposed IFID algorithm can deal with the imbalanced class and incomplete data at the same time. (2) In IFID, a new membership function is provided to reflect the influence of the observed data on the missing values appropriately. Based on this establishment, a more delicate information decomposition is adopted to make a better recovery than the traditional FID. (3) After using IFID, WRelief-F is put forward to take the relationship of the target instance to inter-class instances and the intra-class instances into consideration in a proper manner. Finally, experiments on the seven public data sets are utilized to show the effectiveness and universal applicability of the proposed feature selection algorithm.

Assessing feature selection method performance with class imbalance data

Identifying the most informative features is a crucial step in feature selection. This paper focuses primarily on wrapper feature selection methods designed to detect important features with F1-score as the target metric. As an initial step, most wrapper methods order features according to importance. However, in most cases, the importance is defined according to the classification method used and varies with the characteristics of the data set. Using synthetically simulated data, we examine four existing feature ordering techniques to find the most desirable and the most effective ordering mechanism to identify informative features. Using the results, an improved method is suggested to extract the most informative feature subset from the data set. The method uses the sum of absolute values of the first k principal component loadings to order the features where k is a user-defined application-specific value. It also applies a sequential feature selection method to extract the best subset of features. We further compare the performance of the proposed feature selection method with results from the existing Recursive Feature Elimination (RFE) by simulating data for several practical scenarios with a different number of informative features and different imbalance rates. We also validate the method using a real-world application on several classification methods. The results based on the accuracy measures indicate that the proposed approach performs better than the existing feature selection methods.

A High-Efficiency Fatigued Speech Feature Selection Method for Air Traffic Controllers Based on Improved Compressed Sensing.

Air traffic controller fatigue has recently received considerable attention from researchers because it is one of the main causes of air traffic incidents. Numerous research studies have been conducted to extract speech features related to fatigue, and their practical utilization has achieved some positive detection results. However, there are still challenges associated with the applied speech features usually being of high dimension, which leads to computational complexity and inefficient fatigue detection. This situation makes it meaningful to reduce the dimensionality and select only a few efficient features. This paper addresses these problems by proposing a high-efficiency fatigued speech selection method based on improved compressed sensing. For adapting a method to the specific field of fatigued speech, we propose an improved compressed sensing construction algorithm to decrease the reconstruction error and achieve superior sparse coding. The proposed feature selection method is then applied to optimize the high-dimension fatigued speech features based on the fractal dimension. Finally, a support vector machine classifier is applied to a series of comparative experiments using the Civil Aviation Administration of China radiotelephony corpus to demonstrate that the proposed method provides a significant improvement in the precision of fatigue detection compared with current state-of-the-art approaches.

Automated detection of bioimages using novel deep feature fusion algorithm and effective high-dimensional feature selection approach

The classification of bioimages plays an important role in several biological studies, such as subcellular localisation, phenotype identification and other types of histopathological examinations. The objective of the present study was to develop a computer-aided bioimage classification method for the classification of bioimages across nine diverse benchmark datasets. A novel algorithm was developed, which systematically fused the features extracted from nine different convolution neural network architectures. A systematic fusion of features boosts the performance of a classifier but at the cost of the high dimensionality of the fused feature set. Therefore, non-discriminatory and redundant features need to be removed from a high-dimensional fused feature set to improve the classification performance and reduce the time complexity. To achieve this aim, a method based on analysis of variance and evolutionary feature selection was developed to select an optimal set of discriminatory features from the fused feature set. The proposed method was evaluated on nine different benchmark datasets. The experimental results showed that the proposed method achieved superior performance, with a significant reduction in the dimensionality of the fused feature set for most bioimage datasets. The performance of the proposed feature selection method was better than that of some of the most recent and classical methods used for feature selection. Thus, the proposed method was desirable because of its superior performance and high compression ratio, which significantly reduced the computational complexity.

Particle distance rank feature selection by particle swarm optimization

This paper presents a feature selection method in multi-objective particle swarm optimization space. For this task, a novel particle ranking is proposed based on particle distance from dominated and non-dominated particles and then used for feature rank computation. Position and velocity of particles are updated by a new update rule relies in feature ranks encoded in a vector. Properties of the proposed method are proven mathematically and supported in experiments. The proposed feature selection method is evaluated on 12 UCI datasets and 4 datasets from real-world applications compared with 5 state-of-the-art feature selection methods. As a visual comparison, the proposed method finds better non-dominated particles in two-dimensional optimization space with lower run time. Experiments also showed that the proposed method outperforms existing feature selection methods with regard to Success Counting Measure, C_Metric, Hyper-Volume Indicator and Statistical Analysis.

An efficient feature selection framework based on information theory for high dimensional data

Feature selection plays a vital role in many fields, particularly in pattern recognition and bioinformatics, for selecting informative and relevant features from high dimensional datasets. The increase in dimensionality of data along with the existence of redundant and irrelevant features leads to challenging performance issues when processing and analysing the data. In this paper, an effective feature selection technique called mutual information and Monte Carlo based feature selection (MIMCFS) is proposed. It comprises of two stages. The first stage aims to select predominant features from the high dimensional data. The second stage involves elimination of redundant features that were selected in the first stage. For the purpose of implementing the first stage, a new feature selection strategy based on the approximate Markov blanket and the concept of mutual information is proposed to find out irrelevant and redundant features. In second stage, to avoid misjudgement of redundant features as relevant features, a new strategy based on Monte Carlo tree search technique is proposed in order to completely eradicate redundant features and to improve feature interaction. For experimental evaluation, eight benchmark microarray datasets including imbalanced ones pertaining to cancer analysis are used. Further, in order to compare and justify the performance of the proposed feature selection method, seven state-of-art feature selection techniques namely CFS, Relief, DISR, JMI, CMIM and CMI are employed. The outputs from these feature selection techniques are provided to three standard classifiers namely Naive Bayes, SVM and C4.5 in order to assess the significance of the selected features in building classification models. 10-fold cross validation is adopted to evaluate the classifiers. Accuracy, precision, recall, f-measure, standard deviation, statistical significance metrics are measured to quantify the classifier performance. Experimental results demonstrate the outstanding performance of the proposed algorithm when compared to that of the standard existing methods.

Functional Disruptions of the Brain in Low Back Pain: A Potential Imaging Biomarker of Functional Disability.

Chronic low back pain (LBP) is one of the leading causes of disability worldwide. While LBP research has largely focused on the spine, many studies have demonstrated a restructuring of human brain architecture accompanying LBP and other chronic pain states. Brain imaging presents a promising source for discovering noninvasive biomarkers that can improve diagnostic and prognostication outcomes for chronic LBP. This study evaluated graph theory measures derived from brain resting-state functional connectivity (rsFC) as prospective noninvasive biomarkers of LBP. We also proposed and tested a hybrid feature selection method (Enet-subset) that combines Elastic Net and an optimal subset selection method. We collected resting-state functional MRI scans from 24 LBP patients and 27 age-matched healthy controls (HC). We then derived graph-theoretical features and trained a support vector machine (SVM) to classify patient group. The degree centrality (DC), clustering coefficient (CC), and betweenness centrality (BC) were found to be significant predictors of patient group. We achieved an average classification accuracy of 83.1% (p < 0.004) and AUC of 0.937 (p < 0.002), respectively. Similarly, we achieved a sensitivity and specificity of 87.0 and 79.7%. The classification results from this study suggest that graph matrices derived from rsFC can be used as biomarkers of LBP. In addition, our findings suggest that the proposed feature selection method, Enet-subset, might act as a better technique to remove redundant variables and improve the performance of the machine learning classifier.

A Feature Selection Method for Classifying Highly Similar Text Documents

In the era of big data, the importance of data classification is increasing. However, when it comes to classifying text documents, several obstacles degrade classification performance. These include multi-class documents, high levels of similarity between classes, class size imbalance, high dimensional representation space, and a low frequency of unique and discriminative features. To overcome these obstacles and improve classification performance, this paper proposes a novel feature selection method that effectively utilizes both unique and overlapping features. In general, feature selection methods have ignored unique features that occur only one class because of low frequency while it provides better discriminative-power. On the contrary, overlapping features, which are found in several classes with high frequency, have been also less preferred because of low discriminative-power. The proposed feature selection method attempts to use these two types of features as complementary with aims to improve overall classification performance for highly similar text documents. Extensive numerical analysis have been conducted for three benchmarking datasets with a support vector machine (SVM) classifier. The proposed method showed that not only the class with high similarity but also the general classification performance is superior to the conventional feature selection methods, such as the global feature set, local feature set, discriminative feature set, and information gain.

A novel mutual information-based feature selection approach forefficient transfer learning in aerial scene classification

ABSTRACT Several transfer learning methods based on deep Convolutional Neural Networks (CNNs) have been developed so far but obtaining satisfactory classification accuracy still remains a challenging task. However, effective classification performance mainly depends on the feature set of input data to be classified. During transfer learning, features are extracted from multiple layers or multiple models of pre-trained convolutional neural networks to form feature sets of input data, which may contain some redundant features thus affecting the classification results. To tackle this issue, a feature selection method must be applied to transfer features before being fed to the classifier. Hence, a feature selection method is proposed using the idea of mutual information theory to remove redundant features for effective transfer of the learning task. The proposed feature selection method is the combination of filter and embedded approaches that are cascaded one after another to form the final feature set for classification. The proposed method proves its effectiveness when compared to some existing methods on three aerial scene datasets.

Soil Nutrient Estimation and Mapping in Farmland Based on UAV Imaging Spectrometry.

Soil nutrient is one of the most important properties for improving farmland quality and product. Imaging spectrometry has the potential for rapid acquisition and real-time monitoring of soil characteristics. This study aims to explore the preprocessing and modeling methods of hyperspectral images obtained from an unmanned aerial vehicle (UAV) platform for estimating the soil organic matter (SOM) and soil total nitrogen (STN) in farmland. The results showed that: (1) Multiplicative Scattering Correction (MSC) performed better in reducing image scattering noise than Standard Normal Variate (SNV) transformation or spectral derivatives, and it yielded a result with higher correlation and lower signal-to-noise ratio; (2) The proposed feature selection method combining Successive Projections Algorithm (SPA) and Competitive Adaptive Reweighted Sampling algorithm (CARS), could provide selective preference for hyperspectral bands. Exploiting this method, 24 and 22 feature bands were selected for SOM and STN estimation, respectively; (3) The particle swarm optimization (PSO) algorithm was employed to obtain optimized input weights and bias values of the extreme learning machine (ELM) model for more accurate prediction of SOM and STN. The improved PSO-ELM model based on the selected preference bands achieved higher prediction accuracy (R2 of 0.73 and RPD of 1.91 for SOM, R2 of 0.63, and RPD of 1.53 for STN) than support vector machine (SVM), partial least squares regression (PLSR), and the ELM model. This study provides an important guideline for monitoring soil nutrient for precision agriculture with imaging spectrometry.

An Optimal Feature Selection Method for Automatic Face Retrieval Using Enhanced Grasshopper Optimization Algorithm

Background: Retrieval of facial images based on its contents is one of the main areas of research. However, images contain high dimensional feature vectors and it is a challenging task to select the relevant features due to the variations available in the images of similar objects. Therefore, the selection of relevant features is an important step to make the facial retrieval system computationally efficient and more accurate. Objective: The main aim of this paper is to design and develop an efficient feature selection method for obtaining relevant and non-redundant features from the face images so that the accuracy and computational cost of a face retrieval system can be improved. Methods: The proposed feature selection method uses a new enhanced grasshopper optimization algorithm to obtain the significant features from the high dimensional features vector of face images. The proposed algorithm modifies the target vector by considering more than one best solution which maintain the elitism property and save the search from local optimum. Furthermore, it has been utilized to select the prominent features from the high dimensional facial features vector. Results: The performance of the proposed feature selection method has been tested on Oracle Research Laboratory face database. The proposed method eliminates 89% features which are minimum among the other methods and increases the accuracy of face retrieval system to 96.5%. Conclusion: The enhanced grasshopper optimization algorithm-based feature selection method for face retrieval system outperforms the existing methods in terms of accuracy and computational cost.

Feature Selection Method Based on Grey Wolf Optimization and Simulated Annealing

Background: Feature selection sometimes also known as attribute subset selection is a process in which optimal subset of features are elected with respect to target data by reducing dimensionality and removing irrelevant features. There will be 2n possible solutions for a dataset having n number of features that is difficult to solve by conventional attribute selection method. In such cases metaheuristic-based methods generally outruns the conventional methods. Objective: The main aim of this paper is to enhance the classification accuracy and minimize the number of selected features and error rate. Methods: To achieve the objective, a binary metaheuristic feature selection method bGWOSA based on grey wolf optimization and simulated annealing has been introduced. The proposed feature selection method uses simulated annealing for equalizing the trade-off between exploration and exploitation. The performance of the proposed binary feature selection method has been examined on the ten feature selection benchmark datasets taken from UCI repository and compared with binary cuckoo search, binary particle swarm optimization, binary grey wolf optimization, binary bat algorithm and binary hybrid whale optimization method. Results: The proposed feature selection method achieves the highest accuracy for the most of datasets compared to state-of-the-art. Further, from the experimental and statistical results, efficacy of the proposed feature selection method has been validated. Conclusion: Classification accuracy can be enhanced by employing feature selection methods. Moreover, performance can also be enhanced by tuning the control parameters of metaheuristic methods.

Using Recursive Feature Selection with Random Forest to Improve Protein Structural Class Prediction for Low-Similarity Sequences.

Many combinations of protein features are used to improve protein structural class prediction, but the information redundancy is often ignored. In order to select the important features with strong classification ability, we proposed a recursive feature selection with random forest to improve protein structural class prediction. We evaluated the proposed method with four experiments and compared it with the available competing prediction methods. The results indicate that the proposed feature selection method effectively improves the efficiency of protein structural class prediction. Only less than 5% features are used, but the prediction accuracy is improved by 4.6-13.3%. We further compared different protein features and found that the predicted secondary structural features achieve the best performance. This understanding can be used to design more powerful prediction methods for the protein structural class.

A novel deep learning-based feature selection model for improving the static analysis of vulnerability detection

The automatic detection of software vulnerabilities is considered a complex and common research problem. It is possible to detect several security vulnerabilities using static analysis (SA) tools, but comparatively high false-positive rates are observed in this case. Existing solutions to this problem depend on human experts to identify functionality, and as a result, several vulnerabilities are often overlooked. This paper introduces a novel approach for effectively and reliably finding vulnerabilities in open-source software programs. In this paper, we are motivated to examine the potential of the clonal selection theory. A novel deep learning-based vulnerability detection model is proposed to define features using the clustering theory of the clonal selection algorithm. To our knowledge, this is the first time we have used deep-learned long-lived team-hacker features to process memories of sequential features and mapping from the entire history of previous inputs to target vectors in theory. With an immune-based feature selection model, the proposed approach aimed to improve static analyses' detection abilities. A real-world SA dataset is used based on three open-source PHP applications. Comparisons are conducted based on using a classification model for all features to measure the proposed feature selection methods' classification improvement. The results demonstrated that the proposed method got significant enhancements, which occurred in the classification accuracy also in the true positive rate.

Hybrid feature selection model for classification of lung disorders

Feature selection in computer aided diagnosis is now becoming a challenging part in the classification of lung diseases. This is because, it needs to deliver results with improved accuracy and it also requires a greater number of features for analysis. The major demerit of widely utilized single-objective feature selection (FS) algorithm is that it proffers only a single optimum solution for a feature set. Here, a hybridized multi-objective particle swarm optimization with a local Tabu search (MOPSO-TS) algorithm is proposed to overcome the above demerit of the traditional single objective algorithm by producing a bag of optimum solutions which trade disparate objectives amongst themselves. The work is validated against a feature set which consists of GLCM features, shape features and GLRLM (gray-level run length matrix) extracted from lung chest tomography (CT) images. Classification is done using k-nearest neighbor with class probability and normal distribution (ND). The proposed FS method’s performance is analyzed against widely used bio-inspired FS methods such as Firefly, Particle Swarm Optimization along with Bee Colony Optimization algorithms. The numerical analysis of this model indicates that the proposed hybrid FS algorithm achieves improved performance compared to a single objective optimization algorithm in respect of specificity, accuracy, F-score, precision, sensitivity and error rate. The proposed algorithm obtains the result of 90.588% in both and specificity accuracy rate, (77.143) precision, 87.667 sensitivity rate and error rate of 0.1 which are higher on considering the other prevailing methodologies

Improving performance and generalizability in radiogenomics: a pilot study for prediction of IDH1/2 mutation status in gliomas with multicentric data.

.Purpose: Radiogenomics offers a potential virtual and noninvasive biopsy. However, radiogenomics models often suffer from generalizability issues, which cause a performance degradation on unseen data. In MRI, differences in the sequence parameters, manufacturers, and scanners make this generalizability issue worse. Such image acquisition information may be used to define different environments and select robust and invariant radiomic features associated with the clinical outcome that should be included in radiomics/radiogenomics models.Approach: We assessed 77 low-grade gliomas and glioblastomas multiform patients publicly available in TCGA and TCIA. Radiomics features were extracted from multiparametric MRI images (T1-weighted, contrast-enhanced T1-weighted, T2-weighted, and fluid-attenuated inversion recovery) and different regions-of-interest (enhancing tumor, nonenhancing tumor/necrosis, and edema). A method developed to find variables that are part of causal structures was used for feature selection and compared with an embedded feature selection approach commonly used in radiomics/radiogenomics studies, across two different scenarios: (1) leaving data from a center as an independent held-out test set and tuning the model with the data from the remaining centers and (2) use stratified partitioning to obtain the training and the held-out test sets.Results: In scenario (1), the performance of the proposed methodology and the traditional embedded method was AUC: 0.75 [0.25; 1.00] versus 0.83 [0.50; 1.00], Sens.: 0.67 [0.20; 0.93] versus 0.67 [0.20; 0.93], Spec.: 0.75 [0.30; 0.95] versus 0.75 [0.30; 0.95], and MCC: 0.42 [0.19; 0.68] versus 0.42 [0.19; 0.68] for center 1 as the held-out test set. The performance of both methods for center 2 as the held-out test set was AUC: 0.64 [0.36; 0.91] versus 0.55 [0.27; 0.82], Sens.: 0.00 [0.00; 0.73] versus 0.00 [0.00; 0.73], Spec.: 0.82 [0.52; 0.94] versus 0.91 [0.62; 0.98], and MCC: versus , whereas for center 3 was AUC: 0.80 [0.62; 0.95] versus 0.89 [0.56; 0.96], Sens.: 0.86 [0.48; 0.97] versus 0.86 [0.48; 0.97], Spec.: 0.72 [0.54; 0.85] versus 0.79 [0.61; 0.90], and MCC: 0.47 [0.41; 0.53] versus 0.55 [0.48; 0.60]. For center 4, the performance of both methods was AUC: 0.77 [0.51; 1.00] versus 0.75 [0.47; 0.97], Sens.: 0.53 [0.30; 0.75] versus 0.00 [0.00; 0.15], Spec.: 0.71 [0.35; 0.91] versus 0.86 [0.48; 0.97], and MCC: 0.23 [0.16; 0.31] versus. . In scenario (2), the performance of these methods was AUC: 0.89 [0.71; 1.00] versus 0.79 [0.58; 0.94], Sens.: 0.86 [0.80; 0.92] versus 0.43 [0.15; 0.74], Spec.: 0.87 [0.62; 0.96] versus 0.87 [0.62; 0.96], and MCC: 0.70 [0.60; 0.77] versus 0.33 [0.24; 0.42].Conclusions: This proof-of-concept study demonstrated good performance by the proposed feature selection method in the majority of the studied scenarios, as it promotes robustness of features included in the models and the models’ generalizability by making used imaging data of different scanners or with sequence parameters.

Feature Selection of the Rich Model Based on the Correlation of Feature Components

Currently, the popular Rich Model steganalysis features usually contain a large number of redundant feature components which may bring “curse of dimensionality” and large computation cost, but the existing feature selection methods are difficult to effectively reduce the dimensionality when there are many strongly correlated effective feature components. This paper proposes a novel selection method for Rich Model steganalysis features. First, the separability of each feature component in the submodels of Rich Model is measured based on the Fisher criterion, and the feature components are sorted in the descending order based on the separability. Second, the correlation coefficient between any two feature components in each submodel is calculated, and feature selection is performed according to the Fisher value of each component and the correlation coefficients. Finally, the selected submodels are combined as the final steganalysis feature. The results show that the proposed feature selection method can effectively reduce the dimensionalities of JPEG domain and spatial domain Rich Model steganalysis features without affecting the detection accuracies.

Constructing an Emotion Estimation Model Based on EEG/HRV Indexes Using Feature Extraction and Feature Selection Algorithms

In human emotion estimation using an electroencephalogram (EEG) and heart rate variability (HRV), there are two main issues as far as we know. The first is that measurement devices for physiological signals are expensive and not easy to wear. The second is that unnecessary physiological indexes have not been removed, which is likely to decrease the accuracy of machine learning models. In this study, we used single-channel EEG sensor and photoplethysmography (PPG) sensor, which are inexpensive and easy to wear. We collected data from 25 participants (18 males and 7 females) and used a deep learning algorithm to construct an emotion classification model based on Arousal–Valence space using several feature combinations obtained from physiological indexes selected based on our criteria including our proposed feature selection methods. We then performed accuracy verification, applying a stratified 10-fold cross-validation method to the constructed models. The results showed that model accuracies are as high as 90% to 99% by applying the features selection methods we proposed, which suggests that a small number of physiological indexes, even from inexpensive sensors, can be used to construct an accurate emotion classification model if an appropriate feature selection method is applied. Our research results contribute to the improvement of an emotion classification model with a higher accuracy, less cost, and that is less time consuming, which has the potential to be further applied to various areas of applications.

Feature selection with kernelized multi-class support vector machine

Feature selection is an important procedure in machine learning because it can reduce the complexity of the final learning model and simplify the interpretation. In this paper, we propose a novel non-linear feature selection method that targets multi-class classification problems in the framework of support vector machines. The proposed method is achieved using a kernelized multi-class support vector machine with a fast version of recursive feature elimination. The proposed method selects features that work well for all classes, as the involved classifier simultaneously constructs multiple decision functions that separates each class from the others. We formulate the classifier as a large optimisation problem, and iteratively solve one decision function at a time, leading to a lower computational time complexity than when solving the large optimisation problem directly. The coefficients of the classifier are then used as a ranking criterion in the accelerated recursive feature elimination by adding batch elimination and a rechecking process. Experimental results on several datasets demonstrate the superior performance of the proposed feature selection method.