Fast Feature Selection Algorithm Research Articles

A New Fast Filter-based Unsupervised Feature Selection Algorithm Using Cumulative and Shannon Entropy

A New Fast Filter-based Unsupervised Feature Selection Algorithm Using Cumulative and Shannon Entropy

Multi-label feature selection based on fuzzy rough sets with metric learning and label enhancement

Multi-label feature selection based on fuzzy rough sets, as a key step of multi-label data preprocessing, has been widely concerned by scholars in recent years. Most of the existing multi-label feature selection algorithms directly treat labels as logical labels and use a single distance metric to describe similarity. However, the variability of label descriptions and the limitations of a single distance measure should not be overlooked. In this paper, we propose a fuzzy rough set model with metric learning and label enhancement. Specifically, we use a kernel membership label enhancement algorithm based on JS divergence to convert logical labels into numerical labels, which not only reflects the importance of different labels, but also takes into account the differences in label distribution. In addition, a multi-metric learning algorithm is proposed for multi-label learning, in which the metric distance function under the label space and feature space can be learned autonomously. Then, based on the proposed model, we propose a novel multi-label feature selection algorithm based on metric learning and fuzzy rough sets. On this basis, a fast multi-label feature selection algorithm is further designed to improve the computational efficiency. In the experiments, compared with other nine algorithms on real-world datasets, the results show the superiority of the proposed algorithm.

SFE: A Simple, Fast, and Efficient Feature Selection Algorithm for High-Dimensional Data

In this paper, a new feature selection algorithm, called SFE (Simple, Fast, and Efficient), is proposed for high-dimensional datasets. The SFE algorithm performs its search process using a search agent and two operators: non-selection and selection. It comprises two phases: exploration and exploitation. In the exploration phase, the non-selection operator performs a global search in the entire problem search space for the irrelevant, redundant, trivial, and noisy features, and changes the status of the features from selected mode to non-selected mode. In the exploitation phase, the selection operator searches the problem search space for the features with a high impact on the classification results, and changes the status of the features from non-selected mode to selected mode. The proposed SFE is successful in feature selection from high-dimensional datasets. However, after reducing the dimensionality of a dataset, its performance cannot be increased significantly. In these situations, an evolutionary computational method could be used to find a more efficient subset of features in the new and reduced search space. To overcome this issue, this paper proposes a hybrid algorithm, SFE-PSO (particle swarm optimization) to find an optimal feature subset. The efficiency and effectiveness of the SFE and the SFE-PSO for feature selection are compared on 40 high-dimensional datasets. Their performances were compared with six recently proposed feature selection algorithms. The results obtained indicate that the two proposed algorithms significantly outperform the other algorithms, and can be used as efficient and effective algorithms in selecting features from high-dimensional datasets.

Compactness score: a fast filter method for unsupervised feature selection

The rapid development of big data era incurs the generation of huge amount of data day by day in various fields. Due to the large-scale and high-dimensional characteristics of these data, it is often difficult to achieve better decision-making in practical applications. Therefore, an efficient big data analytical method is urgently necessary. For feature engineering, feature selection seems to be an important research topic which is anticipated to select “excellent” features from candidate ones. The implementation of feature selection can not only achieve the purpose of dimensionality reduction, but also improve the computational efficiency and result performance of the model. In many classification tasks, researchers found that data seem to be usually close to each other if they are from the same class; thus, local compactness is of great importance for the evaluation of a feature. Based on this discovery, we propose a fast unsupervised feature selection algorithm, named Compactness Score (CSUFS), to select desired features. To prove the superiority of the proposed algorithm, several public data sets are considered with extensive experiments being performed. The experiments are presented by applying feature subsets selected through several different algorithms to the clustering task. The performance of clustering tasks is indicated by two well-known evaluation metrics, while the efficiency is reflected by the corresponding running time. As demonstrated, our proposed algorithm is more accurate and efficient compared with existing ones.

Fast Unsupervised Feature Selection with Bipartite Graph and l2,0-Norm Constraint

Since obtaining data labels is a time-consuming and laborious task, unsupervised feature selection has become a popular feature selection technique. However, the current unsupervised feature selection methods are facing three challenges: (1) they rely on a fixed similarity matrix derived from the original data, which will affect their performance; (2) due to the limitation of sparsity, they can only obtain sub-optimal solutions; (3) they have high computational complexity and cannot handle large-scale data. To solve this dilemma, we propose a fast unsupervised feature selection algorithm with bipartite graph and 2;0-norm constraint (BGCFS). We use the original data and the selected anchors to construct an adaptive bipartite graph in the subspace, and apply the l2,0-norm constraint to the projection matrix for feature selection. In this way, we can update the adaptive bipartite graph and the projection matrix simultaneously, and we can get the feature subset directly, without sorting the features. In addition, we propose an iterative algorithm that can solve the proposed problem globally to obtain a closed-form solution, and we provide a strict proof of convergence for it. Experiments on eight real data sets with different scales show that our method can select more valuable feature subsets more quickly

Machine learning for identifying resistance features of Klebsiella pneumoniae using whole-genome sequence single nucleotide polymorphisms.

Introduction. Klebsiella pneumoniae, a gram-negative bacterium, is a common pathogen causing nosocomial infection. The drug-resistance rate of K. pneumoniae is increasing year by year, posing a severe threat to public health worldwide. K. pneumoniae has been listed as one of the pathogens causing the global crisis of antimicrobial resistance in nosocomial infections. We need to explore the drug resistance of K. pneumoniae for clinical diagnosis. Single nucleotide polymorphisms (SNPs) are of high density and have rich genetic information in whole-genome sequencing (WGS), which can affect the structure or expression of proteins. SNPs can be used to explore mutation sites associated with bacterial resistance.Hypothesis/Gap Statement. Machine learning methods can detect genetic features associated with the drug resistance of K. pneumoniae from whole-genome SNP data.Aims. This work used Fast Feature Selection (FFS) and Codon Mutation Detection (CMD) machine learning methods to detect genetic features related to drug resistance of K. pneumoniae from whole-genome SNP data.Methods. WGS data on resistance of K. pneumoniae strains to four antibiotics (tetracycline, gentamicin, imipenem, amikacin) were downloaded from the European Nucleotide Archive (ENA). Sequence alignments were performed with MUMmer 3 to complete SNP calling using K. pneumoniae HS11286 chromosome as the reference genome. The FFS algorithm was applied to feature selection of the SNP dataset. The training set was constructed based on mutation sites with mutation frequency >0.995. Based on the original SNP training set, 70% of SNPs were randomly selected from each dataset as the test set to verify the accuracy of the training results. Finally, the resistance genes were obtained by the CMD algorithm and Venny.Results. The number of strains resistant to tetracycline, gentamicin, imipenem and amikacin was 931, 1048, 789 and 203, respectively. Machine learning algorithms were applied to the SNP training set and test set, and 28 and 23 resistance genes were predicted, respectively. The 28 resistance genes in the training set included 22 genes in the test set, which verified the accuracy of gene prediction. Among them, some genes (KPHS_35310, KPHS_18220, KPHS_35880, etc.) corresponded to known resistance genes (Eef2, lpxK, MdtC, etc). Logistic regression classifiers were established based on the identified SNPs in the training set. The area under the curves (AUCs) of the four antibiotics was 0.939, 0.950, 0.912 and 0.935, showing a strong ability to predict bacterial resistance.Conclusion. Machine learning methods can effectively be used to predict resistance genes and associated SNPs. The FFS and CMD algorithms have wide applicability. They can be used for the drug-resistance analysis of any microorganism with genomic variation and phenotypic data. This work lays a foundation for resistance research in clinical applications.

Fast feature selection algorithm of EEG data based on GPU technology

With the rapid development of EEG technology, the rapid selection of EEG data features makes it possible for subsequent applications. To improve the relevance of EEG data, this paper proposes a feature selection algorithm of EEG data based on GPU technology. This method takes the whole gene pathway as a whole, takes its EEG data path as a specific variable, and quantifies each path by distance measurement; secondly, it uses regularisation dimension reduction technology to brush out the essential features of EEG data and makes use of this method to select the essential features of EEG data. Finally, principal component analysis (PCA) is used to evaluate the path model quickly and accurately. The experimental analysis shows that the method solves the problems of high requirement of EEG data sample distribution and slow evaluation speed. Moreover, the method proposed in this paper shows the great ability of real-time performance.

Fast feature selection algorithm of EEG data based on GPU technology

With the rapid development of EEG technology, the rapid selection of EEG data features makes it possible for subsequent applications. To improve the relevance of EEG data, this paper proposes a feature selection algorithm of EEG data based on GPU technology. This method takes the whole gene pathway as a whole, takes its EEG data path as a specific variable, and quantifies each path by distance measurement; secondly, it uses regularisation dimension reduction technology to brush out the essential features of EEG data and makes use of this method to select the essential features of EEG data. Finally, principal component analysis (PCA) is used to evaluate the path model quickly and accurately. The experimental analysis shows that the method solves the problems of high requirement of EEG data sample distribution and slow evaluation speed. Moreover, the method proposed in this paper shows the great ability of real-time performance.

Fast feature selection for interval-valued data through kernel density estimation entropy

Kernel density estimation, which is a non-parametric method about estimating probability density distribution of random variables, has been used in feature selection. However, existing feature selection methods based on kernel density estimation seldom consider interval-valued data. Actually, interval-valued data exist widely. In this paper, a feature selection method based on kernel density estimation for interval-valued data is proposed. Firstly, the kernel function in kernel density estimation is defined for interval-valued data. Secondly, the interval-valued kernel density estimation probability structure is constructed by the defined kernel function, including kernel density estimation conditional probability, kernel density estimation joint probability and kernel density estimation posterior probability. Thirdly, kernel density estimation entropies for interval-valued data are proposed by the constructed probability structure, including information entropy, conditional entropy and joint entropy of kernel density estimation. Fourthly, we propose a feature selection approach based on kernel density estimation entropy. Moreover, we improve the proposed feature selection algorithm and propose a fast feature selection algorithm based on kernel density estimation entropy. Finally, comparative experiments are conducted from three perspectives of computing time, intuitive identifiability and classification performance to show the feasibility and the effectiveness of the proposed method.

A novel framework of fuzzy oblique decision tree construction for pattern classification

In this paper, some significant efforts on fuzzy oblique decision tree (FODT) have been done to improve classification accuracy and decrease tree size. Firstly, to eliminate data redundancy and improve classification efficiency, a forward greedy fast feature selection algorithm based on neighborhood rough set (NRS_FS_FAST) is introduced. Then, a new fuzzy rule generation algorithm (FRGA) is proposed to generate fuzzy rules. These fuzzy rules are used to construct leaf nodes for each class in each layer of the FODT. Different from the traditional axis-parallel decision trees and oblique decision trees, the FODT takes dynamic mining fuzzy rules as decision functions. Moreover, the parameter δ, which can control the size of the tree, is optimized by genetic algorithm. Finally, a series of comparative experiments are carried out with five traditional decision trees (C4.5, Best First Tree (BFT), amulti-class alternating decision tree (LAD), Simple Cart (SC), Naive Bayes Tree (NBT)), and recently proposed decision trees (FRDT, HHCART, and FMMDT-HB) on UCI machine learning datasets. The experimental results demonstrate that the FODT exhibits better performance on classification accuracy and tree size than the chosen benchmarks.

Heart Disease Identification Method Using Machine Learning Classification in E-Healthcare

Heart disease is one of the complex diseases and globally many people suffered from this disease. On time and efficient identification of heart disease plays a key role in healthcare, particularly in the field of cardiology. In this article, we proposed an efficient and accurate system to diagnosis heart disease and the system is based on machine learning techniques. The system is developed based on classification algorithms includes Support vector machine, Logistic regression, Artificial neural network, K-nearest neighbor, Naive bays, and Decision tree while standard features selection algorithms have been used such as Relief, Minimal redundancy maximal relevance, Least absolute shrinkage selection operator and Local learning for removing irrelevant and redundant features. We also proposed novel fast conditional mutual information feature selection algorithm to solve feature selection problem. The features selection algorithms are used for features selection to increase the classification accuracy and reduce the execution time of classification system. Furthermore, the leave one subject out cross-validation method has been used for learning the best practices of model assessment and for hyperparameter tuning. The performance measuring metrics are used for assessment of the performances of the classifiers. The performances of the classifiers have been checked on the selected features as selected by features selection algorithms. The experimental results show that the proposed feature selection algorithm (FCMIM) is feasible with classifier support vector machine for designing a high-level intelligent system to identify heart disease. The suggested diagnosis system (FCMIM-SVM) achieved good accuracy as compared to previously proposed methods. Additionally, the proposed system can easily be implemented in healthcare for the identification of heart disease.

Multi-objective feature selection based on artificial bee colony: An acceleration approach with variable sample size

Due to the need to repeatedly call a classifier to evaluate individuals in the population, existing evolutionary feature selection algorithms have the disadvantage of high computational cost. In view of it, this paper studies a multi-objective feature selection framework based on sample reduction strategy and evolutionary algorithm, significantly reducing the computational cost of algorithm without affecting optimal results. In the framework, a selection strategy of representative samples, called K-means clustering based differential selection, and a ladder-like sample utilization strategy are proposed to reduce the size of samples used in the evolutionary process. Moreover, a fast multi-objective evolutionary feature selection algorithm, called FMABC-FS, is proposed by embedding an improved artificial bee colony algorithm based on the particle update model into the framework. By applying FMABC-FS to several typical UCI datasets, and comparing with three multi-objective feature selection algorithms, experimental results show that the proposed variable sample size strategy is more suitable to FMABC-FS, and FMABC-FS can obtain better feature subsets with much less running time than those comparison algorithms.

Fast feature selection algorithm for neighborhood rough set model based on Bucket and Trie structures

Feature selection is viewed as the problem of finding the minimal number of features from an original set with the minimum information loss. Due to its high importance in the fields of pattern recognition and data mining, it is necessary to investigate fast and effective search algorithms. In this paper, we introduce a novel fast feature selection algorithm for neighborhood rough set model based on Bucket and Trie structures. This proposed algorithm can guarantee to find the optimal minimal reduct by adopting a global search strategy. In addition, the dependence degree is subsequently used to evaluate the relevance of the attribute subset. Consequently, the proposed algorithm is tested on several standard data sets from UCI repository and compared with the most recent related approaches. The obtained theoretical and experimental results reveal that the present algorithm is very effective and convenient for the problem of feature selection, indicating that it could be useful for many pattern recognition and data mining systems.

A Fast Feature Selection Algorithm by Accelerating Computation of Fuzzy Rough Set-Based Information Entropy.

The information entropy developed by Shannon is an effective measure of uncertainty in data, and the rough set theory is a useful tool of computer applications to deal with vagueness and uncertainty data circumstances. At present, the information entropy has been extensively applied in the rough set theory, and different information entropy models have also been proposed in rough sets. In this paper, based on the existing feature selection method by using a fuzzy rough set-based information entropy, a corresponding fast algorithm is provided to achieve efficient implementation, in which the fuzzy rough set-based information entropy taking as the evaluation measure for selecting features is computed by an improved mechanism with lower complexity. The essence of the acceleration algorithm is to use iterative reduced instances to compute the lambda-conditional entropy. Numerical experiments are further conducted to show the performance of the proposed fast algorithm, and the results demonstrate that the algorithm acquires the same feature subset to its original counterpart, but with significantly less time.

A Fast Feature Selection Algorithm Based on Swarm Intelligence in Acoustic Defect Detection

Feature selection plays an important role in data mining and pattern recognition. However, most existing feature selection methods suffer from stagnation in local optimal and/or high computational cost. In general, the feature selection process can be considered as a global optimization problem. The swarm intelligence algorithm can effectively find the optimal feature subset due to its global search ability. But it usually consumes very long running time when dealing with large data sets. In this paper, feature selection is transformed into a global optimization problem, which provides a fast and efficient method based on swarm intelligence algorithm. First, we propose a global optimization framework for filter-based feature selection and its mathematical model. Furthermore, to solve the feature selection problem for acoustic defect detection, we combine the shuffled frog leaping algorithm (SFLA) with an improved minimum-redundancy maximum-relevancy (ImRMR), named SFLA-ImRMR. In the experiments, a back propagation neural network is employed to evaluate the classification performance of the selected feature subset on the test sets of acoustic defect detection. The results show that SFLA-ImRMR achieved similar performance to the other algorithms within the shortest time.

Performance Comparison Between Fast Fourier Transform-Based Segmentation, Feature Selection, and Fault Identification Algorithm and Neural Network for the Condition Monitoring of Centrifugal Equipment

This paper compares and evaluates the performance of two major feature selection and fault identification methods utilized for the condition monitoring (CM) of centrifugal equipment, namely fast Fourier transform (FFT)-based segmentation, feature selection, and fault identification (FS2FI) algorithm and neural network (NN). Multilayer perceptron (MLP) is the most commonly used NN model for fault pattern recognition. Feature selection and trending play an important role in pattern recognition and hence affect the performance of CM systems. The technical and developmental challenges of both methods were investigated experimentally on a Paxton industrial centrifugal air blower system with a rotational speed of 15,650 RPMs. Five different machine conditions were experimentally emulated in the laboratory. A low training-to-testing ratio of 50% was utilized to evaluate the performance of both methods. In order to maximize fault identification accuracy and minimize computing time and cost, a near-optimal NN configuration was identified. The results showed that both techniques operated with a fault identification accuracy of 100%. However, the FS2FI algorithm showed a number of advantages over NN. These advantages include the ease of implementation and a reduction of cost and time in development and computing, as it processed the data from the first trial in less than 6.2% of the time taken by the NN.

MIL based visual object tracking with kernel and scale adaptation

Multiple instance learning (MIL) is a framework wherein training examples are provided in form of labeled bags rather than labeled instances. For visual object tracking, the MIL framework is used to select a few discriminative features from a pool of features by maximizing the bag likelihood. The feature pool consists of computationally efficient Haar-like features. Each feature is used for the construction of weak classifier also called decision stump. A few classifiers with high discriminative power (to separate target from background) are combined to form a strong classifier. The key point is the selection of a few highly discriminative features from a very large pool in small amount of time to ensure fast and accurate tracking. In this paper, we propose a fast online feature selection algorithm based on maximizing the classifier score (CSR). The tracking performance is better than the existing feature selection methods. This paper also introduces kernel trick on Haar-like features for target tracking. Furthermore, we explore use of Haar-features in half target space. To get the actual target size in successive video frames, a scaling strategy is applied. By having the feature matching using kernel, Haar-features in half target spaces and scale adaptation method in a single tracking framework, we obtain better tracking performance as compared to the state-of-the-art trackers.

考虑了特征协同作用的FAST特征选择算法的改进

交互的特征是指那些分开考虑对目标集不相关或弱相关，但合在一起考虑却对目标集高度相关的特征。特征交互现象广泛存在，但找出有交互作用的特征却是一项具有挑战性的任务。本文旨在对基于聚类的FAST特征选择算法进行改进，在其基础上考虑特征的交互作用，首先去掉FAST的移除不相关特征的部分，接着加入交互权值变量，使得在移除不相关和冗余特征的同时，保留有交互作用的特征。为了对两个算法进行对比分析，我们选取了5个不同领域的16个公开数据集进行实证分析，并使用4种分类器对实验结果进行评估，包括C5.0、Bayes Net、Neural Net和Logistic，接着从选择的特征个数、算法运行时间和分类器的准确率3个方面对两个算法进行比较。实验结果表明，两者选择的特征个数相差不大，有时IWFAST甚至可以减少特征个数，同时IWFAST能提高分类器的准确率，尤其对于特征数量较多的情形，以及Game和Life领域。美中不足的是，IWFAST的运行时间较长，但仍在可接受的范围内。 Interacting features are those appear to be irrelevant or weakly relevant with the class individually, but it may highly correlate to the class when it combined. Feature interaction is almost everywhere, but discovering feature interaction is a challenging task in feature selection. The purpose of this paper is to improve the FAST feature selection algorithm based on cluster by considering feature interaction. Firstly, deleted the irrelevant feature removal section, then brought in an interaction weight factor, so that we can retain interacted features when removed the irrelevant and redundant ones. In order to do the comparison between this two algorithms, we selected 16 public data sets which cover 5 different domains on the empirical analysis, and used 4 types of classifier to evaluate the results, namely, C5.0, Bayes Net, Neural Net and Logistic. Finally, we compared these two algorithms according to the number of selected features, running time of algorithm and the accuracy of classifier. The experimental result showed that it has little difference on the number of selected features, and sometimes IWFAST can produce smaller subsets of features. Meanwhile, IWFAST can improve the accuracy of the classifier, especially for the high- dimensional data set, or especially for the Game and Life area. The defect is that the running time of IWFAST is long, but is acceptable computational complexity.

A Fast and Accurate Feature Selection Algorithm Based on Binary Consistency Measure

Consistency‐based feature selection is an important category of feature selection research, and its advantage over other categories is due to consistency measures used to include the effect of interaction among features into evaluation of relevance of features. Even if features individually appear irrelevant to class labels, they can collectively show strong relevance. In such cases, we say that the features interact with each other. Consistency measures, in this regard, evaluate the collective relevance of a set of features and has been intuitively understood as a metric to measure a distance of an arbitrary feature set from the state of being consistent: A set of features is said to be consistent if, and only if, they as a whole determine class labels. In history, the binary consistency measure, which returns the value 1 if the feature set is consistent and 0 otherwise, was the first consistency measure introduced, and many advanced measures followed. The problem of the binary measure consists in the fact that it always returns 1 if a data set includes no consistent feature set. The measures that followed have solved this problem but sacrificed time efficiency of evaluation. Therefore, feature selection leveraging these measures are not fast enough to apply to large data sets. In this article, we aim to improve time efficiency of consistency‐based feature selection. To achieve the goal, we propose a new idea, which we call data set denoising: We eliminate examples which are viewed as noises from a data set until the data set becomes to include consistent feature sets and then apply the binary measure to find an appropriate feature set that is consistent. In our evaluation through intensive experiments, CWC, a new algorithm that implements data set denoising outperformed in both time efficiency and accuracy the benchmark consistency‐based algorithms. Specifically, CWC was about 31 times faster than the LCC that had been known as the fastest in the literature. Furthermore, in a comparison including feature selection algorithms that are not consistency‐based, CWC has turned out to be one of the fastest and the most accurate feature selection algorithms.

Efficient Feature Subset Selection using Kruskal'sProcess in Big Data

Feature selection involves identifying a subset of the most useful features that produces compatible results as the original entire set of features. A feature selection algorithm may be evaluated from both the efficiency and effectiveness points of view. While the efficiency concerns the time required to find a subset of features, the effectiveness is related to the quality of the subset of features. Based on these criteria, a fast clustering-based feature selection algorithm, FAST, is proposed and experimentally evaluated. The FAST algorithm works in two steps. In the first step, features are divided into clusters by using graph-theoretic clustering methods. In the second step, the most representative feature that is strongly related to target classes is selected from each cluster to form a subset of features.