Hierarchical Feature Selection Research Articles

Feature selection via maximizing inter-class independence and minimizing intra-class redundancy for hierarchical classification

Hierarchical feature selection has proven to be significant in reducing classification difficulty. Many existing hierarchical feature selection methods use the hierarchy in the class space as structural information to improve performance. However, these methods focus on the inter-class structural relations and ignore the intra-class feature correlations, which is challenging as the numbers of classes and features in current datasets are both growing rapidly. In this paper, we propose a hierarchical feature selection method that maximizes inter-class independence and minimizes intra-class redundancy using structure and feature relations. First, we investigate the class hierarchy dependency in the tree structure as the structural relation regularization, which maximizes the independence between unrelated classes in the class structure. Second, we transform the feature correlations into a mathematical representation as feature relation regularization, which minimizes the redundancy in each class on the premise of sparsity. Finally, we unify the two regularization terms into a hierarchical feature selection method to trade off the structural and feature relations. Our method exhibits excellent efficiency and effectiveness compared with other feature selection methods.

A coastal wetlands mapping approach of Yellow River Delta with a hierarchical classification and optimal feature selection framework

Wetlands play an important role in ecological health and sustainable development, their spatial distribution and explicit thematic information are crucial for developing management and conservation measures. The Yellow River Delta is an important coastal wetland reserve in China, its wetland types are complex and diverse, natural and artificial wetlands are easily confused, making refined classification more difficult. To address this challenge, we proposed a new wetland mapping approach by combing hierarchical classification framework (HCF) and optimal feature selection. First, inheritance-based multiscale segmentation was carried out to obtain object-oriented images, and decision tree classification was used for preliminarily identify wetland and non-wetland. Second, recursive feature elimination and cross-validation (RFECV) was used to select optimal features, which was then utilized for wetland refinement extraction by using random forest (RF) algorithm. The experiments were performed based on Sentinel-1, Sentinel-2 and NASADEM datasets. The results show that effective wetland classification features can be selected by using RFECV. The feature scores are as follows, red edge index > spectral features > vegetation/water body index > backscatter coefficient > topographic features > texture features > location feature > urban index > geometric feature. The overall accuracy and Kappa coefficient of the method in this paper are 92.36 % and 0.915, which are 14.62 % and 6.68 % higher than using only HCF or only RFECV. Compared with the GlobeLand30 and CAS_Wetlands datasets, the refinement of wetland mapping in this paper is higher. This study provides a new idea in methodological selection for wetland information extraction, and the resulting coastal wetland map can be used for sustainable management, ecological assessment and conservation of the Yellow River Delta.

Hierarchical Feature Fusion and Selection for Hyperspectral Image Classification

Most existing classification methods design complicated and large deep neural network (DNN) model to deal with the ubiquitous spectral variability and nonlinearity of hyperspectral images (HSIs). However, their application is blocked by limited training samples and considerable computational costs in real scenes. To solve these problems, we propose a simple spectral hierarchical feature fusion and selection network (HFFSNet). Specifically, we apply 1-D grouped convolution for dimensionality reduction and multilevel feature extraction, then the multilevel features are fused to assist the adaptive feature selection of different layer features via the soft attention mechanism, and finally the selected features are fused to further enhance the feature representation. Extensive experimental results on three hyperspectral datasets demonstrate the effectiveness of the proposed network.

Performance Analysis of Coal Gangue Recognition Based on Hierarchical Filtering and Coupled Wrapper Feature Selection Method

Performance Analysis of Coal Gangue Recognition Based on Hierarchical Filtering and Coupled Wrapper Feature Selection Method

Hierarchical Voting-Based Feature Selection and Ensemble Learning Model Scheme for Glioma Grading with Clinical and Molecular Characteristics.

Determining the aggressiveness of gliomas, termed grading, is a critical step toward treatment optimization to increase the survival rate and decrease treatment toxicity for patients. Streamlined grading using molecular information has the potential to facilitate decision making in the clinic and aid in treatment planning. In recent years, molecular markers have increasingly gained importance in the classification of tumors. In this study, we propose a novel hierarchical voting-based methodology for improving the performance results of the feature selection stage and machine learning models for glioma grading with clinical and molecular predictors. To identify the best scheme for the given soft-voting-based ensemble learning model selections, we utilized publicly available TCGA and CGGA datasets and employed four dimensionality reduction methods to carry out a voting-based ensemble feature selection and five supervised models, with a total of sixteen combination sets. We also compared our proposed feature selection method with the LASSO feature selection method in isolation. The computational results indicate that the proposed method achieves 87.606% and 79.668% accuracy rates on TCGA and CGGA datasets, respectively, outperforming the LASSO feature selection method.

New Algorithm of Traditional Chinese Medicine and Protection of Intangible Cultural Heritage Based on Big Data Deep Learning.

Traditional Chinese medicine (TCM) is a summary of the diagnosis and treatment experience formed by the working people in the long-term struggle against diseases, so it is very important to protect the intangible cultural heritage of TCM. How to extract valuable knowledge accurately and conveniently from the massive medical records of TCM is one of the important issues in the current research on the development of TCM. Due to the large amount of data of TCM medical records, many feature attributes, and diverse patterns, the existing classification technology has high computational complexity, low mining efficiency, and poor universality. Therefore, this paper proposed to quantify the medical records of TCM and obtained the main symptoms according to the improved hierarchical clustering feature selection algorithm. This paper also proposed a support vector machine (SVM) classification method using improved particle swarm algorithm to classify TCM information, which not only improves the efficiency and accuracy of TCM information classification but also discovers the potential dialectical and symptom patterns in diagnosis and treatment, so that the intangible cultural heritage protection of TCM can be developed sustainably. This paper showed that the information acquisition accuracy of the improved algorithm was very high. Before the improved algorithm was used, the accuracy of information mining for TCM was 67.90% at the highest and 65.53% at the lowest, but after using the improved algorithm, the accuracy rate of information mining for TCM was 88.02% at the highest and 82.45% at the lowest. It can be seen that using the improved algorithm to mine TCM information can quickly process effective information.

Multi-view dimensionality reduction learning with hierarchical sparse feature selection

Multi-view dimensionality reduction learning with hierarchical sparse feature selection

Hierarchical unsupervised multi-view feature selection

Multi-view feature selection is an important research direction in the field of multi-view learning. To remove the negative effect of irrelevant and redundant information and select the important features from the multi-view data, in this paper, we propose a novel hierarchical unsupervised multi-view feature selection method, which integrates matrix decomposition and hierarchical regularization as a joint model. Specifically, to exploit the consistency information of multiple views for projection matrix, we project the multiple views into a latent basis space based on a matrix decomposition model. The hierarchical regularization containing the [Formula: see text]-norm, dependence and Frobenius-norm is imposed on projection matrix to, respectively, exploit the row-level, dependency-level and view-level feature selection. We also develop an efficient optimization algorithm to optimize our method. Extensive experimental results on six popular multi-view datasets show the effectiveness and superiority of our method by comparing with the state-of-the-art methods.

A novel hierarchical feature selection method based on large margin nearest neighbor learning

The complexity of data has been increasing not only in the dimension of the data, but also in the interrelated class relationships. Pre-defined hierarchical structure of classes helps to model such relationships in a way, which divides the whole classification problem into a set of subtasks. Feature selection, known as a data preprocessing technique, aims to selecting useful feature to improve the performance of learning algorithms. However, the issue of close class relationships, especially under the same parent node in the hierarchical structure, is adverse to the feature selection. In this paper, we introduce a novel distance metric based feature selection method with large margin nearest neighbor strategy for the hierarchical classification problems. Different from other algorithms, our proposed method is to project the data into a low-dimensional feature space for learning correct metrics, and the weights of features will be indicated in the transformation matrix. A sparsity regularization and iterative optimization ensure that superior feature subsets can be selected. Experimental evaluation on several datasets with hierarchical class structure demonstrates that the proposed approach is comparable or better than some other state-of-the-art hierarchical feature selection methods.

Automated prediction of COVID-19 mortality outcome using clinical and laboratory data based on hierarchical feature selection and random forest classifier

Early prediction of COVID-19 mortality outcome can decrease expiration risk by alerting healthcare personnel to assure efficient resource allocation and treatment planning. This study introduces a machine learning framework for the prediction of COVID-19 mortality using demographics, vital signs, and laboratory blood tests (complete blood count (CBC), coagulation, kidney, liver, blood gas, and general). 41 features from 244 COVID-19 patients were recorded on the first day of admission. In this study, first, the features in each of the eight categories were investigated. Afterward, features that have an area under the receiver operating characteristic curve (AUC) above 0.6 and the p-value criterion from the Wilcoxon rank-sum test below 0.005 were used as selected features for further analysis. Then five feature reduction methods, Forward Feature selection, minimum Redundancy Maximum Relevance, Relieff, Linear Discriminant Analysis, and Neighborhood Component Analysis were utilized to select the best combination of features. Finally, seven classifiers frameworks, random forest (RF), support vector machine, logistic regression (LR), K nearest neighbors, Artifical neural network, bagging, and boosting were used to predict the mortality outcome of COVID-19 patients. The results revealed that the combination of features in CBC and then vital signs had the highest mortality classification parameters, respectively. Furthermore, the RF classifier with hierarchical feature selection algorithms via Forward Feature selection had the highest classification power with an accuracy of 92.08 ± 2.56. Therefore, our proposed method can be confidently used as a valuable assistant prognostic tool to sieve patients with high mortality risks.

A fuzzy rough set approach to hierarchical feature selection based on Hausdorff distance

A fuzzy rough set approach to hierarchical feature selection based on Hausdorff distance

LwTool: A data processing toolkit for building a real-time pressure mapping smart textile software system

Pressure mapping smart textile is a new type of sensing modality that transforms the pressure distribution over surfaces into digital ”image” and ”video”, that has rich application scenarios in Human Activity Recognition (HAR), because all human activities are linked with force change over certain surfaces. To speed up its application exploration, we propose a toolkit named LwTool for the data processing, including: (a) a feature library, including 1830 ready-to-use temporal and spatial features, (b) a hierarchical feature selection framework that automatically picks out the best features for a new application from the feature library. As real-time processing capability is important for instant user feedback, we emphasize not only on good recognition result but also on reducing time cost when selecting features. Our library and algorithms are validated on Smart-Toy and Smart-Bedsheet applications, an 89.7% accuracy for Smart-Toy and an 83.8% accuracy for Smart-Bedsheet can be achieved (10-fold cross-validation) using our feature library. Adopting the feature selection algorithm, the processing speed is increased by more than 3 times while maintaining high accuracy for both two applications. We believe our method could be a general and powerful toolkit in building real-time recognition software systems for pressure mapping smart textile.

Hierarchical Feature Selection Based on Label Distribution Learning

Hierarchical classification learning, which organizes data categories into a hierarchical structure, is an effective approach for large-scale classification tasks. The high dimensionality of data feature space, represented in hierarchical class structures, is one of the main research challenges. In addition, the class hierarchy often introduces imbalanced class distributions and causes overfitting. In this paper, we propose a feature selection method based on label distribution learning to address the above challenges. The crux is to alleviate the class imbalance problem and learn a discriminative feature subset for hierarchical classification process. Due to correlation between different class categories in the hierarchical tree structure, sibling categories can provide additional supervisory information for each learning sub tasks, which, in turn, alleviates the problem of under-sampling of minority categories. Therefore, we transform hierarchical labels to a hierarchical label distribution to represent this correlation. After that, a discriminative feature subset is selected recursively, by the common features and label-specific feature constraints, to ensure that downstream classification tasks can achieve the best performance. Experiments and comparisons, using seven well-established feature selection algorithms on six real data sets with different degrees of imbalance, demonstrate the superiority of the proposed method.

Transmission Line Fault-Cause Identification Based on Hierarchical Multiview Feature Selection

Fault-cause identification plays a significant role in transmission line maintenance and fault disposal. With the increasing types of monitoring data, i.e., micrometeorology and geographic information, multiview learning can be used to realize the information fusion for better fault-cause identification. To reduce the redundant information of different types of monitoring data, in this paper, a hierarchical multiview feature selection (HMVFS) method is proposed to address the challenge of combining waveform and contextual fault features. To enhance the discriminant ability of the model, an ε-dragging technique is introduced to enlarge the boundary between different classes. To effectively select the useful feature subset, two regularization terms, namely l2,1-norm and Frobenius norm penalty, are adopted to conduct the hierarchical feature selection for multiview data. Subsequently, an iterative optimization algorithm is developed to solve our proposed method, and its convergence is theoretically proven. Waveform and contextual features are extracted from yield data and used to evaluate the proposed HMVFS. The experimental results demonstrate the effectiveness of the combined used of fault features and reveal the superior performance and application potential of HMVFS.

HFS-SLPEE: A Novel Hierarchical Feature Selection and Second Learning Probability Error Ensemble Model for Precision Cancer Diagnosis.

The emergence of high-throughput RNA-seq data has offered unprecedented opportunities for cancer diagnosis. However, capturing biological data with highly nonlinear and complex associations by most existing approaches for cancer diagnosis has been challenging. In this study, we propose a novel hierarchical feature selection and second learning probability error ensemble model (named HFS-SLPEE) for precision cancer diagnosis. Specifically, we first integrated protein-coding gene expression profiles, non-coding RNA expression profiles, and DNA methylation data to provide rich information; afterward, we designed a novel hierarchical feature selection method, which takes the CpG-gene biological associations into account and can select a compact set of superior features; next, we used four individual classifiers with significant differences and apparent complementary to build the heterogeneous classifiers; lastly, we developed a second learning probability error ensemble model called SLPEE to thoroughly learn the new data consisting of classifiers-predicted class probability values and the actual label, further realizing the self-correction of the diagnosis errors. Benchmarking comparisons on TCGA showed that HFS-SLPEE performs better than the state-of-the-art approaches. Moreover, we analyzed in-depth 10 groups of selected features and found several novel HFS-SLPEE-predicted epigenomics and epigenetics biomarkers for breast invasive carcinoma (BRCA) (e.g., TSLP and ADAMTS9-AS2), lung adenocarcinoma (LUAD) (e.g., HBA1 and CTB-43E15.1), and kidney renal clear cell carcinoma (KIRC) (e.g., IRX2 and BMPR1B-AS1).

Hierarchical feature selection with multi-granularity clustering structure

Hierarchical feature selection addresses the issues caused by the presence of high-dimensional features in multi-category classification systems with hierarchical structures. Granular calculations are made to analyze the hierarchical relationships among categories when selecting the optimal feature subset. However, semantic hierarchy-based feature selection methods are prone to the semantic gap problem, which affects classification accuracy. In this paper, we propose a hierarchical feature selection method with a multi-granularity clustering structure that can effectively alleviate the semantic gap problem. Firstly, a hierarchical structure is constructed via bottom-up multi-granularity clustering based on feature similarities rather than semantic categories. This clustering hierarchy is conducive to solving semantic gap problems in the existing hierarchy. Secondly, the optimal feature subset is selected using the ℓ1,2-norms in each hierarchy’s granularity layer. This joint minimization approach can retain both the granularity layers’ shared features and granularity-specific features. Finally, we execute hierarchical classification according to the granular structure in a coarse to fine sequence. Extensive experiments demonstrate that the proposed method outperforms several state-of-the-art hierarchical feature selection approaches.

Harvestman: a framework for hierarchical feature learning and selection from whole genome sequencing data

BackgroundSupervised learning from high-throughput sequencing data presents many challenges. For one, the curse of dimensionality often leads to overfitting as well as issues with scalability. This can bring about inaccurate models or those that require extensive compute time and resources. Additionally, variant calls may not be the optimal encoding for a given learning task, which also contributes to poor predictive capabilities. To address these issues, we present Harvestman, a method that takes advantage of hierarchical relationships among the possible biological interpretations and representations of genomic variants to perform automatic feature learning, feature selection, and model building.ResultsWe demonstrate that Harvestman scales to thousands of genomes comprising more than 84 million variants by processing phase 3 data from the 1000 Genomes Project, one of the largest publicly available collection of whole genome sequences. Using breast cancer data from The Cancer Genome Atlas, we show that Harvestman selects a rich combination of representations that are adapted to the learning task, and performs better than a binary representation of SNPs alone. We compare Harvestman to existing feature selection methods and demonstrate that our method is more parsimonious—it selects smaller and less redundant feature subsets while maintaining accuracy of the resulting classifier.ConclusionHarvestman is a hierarchical feature selection approach for supervised model building from variant call data. By building a knowledge graph over genomic variants and solving an integer linear program , Harvestman automatically and optimally finds the right encoding for genomic variants. Compared to other hierarchical feature selection methods, Harvestman is faster and selects features more parsimoniously.

Robust hierarchical feature selection with a capped [formula omitted]-norm

Feature selection methods face new challenges in large-scale classification tasks because massive categories are managed in a hierarchical structure. Hierarchical feature selection can take full advantage of the dependencies among hierarchically structured classes. However, most of the existing hierarchical feature selection methods are not robust for dealing with the inevitable data outliers, resulting in a serious inter-level error propagation problem in the following classification process. In this paper, we propose a robust hierarchical feature selection method with a capped ℓ2-norm (HFSCN), which can reduce the adverse effects of data outliers and learn relatively robust and discriminative feature subsets for the hierarchical classification process. Firstly, a large-scale global classification task is split into several small local sub-classification tasks according to the hierarchical class structure and the divide-and-conquer strategy, which makes it easy for feature selection modeling. Secondly, a capped ℓ2-norm based loss function is used in the feature selection process of each local sub-classification task to eliminate the data outliers, which can alleviate the negative effects outliers and improve the robustness of the learned feature weighted matrix. Finally, an inter-level relation constraint based on the similarity between the parent and child classes is added to the feature selection model, which can enhance the discriminative ability of the selected feature subset for each sub-classification task with the learned robust feature weighted matrix. Compared with seven traditional and state-of-art hierarchical feature selection methods, the superior performance of HFSCN is verified on 16 real and synthetic datasets.

Mental Arithmetic Task Recognition Using Effective Connectivity and Hierarchical Feature Selection From EEG Signals.

Introduction:Mental arithmetic analysis based on Electroencephalogram (EEG) signals can help understand disorders, such as attention-deficit hyperactivity, dyscalculia, or autism spectrum disorder where the difficulty in learning or understanding the arithmetic exists. Most mental arithmetic recognition systems rely on features of a single channel of EEG; however, the relationships between EEG channels in the form of effective brain connectivity analysis can contain valuable information. This study aims to find distinctive, effective brain connectivity features and create a hierarchical feature selection for effectively classifying mental arithmetic and baseline tasks.Methods:We estimated effective connectivity using Directed Transfer Function (DTF), direct DTF (dDTF) and Generalized Partial Directed Coherence (GPDC) methods. These measures determine the causal relationship between different brain areas. A hierarchical feature subset selection method selects the most significant effective connectivity features. Initially, Kruskal– Wallis test was performed. Consequently, five feature selection algorithms, namely, Support Vector Machine (SVM) method based on Recursive Feature Elimination, Fisher score, mutual information, minimum Redundancy Maximum Relevance (RMR), and concave minimization and SVM are used to select the best discriminative features. Finally, the SVM method was used for classification.Results:The obtained results indicated that the best EEG classification performance in 29 participants and 60 trials is obtained using GPDC and feature selection via concave minimization method in Beta2 (15-22Hz) frequency band with 89% accuracy.Conclusion:This new hierarchical automated system could be helpful in the discrimination of mental arithmetic and baseline tasks from EEG signals effectively.HighlightsPropose effective connectivity to describe EEG signals during mental arithmetic task.Most significant connectivity features from generalized partial directed coherence method.Hierarchical feature selection from Kruskal-Wallis test and concave minimization method.Plain Language SummaryBrain analysis methods by Electroencephalogram (EEG) signals provide a suitable method to monitor human brain activity due to having high temporal resolution, being noninvasive, inexpensive, and portable method. Analysis of mental arithmetic based EEG signal is helpful for psychological disorders like dyscalculia where they have learning understanding arithmetic, attention deficit hyperactivity, and autism spectrum disorders with attention deficit problem. This study finds distinctive effective brain connectivity features and creates a hierarchical feature selection for classification of mental arithmetic and baseline tasks effectively. Best EEG classification performance in 29 participants and 60 trials is obtained using Generalized Partial Directed Coherence (GPDC) methods and feature selection via concave minimization method in Beta2 (15−22Hz) frequency band with 89% accuracy. Thus, this new hierarchical automated system is useful for discrimination of mental arithmetic and baseline tasks from EEG signal effectively.

Graph Synthesis for Pig Breed Classification From Muzzle Images

Non-intrusive and automated detection of pig breeds, particularly from visual standpoint, is important from a food quality tracking perspective. In this work, colour as well as texture based visual descriptors from muzzle images have been identified, which, serve as breed-identifiers to separate four common pig-breeds: Duroc, Ghungroo, Hampshire and Yorkshire. While these handcrafted visual descriptors by themselves are fairly robust and discriminative, it is recognized that by controlling the decision space by choosing the feature-type based on colour or texture or both and the order in which particular breeds are siphoned, classification accuracy can be improved considerably. In that light, a stable, relatively data-independent, breed-specific, hierarchical tree synthesis and feature selection procedure is proposed based on a breed-pair cluster separation table. The proposed approach has been compared with the state of the art Phylogenetic distance based Hierarchical Agglomerative Clustering algorithm (AGNES) and also with the standard decision tree classification algorithm. On cross-validation, When completely different sets of pigs were used for training and testing (50-50 split), the proposed algorithm reported relatively high mean classification accuracies of 86.45% for Duroc, 93.02% for Ghungroo, 86.91% for Hampshire and 98.54% for Yorkshire, respectively.