Hierarchical Feature Selection Research Articles

Classification of mental workload using brain connectivity and machine learning on electroencephalogram data

Mental workload refers to the cognitive effort required to perform tasks, and it is an important factor in various fields, including system design, clinical medicine, and industrial applications. In this paper, we propose innovative methods to assess mental workload from EEG data that use effective brain connectivity for the purpose of extracting features, a hierarchical feature selection algorithm to select the most significant features, and finally machine learning models. We have used the Simultaneous Task EEG Workload (STEW) dataset, an open-access collection of raw EEG data from 48 subjects. We extracted brain-effective connectivities by the direct directed transfer function and then selected the top 30 connectivities for each standard frequency band. Then we applied three feature selection algorithms (forward feature selection, Relief-F, and minimum-redundancy-maximum-relevance) on the top 150 features from all frequencies. Finally, we applied sevenfold cross-validation on four machine learning models (support vector machine (SVM), linear discriminant analysis, random forest, and decision tree). The results revealed that SVM as the machine learning model and forward feature selection as the feature selection method work better than others and could classify the mental workload levels with accuracy equal to 89.53% (± 1.36).

A novel hierarchical feature selection with local shuffling and models reweighting for stock price forecasting

Stock price forecasting is a challenging task due to the complexity of financial markets and the high volatility of stocks. Because of the strong nonlinear representation ability of neural network models, such as long-short memory network (LSTM) and deep learning, they are used extensively in recent years for stock price forecasting. However, due to the volatility of financial data, neural network models often suffer from overfitting or instability problems. In addition, quantitative trading with feature mining tools can generate a growing number of features for financial data. Therefore, selecting effective features for financial data is an urgent problem. To address these problems, we propose a novel hierarchical feature selection with local shuffling (HFSLS) and models reweighting (MR) based on LSTM, named HFSLSMR-LSTM, for stock price forecasting. Specifically, for each layer, local shuffling perturbs each feature to re-predict, and its predicted value is compared with the true value to calculate the feature importance, and the important features are selected and returned to the next layer. Besides, a proximity reweighting scheme is presented to adjust the weight for each layer model that learns from hierarchical features. The HFSLSMR-LSTM model is still effective for financial data with multiple features and frequent fluctuations. Experimental results on stock index dataset and Dow Jones Industrial Average dataset demonstrate that the HFSLSMR-LSTM outperforms Informer, DoubleEnsemble, LSTM, GRU, BI-GRU and BI-LSTM on the metrics MSE, RMSE, MAE, MAPE and R2.

Online hierarchical streaming feature selection based on adaptive neighborhood rough set

In the era of open machine learning, a kind of data is accompanied by a hierarchical structure between classes in the label space and the increasing number of features. Therefore, hierarchical classification learning in dynamic changing feature spaces remains an essential research challenge. To address this challenge, we propose an online hierarchical streaming feature selection algorithm based on adaptive neighborhood rough set in this paper, which effectively selects closely interactive features in high-dimensional data with a hierarchical structure. First, a subtree instance set is obtained for each internal node based on the parent–child relationship of the hierarchical structure. Then, an adaptive neighborhood rough set is constructed, and the neighborhood granularity of each instance is auto selected according to the subtree instance set of its parent node. Moreover, an online streaming feature selection framework for hierarchical data is proposed, in which, the streaming feature is evaluated by three steps: online significance analysis, online relevance analysis, and online redundancy analysis. Finally, experiments are conducted on seven hierarchical datasets to evaluate the performance of the proposed algorithm, and extensive results demonstrate that the proposed algorithm outperforms other state-of-the-art online streaming feature selection algorithms.

A portfolio trading system using a novel pixel graph network for stock selection and a mean-CDaR optimization for portfolio rebalancing

Due to the complexity and dynamic nature of financial markets, portfolio management tasks require continuous adaptation with market intelligence. Furthermore, to make profitable trading decisions, it is essential to predict the future performance of the market. To that end, this study proposes a dynamic portfolio trading system in two stages: stock trend prediction and portfolio optimization. In the first stage, although many researchers proposed powerful classification models to predict BUY/SELL trading signals, some scholars only focused on converting time series' features into multichannel images without considering any spatial relationships between features. Considering the relational capacity of graphs, this study proposes a novel image-based classification model, called Pixel Graph Network (PGN), to analyze the constructed images with propagating information from all neighboring pixels, build up a representation of the images as special graphs, and get advantages of a graph-based learning algorithm. In the second stage, stocks with predicted BUY trading signals in the following days, are fed into the Mean–Conditional Drawdown at Risk (M-CDaR) optimization model to prevent any significant drawdown of the investment portfolio. Moreover, a hierarchical feature selection algorithm has been proposed, combined with a continuous trend labeling method to improve the efficiency of the training process. To demonstrate the superiority of our proposed PGN model in comparison with benchmarks, 18 stocks from the New York Stock Exchange (NYSE) are used for numerical experiments. The obtained results show the improvement and effectiveness of the proposed model in both stock trend prediction and portfolio optimization. In terms of financial performance, the proposed PGN+M-CDaR portfolio-based trading system achieves an average annual return of 112%, an average annual Sortino ratio of 10.78, and an average annual Sharpe ratio of 2.79 which are promising to achieve substantial profits while mitigating downside risk.

Research on multi-granularity imbalanced knowledge condition monitoring for mechanical equipment based on hierarchical ELM in multi-entropy space

Condition monitoring is essentially a complex multi-classification task that requires struggling with multiple conditions and imbalanced data. To address this issue, this study proposes a hierarchical cognition method that utilizes multi-entropy feature representation to capture the state of equipment under complex working conditions. Firstly, this study proposes a feature modeling method for mechanical equipment based on multi-entropy space, utilizing the cloud model as a space component. Secondly, the study uses multi-entropy feature data to develop a hierarchical Extreme Learning Machine (HELM) fault identification model for complex working conditions and data imbalance inspired by human hierarchical cognitive behavior. The feature data are decomposed into four levels, namely speed, load, abnormality, and fault, respectively. Additionally, a hierarchical feature selection mechanism is designed to improve the efficiency of HELM. Finally, comparative experiments are carried out on two platforms. Experiments show that the If factors of the two datasets in the multi-entropy space reach 0.6888 and 10.4881, respectively. At the same time, in the classification experiment for the imbalanced data, the classification accuracy reaches 90.68 when the fault samples are 10. The results show that the multi-entropy feature created in this study has a specific representation ability for mechanical states. Furthermore, the proposed hierarchical extreme learning method can effectively identify faults under multiple conditions and data imbalance backgrounds.

An incremental approach to hierarchical feature selection by applying fuzzy rough set technique

An incremental approach to hierarchical feature selection by applying fuzzy rough set technique

Accurate non-invasive grading of nonfunctional pancreatic neuroendocrine tumors with a CT derived radiomics signature

PurposeThe purpose of this study was to develop a radiomics-signature using computed tomography (CT) data for the preoperative prediction of grade of nonfunctional pancreatic neuroendocrine tumors (NF-PNETs). Materials and methodsA retrospective study was performed on patients undergoing resection for NF-PNETs between 2010 and 2019. A total of 2436 radiomic features were extracted from arterial and venous phases of pancreas-protocol CT examinations. Radiomic features that were associated with final pathologic grade observed in the surgical specimens were subjected to joint mutual information maximization for hierarchical feature selection and the development of the radiomic-signature. Youden-index was used to identify optimal cutoff for determining tumor grade. A random forest prediction model was trained and validated internally. The performance of this tool in predicting tumor grade was compared to that of EUS-FNA sampling that was used as the standard of reference. ResultsA total of 270 patients were included and a fusion radiomic-signature based on 10 selected features was developed using the development cohort (n = 201). There were 149 men and 121 women with a mean age of 59.4 ± 12.3 (standard deviation) years (range: 23.3–85.0 years). Upon internal validation in a new set of 69 patients, a strong discrimination was observed with an area under the curve (AUC) of 0.80 (95% confidence interval [CI]: 0.71–0.90) with corresponding sensitivity and specificity of 87.5% (95% CI: 79.7–95.3) and 73.3% (95% CI: 62.9–83.8) respectively. Of the study population, 143 patients (52.9%) underwent EUS-FNA. Biopsies were non-diagnostic in 26 patients (18.2%) and could not be graded due to insufficient sample in 42 patients (29.4%). In the cohort of 75 patients (52.4%) in whom biopsies were graded the radiomic-signature demonstrated not different AUC as compared to EUS-FNA (AUC: 0.69 vs. 0.67; P = 0.723), however greater sensitivity (i.e., ability to accurately identify G2/3 lesion was observed (80.8% vs. 42.3%; P < 0.001). ConclusionNon-invasive assessment of tumor grade in patients with PNETs using the proposed radiomic-signature demonstrated high accuracy. Prospective validation and optimization could overcome the commonly experienced diagnostic uncertainty in the assessment of tumor grade in patients with PNETs and could facilitate clinical decision-making.

ABVS-Based Radiomics for Early Predicting the Efficacy of Neoadjuvant Chemotherapy in Patients with Breast Cancers.

Neoadjuvant chemotherapy (NAC) plays a significant role in breast cancer (BC) management; however, its efficacy varies among patients. Current evaluation methods may lead to delayed treatment alterations, and traditional imaging modalities often yield inaccurate results. Radiomics, an emerging field in medical imaging, offers potential for improved tumor characterization and personalized medicine. Nevertheless, its application in early and accurately predicting NAC response remains underinvestigated. This study aims to develop an automated breast volume scanner (ABVS)-based radiomics model to facilitate early detection of suboptimal NAC response, ultimately promoting personalized therapeutic approaches for BC patients. This retrospective study involved 248 BC patients receiving NAC. Standard guidelines were followed, and patients were classified as responders or non-responders based on treatment outcomes. ABVS images were obtained before and during NAC, and radiomics features were extracted using the PyRadiomics toolkit. Inter-observer consistency and hierarchical feature selection were assessed. Three machine learning classifiers, logistic regression, support vector machine, and random forest, were trained and validated using a five-fold cross-validation with three repetitions. Model performance was comprehensively evaluated based on discrimination, calibration, and clinical utility. Of the 248 BC patients, 157 (63.3%) were responders, and 91 (36.7%) were non-responders. Radiomics feature selection revealed 7 pre-NAC and 6 post-NAC ABVS features, with higher weights for post-NAC features (min >0.05) than pre-NAC (max <0.03). The three post-NAC classifiers demonstrated AUCs of approximately 0.9, indicating excellent discrimination. DCA curves revealed a substantial net benefit when the threshold probability exceeded 40%. Conversely, the three pre-NAC classifiers had AUCs between 0.7 and 0.8, suggesting moderate discrimination and limited clinical utility based on their DCA curves. The ABVS-based radiomics model effectively predicted suboptimal NAC responses in BC patients, with early post-NAC classifiers outperforming pre-NAC classifiers in discrimination and clinical utility. It could enhance personalized treatment and improve patient outcomes in BC management.

Online feature selection for hierarchical classification learning based on improved ReliefF

AbstractIn hierarchical classification learning, the feature space of data has high dimensionality and is unknown with emergent features. To solve the above problems, we propose an online hierarchical feature selection algorithm based on adaptive ReliefF. Firstly, ReliefF is adaptively improved via using the density information of instances around the target sample, making it unnecessary to prespecify parameters. Secondly, the hierarchical relationship between classes is used, and a new method for calculating the feature weight of hierarchical data is defined. Then, an online correlation analysis method based on feature interaction is designed. Finally, the adaptive ReliefF algorithm is improved based on feature redundancy, and the feature weight is scaled by the correlation between features in order to achieve the dynamic updating of feature redundancy. A large number of experiments verify the effectiveness of the proposed algorithm.

HHFS: A Hybrid Hierarchical Feature Selection Method for Ageing Gene Classification

As one of the most complicated processes in biological development, ageing remains poorly understood. These days more and more ageing-related gene datasets become available on the web, where each instance is characterized by a set of hierarchically-organized binary features. Traditional data mining methods show limitations in exploiting this hierarchical feature space. This paper proposes a hybrid hierarchical feature selection (HHFS) method for classifying genes into pro-longevity or anti-longevity ones. HHFS conducts lazy and eager feature selections sequentially, taking into account both uniqueness of a test instance and the whole characteristics of datasets. It adopts two complementary relevancy metrics (i.e., Gini purity and mutual information) to remove hierarchical redundancy. The experiments are conducted based on the ageing-related gene data of four model organisms. The results show that HHFS achieves significantly better prediction performance than several state-of-the-art methods.

Semantic-gap-oriented feature selection in hierarchical classification learning

Semantic gap is common issues in classification tasks. Different from traditional classification tasks, the semantic gap problem can introduce serious error propagation for hierarchical classification. In this paper, we propose a hierarchical feature selection method with label enhancement (HFSLE) to solve this challenge, which estimates the label distribution and selects subsets of features for each subtask. Firstly, HFSLE combines the label enhancement and hierarchical feature selection process so that the label enhancement benefits from the feedback of the loss function. Then, the label enhancement is constrained by the consistency of the parent-child distribution to obtain a plausible label distribution. Finally, feature selection is optimized based on intra-class consistency hierarchical regularization terms, which unifies feature subsets with hierarchical semantic descriptions to narrow the semantic gap. HFSLE gives a new perspective on the hierarchical feature selection based on label distribution rather than the original logical label. To demonstrate the superiority of the method, we compared it with six well-established feature selection methods on eight hierarchical data sets, and analyzed the convergence and robustness of the method. Extensive experiments demonstrate that the proposed method can perform favorably against advanced hierarchical feature selection approaches.

Extracting the most discriminating functional connections in mild traumatic brain injury based on machine learning

BackgroundMild traumatic brain injury (mTBI) is characterized as brain microstructural damage, which may cause a wide range of brain functional disturbances and emotional problems. Brain network analysis based on machine learning is an important means of neuroimaging research. Obtaining the most discriminating functional connection is of great significance to analyze the pathological mechanism of mTBI. MethodsTo better obtain the most discriminating features of functional connection networks, this study proposes a hierarchical feature selection pipeline (HFSP) composed of Variance Filtering (VF), Lasso, and Principal Component Analysis (PCA). Ablation experiments indicate that each module plays a positive role in classification, validating the robustness and reliability of the HFSP. Furthermore, the HFSP is compared with recursive feature elimination (RFE), elastic net (EN), and locally linear embedding (LLE), verifying its superiority. In addition, this study also utilizes random forest (RF), SVM, Bayesian, linear discriminant analysis (LDA), and logistic regression (LR) as classifiers to evaluate the generalizability of HFSP. ResultsThe results show that the indexes obtained from RF are the highest, with accuracy = 89.74%, precision = 91.26%, recall = 89.74%, and F1 score = 89.42%. The HFSP selects 25 pairs of the most discriminating functional connections, mainly distributed in the frontal lobe, occipital lobe, and cerebellum. Nine brain regions show the largest node degree.Limitations: The number of samples is small. This study only includes acute mTBI. ConclusionsThe HFSP is a useful tool for extracting discriminating functional connections and may contribute to the diagnostic processes.

A hierarchical clustering approach for colorectal cancer molecular subtypes identification from gene expression data

BackgroundColorectal cancer (CRC) is the second leading cause of cancer fatalities and the third most common human disease. Identifying molecular subgroups of CRC and treating patients accordingly could result in better therapeutic success compared with treating all CRC patients similarly. Studies have highlighted the significance of CRC as a major cause of mortality worldwide and the potential benefits of identifying molecular subtypes to tailor treatment strategies and improve patient outcomes. MethodsThis study proposed an unsupervised learning approach using hierarchical clustering and feature selection to identify molecular subtypes and compares its performance with that of conventional methods. The proposed model contained gene expression data from CRC patients obtained from Kaggle and used dimension reduction techniques followed by Z-score-based outlier removal. Agglomerative hierarchy clustering was used to identify molecular subtypes, with a P-value-based approach for feature selection. The performance of the model was evaluated using various classifiers including multilayer perceptron (MLP). ResultsThe proposed methodology outperformed conventional methods, with the MLP classifier achieving the highest accuracy of 89% after feature selection. The model successfully identified molecular subtypes of CRC and differentiated between different subtypes based on their gene expression profiles. ConclusionThis method could aid in developing tailored therapeutic strategies for CRC patients, although there is a need for further validation and evaluation of its clinical significance.

A hierarchical feature selection strategy for deepfake video detection

A hierarchical feature selection strategy for deepfake video detection

Cluster-based segmentation for tobacco plant detection and classification

Tobacco is one of the major economical crops in the agriculture sector. It is essential to detect tobacco plants using unmanned aerial vehicle (UAV) images for improved crop yield and plays an important role in the early treatment of tobacco plants. The proposed research work is carried out in three phases: In the first phase, we collect images from UAV’s and apply the French Commision Internationale de l'eclairage (CIE) L*a*b colour space model as pre-processing operations and segmentation. And then two prominent motion descriptors namely histogram of flow (HOF) and motion boundary histogram (MBH) are combined with the optimal histogram of oriented gradients (HOG) descriptor for exploring optimal motion trajectory and spatial measurements. And finally, the spatial variations with respect to the scale and illumination changes are incorporated using the optimal HOG descriptor. Here both dense motion patterns and HOG are refined using hierarchical feature selection using principal component analysis (PCA). The proposed model is trained and evaluated on different tobacco UAV image datasets and done a comparative analysis of different machine learning (ML) algorithms. The proposed model achieves good performance with 95% accuracy and 92% of sensitivity.

Feature selection via maximizing inter-class independence and minimizing intra-class redundancy for hierarchical classification

Hierarchical feature selection has proven to be significant in reducing classification difficulty. Many existing hierarchical feature selection methods use the hierarchy in the class space as structural information to improve performance. However, these methods focus on the inter-class structural relations and ignore the intra-class feature correlations, which is challenging as the numbers of classes and features in current datasets are both growing rapidly. In this paper, we propose a hierarchical feature selection method that maximizes inter-class independence and minimizes intra-class redundancy using structure and feature relations. First, we investigate the class hierarchy dependency in the tree structure as the structural relation regularization, which maximizes the independence between unrelated classes in the class structure. Second, we transform the feature correlations into a mathematical representation as feature relation regularization, which minimizes the redundancy in each class on the premise of sparsity. Finally, we unify the two regularization terms into a hierarchical feature selection method to trade off the structural and feature relations. Our method exhibits excellent efficiency and effectiveness compared with other feature selection methods.

A coastal wetlands mapping approach of Yellow River Delta with a hierarchical classification and optimal feature selection framework

Wetlands play an important role in ecological health and sustainable development, their spatial distribution and explicit thematic information are crucial for developing management and conservation measures. The Yellow River Delta is an important coastal wetland reserve in China, its wetland types are complex and diverse, natural and artificial wetlands are easily confused, making refined classification more difficult. To address this challenge, we proposed a new wetland mapping approach by combing hierarchical classification framework (HCF) and optimal feature selection. First, inheritance-based multiscale segmentation was carried out to obtain object-oriented images, and decision tree classification was used for preliminarily identify wetland and non-wetland. Second, recursive feature elimination and cross-validation (RFECV) was used to select optimal features, which was then utilized for wetland refinement extraction by using random forest (RF) algorithm. The experiments were performed based on Sentinel-1, Sentinel-2 and NASADEM datasets. The results show that effective wetland classification features can be selected by using RFECV. The feature scores are as follows, red edge index > spectral features > vegetation/water body index > backscatter coefficient > topographic features > texture features > location feature > urban index > geometric feature. The overall accuracy and Kappa coefficient of the method in this paper are 92.36 % and 0.915, which are 14.62 % and 6.68 % higher than using only HCF or only RFECV. Compared with the GlobeLand30 and CAS_Wetlands datasets, the refinement of wetland mapping in this paper is higher. This study provides a new idea in methodological selection for wetland information extraction, and the resulting coastal wetland map can be used for sustainable management, ecological assessment and conservation of the Yellow River Delta.

Hierarchical Feature Fusion and Selection for Hyperspectral Image Classification

Most existing classification methods design complicated and large deep neural network (DNN) model to deal with the ubiquitous spectral variability and nonlinearity of hyperspectral images (HSIs). However, their application is blocked by limited training samples and considerable computational costs in real scenes. To solve these problems, we propose a simple spectral hierarchical feature fusion and selection network (HFFSNet). Specifically, we apply 1-D grouped convolution for dimensionality reduction and multilevel feature extraction, then the multilevel features are fused to assist the adaptive feature selection of different layer features via the soft attention mechanism, and finally the selected features are fused to further enhance the feature representation. Extensive experimental results on three hyperspectral datasets demonstrate the effectiveness of the proposed network.

Performance Analysis of Coal Gangue Recognition Based on Hierarchical Filtering and Coupled Wrapper Feature Selection Method

Performance Analysis of Coal Gangue Recognition Based on Hierarchical Filtering and Coupled Wrapper Feature Selection Method

Hierarchical Voting-Based Feature Selection and Ensemble Learning Model Scheme for Glioma Grading with Clinical and Molecular Characteristics.

Determining the aggressiveness of gliomas, termed grading, is a critical step toward treatment optimization to increase the survival rate and decrease treatment toxicity for patients. Streamlined grading using molecular information has the potential to facilitate decision making in the clinic and aid in treatment planning. In recent years, molecular markers have increasingly gained importance in the classification of tumors. In this study, we propose a novel hierarchical voting-based methodology for improving the performance results of the feature selection stage and machine learning models for glioma grading with clinical and molecular predictors. To identify the best scheme for the given soft-voting-based ensemble learning model selections, we utilized publicly available TCGA and CGGA datasets and employed four dimensionality reduction methods to carry out a voting-based ensemble feature selection and five supervised models, with a total of sixteen combination sets. We also compared our proposed feature selection method with the LASSO feature selection method in isolation. The computational results indicate that the proposed method achieves 87.606% and 79.668% accuracy rates on TCGA and CGGA datasets, respectively, outperforming the LASSO feature selection method.