Heuristic Feature Research Articles

HFS: an intelligent heuristic feature selection scheme to correct uncertainty

HFS: an intelligent heuristic feature selection scheme to correct uncertainty

FBDPN: CNN-Transformer hybrid feature boosting and differential pyramid network for underwater object detection

Despite advancements in underwater object detection (UOD) from optical underwater images in recent years, the task still poses significant challenges due to the chaotic underwater environment, as well as the substantial variations in scale and contour of objects. Existing deep learning-based schemes generally overlook the enhancement and refinement between multi-scale features of densely distributed underwater objects, leading to inaccurate localization and classification predictions with excessive information redundancy. To tackle the above issues, this article presents a novel feature boosting and differential pyramid network (FBDPN) for precise and efficient UOD. The salient properties of our paper are: (1) a heuristic feature pyramid network (FPN)-inspired architecture is constructed, which employs a convolutional neural network (CNN)-Transformer hybrid strategy to simultaneously facilitate the learning of multi-scale features and the capture of long-distance dependencies among pixels. (2) A neighborhood-scale feature boosting module (NSFBM) is developed to enhance contextual information between features of neighborhood scales. (3) A cross-scale feature differential module (CSFDM) is designed further to achieve effective information redundancy between features of different scales. Extensive experiments are conducted to reveal that our proposed FBDPN can outperform other state-of-the-art methods in both UOD performance and computational complexity. In addition, sufficient ablation studies are also performed to demonstrate the effectiveness of each component in our FBDPN. The source code is available at https://github.com/jixun-dmu/FBDPN.

Matrix-based incremental feature selection method using weight-partitioned multigranulation rough set

Incremental feature selection methods have gained increasing research attention as they improve the efficiency of feature selection for dynamic datasets. Multigranulation rough set, as an extension of rough set theory, allows for a comprehensive and rational analysis of problems from multiple hierarchical and granular perspectives. However, existing research on granularity partitioning relies on the decision maker's subjective experience, which lacks convincing power. In this paper, we propose a generalized multigranulation neighborhood rough set based on weight partition model, using a matrix form. We discuss several properties and define a new entropy measure to evaluate feature importance. A heuristic feature selection algorithm is developed based on this entropy to search for the optimal subset. Furthermore, we discuss dynamic updating mechanism and design two incremental feature selection algorithms. Finally, we conduct experiments on 12 public datasets to evaluate the performance of the proposed algorithms and validate their effectiveness and efficiency in feature selection for both static and dynamic datasets.

Data-driven definition and modelling of plant growth

Many horticultural tasks require a definition of and means to measure ‘plant state’, a quantity that is used to represent and compare plants. The plant state's evolution through time (plant growth) can then be tracked. This is often performed informally by humans based on heuristic features for particular plant types. It can be used for example to recommend interventions to catch up on expected growth, or to measure the effects of experimental interventions on growth. This work provides a purely data-driven definition, that is easy to train, easy to non-destructively acquire training data, does not require expert annotations, and is easy to compute for any new plant type and growing conditions. The presented method is applied to a dataset of lettuce plants where it exceeds the performance of a hard baseline. This work also demonstrates that the presented method retains the intuitive properties expected for a plant growth model. Open source code implementation and data is provided.

Feature selection considering feature relevance, redundancy and interactivity for neighbourhood decision systems

Feature selection is an effective method to simplify data analysis and obtain key features, which improves the accuracy and generalization ability of classifiers. Neighbourhood rough set is a typical granular computing model that enables data analysis at different granularity by setting different neighbourhood radii. In recent years, feature selection based on neighbourhood rough sets has received widespread attention from researchers. However, these models still have the following shortcomings. On the one hand, they rarely simultaneously consider the relevance, the redundancy and interactivity among features, making it difficult to accurately depict classification information in the data. On the other hand, there is still a lack of effective uncertainty measures to evaluate the classification ability of feature subsets. For these reasons, this article proposes a feature selection model that considers the relevance, redundancy, and interactivity of features. Firstly, the relevance, the redundancy and interactivity between features, are defined based on the neighbourhood, respectively. Secondly, a new feature evaluation function (MMM function) is proposed based on the principles of maximum relevance, minimum redundancy, and maximum interactivity. Finally, from the perspective of maintaining classification ability, a heuristic feature selection algorithm (FS_CRRI) was designed. To evaluate the performance of the proposed algorithm, it was compared with some representative feature selection algorithms on twelve common datasets. The experimental results on three different classifiers indicate that the FS_CRRI algorithm can effectively reduce data and achieve higher classification performance.

DHFS-ECM: Design of a Dual Heuristic Feature Selection-based Ensemble Classification Model for the Identification of Bamboo Species from Genomic Sequences.

Analyzing genomic sequences plays a crucial role in understanding biological diversity and classifying Bamboo species. Existing methods for genomic sequence analysis suffer from limitations such as complexity, low accuracy, and the need for constant reconfiguration in response to evolving genomic datasets. This study addresses these limitations by introducing a novel Dual Heuristic Feature Selection-based Ensemble Classification Model (DHFS-ECM) for the precise identification of Bamboo species from genomic sequences. The proposed DHFS-ECM method employs a Genetic Algorithm to perform dual heuristic feature selection. This process maximizes inter-class variance, leading to the selection of informative N-gram feature sets. Subsequently, intra-class variance levels are used to create optimal training and validation sets, ensuring comprehensive coverage of class-specific features. The selected features are then processed through an ensemble classification layer, combining multiple stratification models for species-specific categorization. Comparative analysis with state-of-the-art methods demonstrate that DHFS-ECM achieves remarkable improvements in accuracy (9.5%), precision (5.9%), recall (8.5%), and AUC performance (4.5%). Importantly, the model maintains its performance even with an increased number of species classes due to the continuous learning facilitated by the Dual Heuristic Genetic Algorithm Model. DHFS-ECM offers several key advantages, including efficient feature extraction, reduced model complexity, enhanced interpretability, and increased robustness and accuracy through the ensemble classification layer. These attributes make DHFS-ECM a promising tool for real-time clinical applications and a valuable contribution to the field of genomic sequence analysis.

Frequency-Separated Attention Network for Image Super-Resolution

The use of deep convolutional neural networks has significantly improved the performance of super-resolution. Employing deeper networks to enhance the non-linear mapping capability from low-resolution (LR) to high-resolution (HR) images has inadvertently weakened the information flow and disrupted long-term memory. Moreover, overly deep networks are challenging to train, thus failing to exhibit the expressive capability commensurate with their depth. High-frequency and low-frequency features in images play different roles in image super-resolution. Networks based on CNNs, which should focus more on high-frequency features, treat these two types of features equally. This results in redundant computations when processing low-frequency features and causes complex and detailed parts of the reconstructed images to appear as smooth as the background. To maintain long-term memory and focus more on the restoration of image details in networks with strong representational capabilities, we propose the Frequency-Separated Attention Network (FSANet), where dense connections ensure the full utilization of multi-level features. In the Feature Extraction Module (FEM), the use of the Res ASPP Module expands the network’s receptive field without increasing its depth. To differentiate between high-frequency and low-frequency features within the network, we introduce the Feature-Separated Attention Block (FSAB). Furthermore, to enhance the quality of the restored images using heuristic features, we incorporate attention mechanisms into the Low-Frequency Attention Block (LFAB) and the High-Frequency Attention Block (HFAB) for processing low-frequency and high-frequency features, respectively. The proposed network outperforms the current state-of-the-art methods in tests on benchmark datasets.

Identification of important gene signatures in schizophrenia through feature fusion and genetic algorithm.

Schizophrenia is a debilitating psychiatric disorder that can significantly affect a patient's quality of life and lead to permanent brain damage. Although medical research has identified certain genetic risk factors, the specific pathogenesis of the disorder remains unclear. Despite the prevalence of research employing magnetic resonance imaging, few studies have focused on the gene level and gene expression profile involving a large number of screened genes. However, the high dimensionality of genetic data presents a great challenge to accurately modeling the data. To tackle the current challenges, this study presents a novel feature selection strategy that utilizes heuristic feature fusion and a multi-objective optimization genetic algorithm. The goal is to improve classification performance and identify the key gene subset for schizophrenia diagnostics. Traditional gene screening techniques are inadequate for accurately determining the precise number of key genes associated with schizophrenia. Our innovative approach integrates a filter-based feature selection method to reduce data dimensionality and a multi-objective optimization genetic algorithm for improved classification tasks. By combining the filtering and wrapper methods, our strategy leverages their respective strengths in a deliberate manner, leading to superior classification accuracy and a more efficient selection of relevant genes. This approach has demonstrated significant improvements in classification results across 11 out of 14 relevant datasets. The performance on the remaining three datasets is comparable to the existing methods. Furthermore, visual and enrichment analyses have confirmed the practicality of our proposed method as a promising tool for the early detection of schizophrenia.

Visual representation learning using graph-based higher-order heuristic distillation for cell detection in blood smear images

Background and objectiveIn many real-world scenarios, including the blood smear domain, it is difficult for detection networks to achieve good performance because image annotation is usually time consuming and expensive. To address this issue, similarity-based distillation (SD) methods, considered the soft version of contrastive learning, are applied to learn a better visual representation without requiring any supervision of the downstream task. Motivated by our theoretical analysis, we treat standard SD methods as the maximization of common 1-hop neighboring key points between two queries in an attributed graph, where nodes represent query and key data points. However, such first-order graph heuristic methods that calculate the likelihood of an unseen link between target nodes by using up to 1-hop neighborhoods are normally limited by insufficient representation power and even lack of generalization ability. MethodsTherefore, in this paper, we propose a novel higher-order heuristic distillation (H2D) method that distills knowledge about more general and powerful higher-order heuristic features based on a more than 1-hop relationship in the attributed graph. To do this, we utilize a graph neural network model to learn the higher-order heuristic features on the attributed graph constructed by query and key data representations and transfer the knowledge from the teacher to the student encoder. ResultsOur method outperforms the previous state-of-the-art SD methods in the cell detection task on the blood smear dataset as well on open databases (Pascal VOC and MS COCO). Conclusions: Our proposed model allow teacher encoder to transfer the knowledge about more general and powerful higher-order heuristic embeddings to the student and enables better learning for visual representation on cell detection task using blood smear images.

A machine learning model to predict heart failure readmission: toward optimal feature set.

Hospital readmissions for heart failure patients remain high despite efforts to reduce them. Predictive modeling using big data provides opportunities to identify high-risk patients and inform care management. However, large datasets can constrain performance. This study aimed to develop a machine learning based prediction model leveraging a nationwide hospitalization database to predict 30-day heart failure readmissions. Another objective of this study is to find the optimal feature set that leads to the highest AUC value in the prediction model. Heart failure patient data was extracted from the 2020 Nationwide Readmissions Database. A heuristic feature selection process incrementally incorporated predictors into logistic regression and random forest models, which yields a maximum increase in the AUC metric. Discrimination was evaluated through accuracy, sensitivity, specificity and AUC. A total of 566,019 discharges with heart failure diagnosis were recognized. Readmission rate was 8.9% for same-cause and 20.6% for all-cause diagnoses. Random forest outperformed logistic regression, achieving AUCs of 0.607 and 0.576 for same-cause and all-cause readmissions respectively. Heuristic feature selection resulted in the identification of optimal feature sets including 20 and 22 variables from a pool of 30 and 31 features for the same-cause and all-cause datasets. Key predictors included age, payment method, chronic kidney disease, disposition status, number of ICD-10-CM diagnoses, and post-care encounters. The proposed model attained discrimination comparable to prior analyses that used smaller datasets. However, reducing the sample enhanced performance, indicating big data complexity. Improved techniques like heuristic feature selection enabled effective leveraging of the nationwide data. This study provides meaningful insights into predictive modeling methodologies and influential features for forecasting heart failure readmissions.

The effectiveness of simple heuristic features in sensor orientation and placement problems in human activity recognition using a single smartphone accelerometer

BackgroundHuman activity Recognition (HAR) using smartphone sensors suffers from two major problems: sensor orientation and placement. Sensor orientation and sensor placement problems refer to the variation in sensor signal for a particular activity due to sensors’ altering orientation and placement. Extracting orientation and position invariant features from raw sensor signals is a simple solution for tackling these problems. Using few heuristic features rather than numerous time-domain and frequency-domain features offers more simplicity in this approach. The heuristic features are features which have very minimal effects of sensor orientation and placement. In this study, we evaluated the effectiveness of four simple heuristic features in solving the sensor orientation and placement problems using a 1D-CNN–LSTM model for a data set consisting of over 12 million samples.MethodsWe accumulated data from 42 participants for six common daily activities: Lying, Sitting, Walking, and Running at 3-Metabolic Equivalent of Tasks (METs), 5-METs and 7-METs from a single accelerometer sensor of a smartphone. We conducted our study for three smartphone positions: Pocket, Backpack and Hand. We extracted simple heuristic features from the accelerometer data and used them to train and test a 1D-CNN–LSTM model to evaluate their effectiveness in solving sensor orientation and placement problems.ResultsWe performed intra-position and inter-position evaluations. In intra-position evaluation, we trained and tested the model using data from the same smartphone position, whereas, in inter-position evaluation, the training and test data was from different smartphone positions. For intra-position evaluation, we acquired 70–73% accuracy; for inter-position cases, the accuracies ranged between 59 and 69%. Moreover, we performed participant-specific and activity-specific analyses.ConclusionsWe found that the simple heuristic features are considerably effective in solving orientation problems. With further development, such as fusing the heuristic features with other methods that eliminate placement issues, we can also achieve a better result than the outcome we achieved using the heuristic features for the sensor placement problem. In addition, we found the heuristic features to be more effective in recognizing high-intensity activities.

Noise-Tolerant Discrimination Indexes for Fuzzy ɣ Covering and Feature Subset Selection.

Fuzzy β covering (FBC) has attracted considerable attention in recent years. Nevertheless, as the basic information granularity of FBC, fuzzy β neighborhood does not satisfy reflexivity, which may lead to instability in classification learning and decision-making. Although a few studies have involved reflexive fuzzy β neighborhoods, they only focus on a single fuzzy covering and cannot effectively deal with the information representation and information fusion of multiple fuzzy coverings. Moreover, there is a lack of investigation on noise-tolerant uncertainty measures for FBC, as well as their application in feature selection. Motivated by these issues, we investigate a noise-tolerant variable precision discrimination index (VPDI) by means of a new reflexive fuzzy covering neighborhood. To this end, fuzzy ɣ neighborhood with reflexivity is introduced to characterize the information fusion of a fuzzy covering family. An uncertainty measure called fuzzy ɣ neighborhood discrimination index is then presented to reflect the discriminatory power of fuzzy covering families. Some variants of the uncertainty measure, such as variable precision joint discrimination index, variable precision conditional discrimination index, and variable precision mutual discrimination index, are then put forth by means of fuzzy decision. These VPDIs can be used as an evaluation metric for a family of fuzzy coverings. Finally, the knowledge reduction of fuzzy covering decision systems is addressed from the point of keeping the discriminatory power, and a heuristic feature selection algorithm is designed by means of the variable precision conditional discrimination index. The experiments on 16 public datasets exhibit that the proposed algorithm can effectively reduce redundant features and achieve competitive results compared with six state-of-the-art feature selection algorithms. Moreover, it demonstrates strong robustness to the interference of random noise.

New Subspace Method for Unconstrained Derivative-Free Optimization

This article defines an efficient subspace method, called SSDFO , for unconstrained derivative-free optimization problems where the gradients of the objective function are Lipschitz continuous but only exact function values are available. SSDFO employs line searches along directions constructed on the basis of quadratic models. These approximate the objective function in a subspace spanned by some previous search directions. A worst-case complexity bound on the number of iterations and function evaluations is derived for a basic algorithm using this technique. Numerical results for a practical variant with additional heuristic features show that, on the unconstrained CUTEst test problems, SSDFO has superior performance compared to the best solvers from the literature.

Black Transmission: Toward a Hieroglyph-Analysis

This meditation is an experimentation with Hortense Spiller’s concept of psychoanalytics. It considers the hieroglyph as a repetitive and enacted (but unrepresentable) force of historical trauma marking black bodies and passed between them generationally (much like the Lacanian object a). Reading the Lacanian pass and erotogenic zones as heuristic features of the hieroglyph, this meditation presents black nihilism as an analytic setting enabling the hieroglyph to speak its ineffable repetition, rather than just a repertoire of hedonistic pleasures and destructive behaviors as described in black optimism.

Understanding the relationship between reviews, search and sales: a study of the Indian car market

PurposeWhile there has been extensive research on understanding the effects of online reviews on product sales, there is not enough investigation of the inter-relationships between online reviews, online search and product sales. The study attempts to address this gap in the context of the Indian car market.Design/methodology/approachThe research uses text mining and considers six important review features volume, valence, length, deviation of valence, sentiment and readability within the heuristic and systematic model of information processing. Panel data regression is used along with mediation analysis to study the inter-relationships between features of reviews, online search and sales.FindingsThe study finds that numerical heuristic features significantly affect sales and online search, numerical systematic feature affects sales and the textual heuristic and systematic features do not affect sales or online search in the Indian car market. Further, online search mediates the association between features of reviews and sales of cars.Research limitations/implicationsAlthough only car sales data from India is considered in this research, similar relationships between review features, online search and sales could exist for the car market of other countries as well.Originality/valueThis research uncovers the unique role of online search as a mediator between review features and sales, whereas prior literature has considered review features and online search as independent variables that affect sales.

TC-SEPM: Characterizing soft error resilience of CNNs on Tensor Cores from program and microarchitecture perspectives

As an architectural CNN accelerator integrated into NVIDIA’s GPUs, existing research mainly focuses on improving the performance of Tensor Cores. However, the highly integrated Tensor Cores are vulnerable to transient faults (i.e., soft errors), causing catastrophic consequences in safety-critical applications like automatic driving. Thus, it is imperative to estimate the reliability of CNNs on Tensor Cores. However, obtaining a statistically significant resilience profile of CNNs on Tensor Cores with the existing fault injection (FI)-based reliability estimation methods is expensive. To this end, we build TC-SEPM to predict the error resilience of CNNs on Tensor Cores instead of FI methods. To ensure the accuracy of TC-SEPM, we first investigate resilience-related features from program and microarchitecture perspectives. Then, leveraging these heuristic features, we train machine learning models to learn the hidden relationship between error resilience and the investigated features, enabling us to predict the impact of soft errors in Tensor Cores on CNN output. Experimental results show that TC-SEPM achieves high accuracy for individual soft error resiliency prediction and overall program resilience estimation while its overhead is only 1/27 of FI methods. Additionally, TC-SEPM can provide valuable insights for programmers or architects to design more robust CNN models on Tensor Cores.

Neighborhood multigranulation rough sets for cost-sensitive feature selection on hybrid data

Feature selection is a vital preprocessing step in real applications of data mining and machine learning. With the prevalence of high-dimensional hybrid data sets in real-world scenarios, along with the presence of test costs and misclassification costs, the need for effective feature selection methods has become more prominent. However, existing feature selection approaches mainly focus on cost-sensitive data from a single granularity perspective, and they are primarily applicable to single-typed data sets. To address these limitations, this paper presents a novel feature selection approach specifically designed for hybrid data, considering variable test costs and misclassification costs. The proposed method is based on neighborhood multigranulation rough sets (NMRS), which provides more comprehensive and multi-angle data analysis for cost-sensitive hybrid data. First, a novel multigranulation model is developed to effectively process cost-sensitive hybrid data. Building upon this model, a cost-based multi-criteria measure is proposed to evaluate the significance of features. This measure takes into account the comprehensive information of candidate features, including their power in algebraic view, information view, and associated costs. Furthermore, a heuristic feature selection algorithm based on NMRS is proposed to handle hybrid data with the test costs and misclassification costs. This algorithm leverages the benefits of the proposed multigranulation model and cost-based measure to identify the most discriminative features efficiently. Finally, the experimental results on twelve different datasets show that the proposed heuristic algorithm outperforms other compared algorithms, especially in total cost and classification accuracy.

Adaptive fuzzy multi-neighborhood feature selection with hybrid sampling and its application for class-imbalanced data

For imbalanced data, classification efficiency degrades significantly due to the missing information for the positive class, and existing sampling schemes do not consider the distributions of samples. Additionally, the global parameters of fuzzy neighborhoods are set manually. These defects affect the effectiveness of classifier. To address these problems, we offer an adaptive fuzzy multi-neighborhood feature selection methodology with intercluster distance-based hybrid sampling for class-imbalanced data. First, the number of clusters can be defined in terms of the number of samples in the negative or positive class. The initial centers of the clusters are determined according to the number of clusters, and the dissimilarity and similarity measures are calculated by using the intercluster distances between samples. Then, the cluster center, fuzzy membership matrix, and intercluster distance are studied, and then the optimization objective function is designed. The hybrid sampling scheme can be used to combine the generated positive class samples and negative class samples and obtain a class-balanced system. Second, according to the sample distribution, the standard deviation and a set of adaptive fuzzy multi-neighborhood radii are designed. A fuzzy multi-neighborhood similarity relation is defined by introducing a Gaussian kernel model to obtain a fuzzy multi-neighborhood granule, and an improved fuzzy multi-neighborhood rough set model is provided. Uncertain measures of fuzzy neighborhood systems are evaluated by the positive region and dependency. Third, by integrating fuzzy dependence with fuzzy complementary condition entropy, fuzzy multi-neighborhood complementary mutual information is provided on two viewpoints of algebra and information. Finally, a heuristic feature subset selection methodology for imbalanced classification with hybrid sampling using fuzzy c-means clustering is studied to obtain this excellent set of features. Experiments on 26 imbalanced datasets show the effectiveness of our designed algorithm.

Automatic author name disambiguation by differentiable feature selection

Author name disambiguation (AND) is the task of resolving the ambiguity problem in bibliographic databases, where distinct real-world authors may share the same name or same author may have distinct names. The aim of AND is to split the name-ambiguous entities (articles) into the corresponding authors. Existing AND algorithms mainly focus on designing different similarity metrics between two ambiguous articles. However, most previous methods empirically select and process the features of entities, then use features to predict the similarity by data-driven models. In this article, we are motivated by natural questions: Which features are most useful for splitting name-ambiguous entities? Can they be automatically determined by an optimisation approach rather than heuristic feature engineering? Therefore, we proposed a novel end-to-end differentiable feature selection algorithm, automatically searching the optimal features for AND task (AAND). AAND optimises the discrete feature selection by differentiable Gumbel-Softmax, leading to the joint learning of feature selection policy and similarity prediction model. The experiments are conducted on a benchmark data set, S2AND, which harmonises eight different AND data sets. The results show that the performance of our proposal is superior to the advanced AND methods and feature selection algorithms. Meanwhile, deep insights into AND features are also given.

Identification of stromal cell proportion-related genes in the breast cancer tumor microenvironment using CorDelSFS feature selection: implications for tumor progression and prognosis.

Background: The tumor microenvironment (TME) of breast cancer (BRCA) is a complex and dynamic micro-ecosystem that influences BRCA occurrence, progression, and prognosis through its cellular and molecular components. However, as the tumor progresses, the dynamic changes of stromal and immune cells in TME become unclear. Objective: The aim of this study was to identify differentially co-expressed genes (DCGs) associated with the proportion of stromal cells in TME of BRCA, to explore the patterns of cell proportion changes, and ultimately, their impact on prognosis. Methods: A new heuristic feature selection strategy (CorDelSFS) was combined with differential co-expression analysis to identify TME-key DCGs. The expression pattern and co-expression network of TME-key DCGs were analyzed across different TMEs. A prognostic model was constructed using six TME-key DCGs, and the correlation between the risk score and the proportion of stromal cells and immune cells in TME was evaluated. Results: TME-key DCGs mimicked the dynamic trend of BRCA TME and formed cell type-specific subnetworks. The IG gene-related subnetwork, plasmablast-specific expression, played a vital role in the BRCA TME through its adaptive immune function and tumor progression inhibition. The prognostic model showed that the risk score was significantly correlated with the proportion of stromal cells and immune cells in TME, and low-risk patients had stronger adaptive immune function. IGKV1D-39 was identified as a novel BRCA prognostic marker specifically expressed in plasmablasts and involved in adaptive immune responses. Conclusions: This study explores the role of proportionate-related genes in the tumor microenvironment using a machine learning approach and provides new insights for discovering the key biological processes in tumor progression and clinical prognosis.