Feature Selection Learning Research Articles

A novel granular ball computing-based fuzzy rough set for feature selection in label distribution learning

Label distribution learning is a widely studied supervised learning diagram that can handle the problem of label ambiguity. The increasing size of datasets is accompanied by the disaster of dimensionality, which implies that the arrival of redundant and noisy features undermines the effect of label distribution learning. As a crucial data-preprocessing technique, feature selection is capable of choosing discriminative features. However, due to the complex issue of label ambiguity, traditional feature selection approaches for datasets with logical labels cannot be applied to label distribution data. In this paper, a novel granular ball computing-based fuzzy rough set (GBFRS) is proposed for label distribution feature selection. Specifically, the proposed method is first introduced at the finest granularity, i.e., calculating similarity relations between single data points. Considering that the label ambiguity issue is exacerbated by the label imbalance phenomenon, the relative similarity in label distribution space among samples is computed for better generalization of the model. Then, a robust approximation strategy is devised for the target sample by using its true different and partially different class samples. Finally, with the concept of granular balls, the method explores the similarity relations between balls and samples, and the granular ball computing-based fuzzy rough set method is developed , which is endowed with the ability to simulate the characteristics of large-scale priorities in human thinking and considers local consistency. Extensive experiments conducted on twenty-two datasets show that GBFRS can effectively select more significant features than seven state-of-the-art feature selection algorithms.

Snake-Efficient Feature Selection-Based Framework for Precise Early Detection of Chronic Kidney Disease.

Chronic kidney disease (CKD) refers to impairment of the kidneys that may worsen over time. Early detection of CKD is crucial for saving millions of lives. As a result, several studies are currently focused on developing computer-aided systems to detect CKD in its early stages. Manual screening is time-consuming and subject to personal judgment. Therefore, methods based on machine learning (ML) and automatic feature selection are used to support graders. The goal of feature selection is to identify the most relevant and informative subset of features in a given dataset. This approach helps mitigate the curse of dimensionality, reduce dimensionality, and enhance model performance. The use of natural-inspired optimization algorithms has been widely adopted to develop appropriate representations of complex problems by conducting a blackbox optimization process without explicitly formulating mathematical formulations. Recently, snake optimization algorithms have been developed to identify optimal or near-optimal solutions to difficult problems by mimicking the behavior of snakes during hunting. The objective of this paper is to develop a novel snake-optimized framework named CKD-SO for CKD data analysis. To select and classify the most suitable medical data, five machine learning algorithms are deployed, along with the snake optimization (SO) algorithm, to create an extremely accurate prediction of kidney and liver disease. The end result is a model that can detect CKD with 99.7% accuracy. These results contribute to our understanding of the medical data preparation pipeline. Furthermore, implementing this method will enable health systems to achieve effective CKD prevention by providing early interventions that reduce the high burden of CKD-related diseases and mortality.

Dynamic mutual information-based feature selection for multi-label learning

In classification problems, feature selection is used to identify important input features to reduce the dimensionality of the input space while improving or maintaining classification performance. Traditional feature selection algorithms are designed to handle single-label learning, but classification problems have recently emerged in multi-label domain. In this study, we propose a novel feature selection algorithm for classifying multi-label data. This proposed method is based on dynamic mutual information, which can handle redundancy among features controlling the input space. We compare the proposed method with some existing problem transformation and algorithm adaptation methods applied to real multi-label datasets using the metrics of multi-label accuracy and hamming loss. The results show that the proposed method demonstrates more stable and better performance for nearly all multi-label datasets.

Online feature selection for hierarchical classification learning based on improved ReliefF

AbstractIn hierarchical classification learning, the feature space of data has high dimensionality and is unknown with emergent features. To solve the above problems, we propose an online hierarchical feature selection algorithm based on adaptive ReliefF. Firstly, ReliefF is adaptively improved via using the density information of instances around the target sample, making it unnecessary to prespecify parameters. Secondly, the hierarchical relationship between classes is used, and a new method for calculating the feature weight of hierarchical data is defined. Then, an online correlation analysis method based on feature interaction is designed. Finally, the adaptive ReliefF algorithm is improved based on feature redundancy, and the feature weight is scaled by the correlation between features in order to achieve the dynamic updating of feature redundancy. A large number of experiments verify the effectiveness of the proposed algorithm.

CNN-CNN: Dual Convolutional Neural Network Approach for Feature Selection and Attack Detection on Internet of Things Networks.

The Internet of Things (IoT) has brought significant advancements that have connected our world more closely than ever before. However, the growing number of connected devices has also increased the vulnerability of IoT networks to several types of attacks. In this paper, we present an approach for detecting attacks on IoT networks using a combination of two convolutional neural networks (CNN-CNN). The first CNN model is leveraged to select the significant features that contribute to IoT attack detection from the raw data on network traffic. The second CNN utilizes the features identified by the first CNN to build a robust detection model that accurately detects IoT attacks. The proposed approach is evaluated using the BoT IoT 2020 dataset. The results reveal that the proposed approach achieves 98.04% detection accuracy, 98.09% precision, 99.85% recall, 98.96% recall, and a 1.93% false positive rate (FPR). Furthermore, the proposed approach is compared with other deep learning algorithms and feature selection methods; the results show that it outperforms these algorithms.

Energy Performance Analysis of Photovoltaic Integrated with Microgrid Data Analysis Using Deep Learning Feature Selection and Classification Techniques

Microgrids are an essential element of smart grids, which contain distributed renewable energy sources (RESs), energy storage devices, and load control strategies. Models built based on machine learning (ML) and deep learning (DL) offer hope for anticipating consumer demands and energy production from RESs. This study suggests an innovative approach for energy analysis based on the feature extraction and classification of microgrid photovoltaic cell data using deep learning algorithms. The energy optimization of a microgrid was carried out using a photovoltaic energy system with distributed power generation. The data analysis has been carried out for feature analysis and classification using a Gaussian radial Boltzmann with Markov encoder model. Based on microgrid energy optimization and data analysis, an experimental analysis of power analysis, energy efficiency, quality of service (QoS), accuracy, precision, and recall has been conducted. The proposed technique attained power analysis of 88%, energy efficiency of 95%, QoS of 77%, accuracy of 93%, precision of 85%, and recall of 77%.

Comparison of the Strengths and Weaknesses of Machine Learning Algorithms and Feature Selection on KEGG Database Microbial Gene Pathway Annotation and Its Effects on Reconstructed Network Topology.

The development of tools for the annotation of genes from newly sequenced species has not evolved much from homologous alignment to prior annotated species. While the quality of gene annotations continues to decline as we sequence and assemble more evolutionary distant gut microbiome species, machine learning presents a high quality alternative to traditional techniques. In this study, we investigate the relative performance of common classical and nonclassical machine learning algorithms in the problem of gene annotation using human microbiome-associated species genes from the KEGG database. The majority of the ensemble, clustering, and deep learning algorithms that we investigated showed higher prediction accuracy than CD-Hit in predicting partial KEGG function. Motif-based, machine-learning methods of annotation in new species were faster and had higher precision-recall than methods of homologous alignment or orthologous gene clustering. Gradient boosted ensemble methods and neural networks also predicted higher connectivity in reconstructed KEGG pathways, finding twice as many new pathway interactions than blast alignment. The use of motif-based, machine-learning algorithms in annotation software will allow researchers to develop powerful tools to interact with bacterial microbiomes in ways previously unachievable through homologous sequence alignment alone.

Adaptive Collaborative Soft Label Learning for Unsupervised Multi-View Feature Selection

Unsupervised multi-view feature selection aims to select informative features with multi-view features and unsupervised learning. It is a challenging problem due to the absence of explicit semantic supervision. Recently, graph theory and hard pseudo-label learning have been adopted to solve multi-view feature selection problems under the unsupervised learning paradigm. However, graph-based methods are difficult to support large-scale real scenarios due to the high computational complexity of graph construction. Moreover, existing methods based on hard pseudo-label learning generally result in significant information loss. In this article, we propose an Adaptive Collaborative Soft Label Learning (ACSLL) model for unsupervised multi-view feature selection. In this model, collaborative soft label learning and multi-view feature selection are integrated into a unified framework. Specifically, we learn the pseudo soft labels from each view feature by a simple and efficient method and fuse them with an adaptive weighting strategy into a joint soft label matrix. This matrix is further used for guiding the feature selection process to identify valuable features. An effective optimization strategy guaranteed with proven convergence is derived to iteratively solve this problem. Experiments demonstrate the superiority of the proposed method in both feature selection accuracy and efficiency.

Diabetes Detection Models in Mexican Patients by Combining Machine Learning Algorithms and Feature Selection Techniques for Clinical and Paraclinical Attributes: A Comparative Evaluation.

The development of medical diagnostic models to support healthcare professionals has witnessed remarkable growth in recent years. Among the prevalent health conditions affecting the global population, diabetes stands out as a significant concern. In the domain of diabetes diagnosis, machine learning algorithms have been widely explored for generating disease detection models, leveraging diverse datasets primarily derived from clinical studies. The performance of these models heavily relies on the selection of the classifier algorithm and the quality of the dataset. Therefore, optimizing the input data by selecting relevant features becomes essential for accurate classification. This research presents a comprehensive investigation into diabetes detection models by integrating two feature selection techniques: the Akaike information criterion and genetic algorithms. These techniques are combined with six prominent classifier algorithms, including support vector machine, random forest, k-nearest neighbor, gradient boosting, extra trees, and naive Bayes. By leveraging clinical and paraclinical features, the generated models are evaluated and compared to existing approaches. The results demonstrate superior performance, surpassing accuracies of 94%. Furthermore, the use of feature selection techniques allows for working with a reduced dataset. The significance of feature selection is underscored in this study, showcasing its pivotal role in enhancing the performance of diabetes detection models. By judiciously selecting relevant features, this approach contributes to the advancement of medical diagnostic capabilities and empowers healthcare professionals in making informed decisions regarding diabetes diagnosis and treatment.

Application of machine learning algorithms and feature selection in rapeseed (Brassica napus L.) breeding for seed yield

BackgroundStudying the relationships between rapeseed seed yield (SY) and its yield-related traits can assist rapeseed breeders in the efficient indirect selection of high-yielding varieties. However, since the conventional and linear methods cannot interpret the complicated relations between SY and other traits, employing advanced machine learning algorithms is inevitable. Our main goal was to find the best combination of machine learning algorithms and feature selection methods to maximize the efficiency of indirect selection for rapeseed SY.ResultsTo achieve that, twenty-five regression-based machine learning algorithms and six feature selection methods were employed. SY and yield-related data from twenty rapeseed genotypes were collected from field experiments over a period of 2 years (2019–2021). Root mean square error (RMSE), mean absolute error (MAE), and determination coefficient (R2) were used to evaluate the performance of the algorithms. The best performance with all fifteen measured traits as inputs was achieved by the Nu-support vector regression algorithm with quadratic polynomial kernel function (R2 = 0.860, RMSE = 0.266, MAE = 0.210). The multilayer perceptron neural network algorithm with identity activation function (MLPNN-Identity) using three traits obtained from stepwise and backward selection methods appeared to be the most efficient combination of algorithms and feature selection methods (R2 = 0.843, RMSE = 0.283, MAE = 0.224). Feature selection suggested that the set of pods per plant and days to physiological maturity along with plant height or first pod height from the ground are the most influential traits in predicting rapeseed SY.ConclusionThe results of this study showed that MLPNN-Identity along with stepwise and backward selection methods can provide a robust combination to accurately predict the SY using fewer traits and therefore help optimize and accelerate SY breeding programs of rapeseed.

Patient-independent epileptic seizure detection by stable feature selection

The detection of epileptic seizures in EEG signals is a challenging task because it requires careful review of multi-channel EEG recordings over a lengthy time interval. In general, EEG-based seizure detection is strongly dependent on the ability to select descriptive features that are also stable in the sense that they are not sensitive to changes in the training data. This study proposes and investigates a patient-independent seizure detection model that uses stable EEG-based features obtained by comparing multiple feature selection methods. The schemes considered can be divided into five categories, often referred to as similarity, information theoretic, sparse learning, statistical, and graph centrality feature selection methods. The stability of the features these schemes produce was evaluated by random forest classification, using the intersection, frequency, correlation, and similarity measures. In experiments with Temple University Hospital’s Seizure database of EEG records of 341 patients, unsupervised graph centrality feature selection was the most effective method, with a correct classification rate of 91.5%.

GastroNet: A robust attention‐based deep learning and cosine similarity feature selection framework for gastrointestinal disease classification from endoscopic images

AbstractDiseases of the Gastrointestinal (GI) tract significantly affect the quality of human life and have a high fatality rate. Accurate diagnosis of GI diseases plays a pivotal role in healthcare systems. However, processing large amounts of medical image data can be challenging for radiologists and other medical professionals, increasing the risk of inaccurate medical assessments. Computer‐aided Diagnosis systems provide help to doctors for rapid and accurate diagnosis, thus resulting in saving lives. Recently, many techniques are found in the literature that uses deep Convolutional Neural Network (CNN) models for accurate disease classification. However, they have limitations in their ability to detect deformation‐invariant features and lack robustness. The diseased region is highlighted, using attention‐based image generation and superimposition with original images. A lightweight deep CNN model is employed to get significant features. These features are further reduced using a Cosine similarity‐based technique. The proposed framework is assessed using the Kvasir dataset. To verify the effectiveness of the proposed framework, vast experiments are conducted. The overall accuracy of 97.68%, 99.02% precision, 96.37% recall, and an F‐measure of 97.68% are achieved using the 810 significant features. This reduction in features resulted in a significant reduction in classification time. The robustness of the framework can be observed not only in terms of considerable improvement in accuracy, but also in terms of precision as well as recall, and F‐measure.

High-order interaction feature selection for classification learning: A robust knowledge metric perspective

Feature selection is an important learning task in data mining and knowledge discovery. Nevertheless, the fuzziness, uncertainty, and noise presented by the data greatly complicate the construction of learning models. Moreover, most works focus on exploring low-order correlations between variables using low-dimensional mutual information, without paying attention to high-order interaction for multiple variables, resulting in the loss of some potentially important dependency information. Driven by these two issues, a robust knowledge metric approach is invented to perceive and excavate the latent information hidden in interaction. In this study, firstly, a robust fuzzy granularity space is constructed from different granular structures induced by different features, and the robust fuzzy uncertainty measures (RFUMs) are successively devised. Then, RFUMs are used to measure pair-wise, three-order, and even higher-order interaction dependencies among features. Further, a constrained high-order interaction evaluation function inspired by the N-gram language model is formulated, and a corresponding high-order interaction feature selection algorithm with RFUMs (HIFS-RFUMs) is designed. Next, comparative experiments with seven representative algorithms on twenty datasets illustrate its effectiveness. In addition, ablation experiments are conducted on the high-order interaction feature selection algorithm with fuzzy uncertainty measures (HIFS-FUMs) and the relative reduction algorithm with RFUMs (R2-RFUMs), which demonstrate the robustness of the metric and the effectiveness for mining high-order interactive features, respectively.

Analysis of Cardiovascular Disease using Machine Learning Techniques

Abstract: Detecting cardiovascular disease early is crucial in healthcare, especially in the field of cardiology. With 12 million deaths worldwide annually, the disease is a significant health concern. Early detection can lead to lifestyle changes that reduce the risk of complications. Machine learning techniques can be used to extract useful data from large datasets and make accurate predictions, requiring less investment. Machine learning algorithms can also be used to address unbalanced datasets and feature selection, which can improve diagnostic accuracy. Using ensemble learning with a machine learning algorithm, feature selection, and biomedical test values can help classify cardiovascular disease. Multiple classifier models can also be applied to improve accuracy with an ensemble classifier.

Using bacterial pan-genome-based feature selection approach to improve the prediction of minimum inhibitory concentration (MIC).

Background: Predicting the resistance profiles of antimicrobial resistance (AMR) pathogens is becoming more and more important in treating infectious diseases. Various attempts have been made to build machine learning models to classify resistant or susceptible pathogens based on either known antimicrobial resistance genes or the entire gene set. However, the phenotypic annotations are translated from minimum inhibitory concentration (MIC), which is the lowest concentration of antibiotic drugs in inhibiting certain pathogenic strains. Since the MIC breakpoints that classify a strain to be resistant or susceptible to specific antibiotic drug may be revised by governing institutes, we refrained from translating these MIC values into the categories "susceptible" or "resistant" but instead attempted to predict the MIC values using machine learning approaches. Results: By applying a machine learning feature selection approach on a Salmonella enterica pan-genome, in which the protein sequences were clustered to identify highly similar gene families, we showed that the selected features (genes) performed better than known AMR genes, and that models built on the selected genes achieved very accurate MIC prediction. Functional analysis revealed that about half of the selected genes were annotated as hypothetical proteins (i.e., with unknown functional roles), and that only a small portion of known AMR genes were among the selected genes, indicating that applying feature selection on the entire gene set has the potential of uncovering novel genes that may be associated with and may contribute to pathogenic antimicrobial resistances. Conclusion: The application of the pan-genome-based machine learning approach was indeed capable of predicting MIC values with very high accuracy. The feature selection process may also identify novel AMR genes for inferring bacterial antimicrobial resistance phenotypes.

Semantic-gap-oriented feature selection in hierarchical classification learning

Semantic gap is common issues in classification tasks. Different from traditional classification tasks, the semantic gap problem can introduce serious error propagation for hierarchical classification. In this paper, we propose a hierarchical feature selection method with label enhancement (HFSLE) to solve this challenge, which estimates the label distribution and selects subsets of features for each subtask. Firstly, HFSLE combines the label enhancement and hierarchical feature selection process so that the label enhancement benefits from the feedback of the loss function. Then, the label enhancement is constrained by the consistency of the parent-child distribution to obtain a plausible label distribution. Finally, feature selection is optimized based on intra-class consistency hierarchical regularization terms, which unifies feature subsets with hierarchical semantic descriptions to narrow the semantic gap. HFSLE gives a new perspective on the hierarchical feature selection based on label distribution rather than the original logical label. To demonstrate the superiority of the method, we compared it with six well-established feature selection methods on eight hierarchical data sets, and analyzed the convergence and robustness of the method. Extensive experiments demonstrate that the proposed method can perform favorably against advanced hierarchical feature selection approaches.

Classification of breast tumors by using a novel approach based on deep learning methods and feature selection

Cancer is one of the most insidious diseases that the most important factor in overcoming the cancer is early diagnosis and detection. The histo-pathological images are used to determine whether the tissue is cancerous and the type of cancer. As the result of examination on tissue images by the expert personnel, the cancer type, and stage of the tissue can be determined. However, this situation can cause both time and energy loss as well as personnel-related inspection errors. By the increased usage of computer-based decision methods in the last decades, it would be more efficient and accurate to detect and classify the cancerous tissues with computer-aided systems. As classical image processing methods were used for cancer-type detection in early studies, advanced deep learning methods based on recurrent neural networks and convolutional neural networks have been used more recently. In this paper, popular deep learning methods such as ResNet-50, GoogLeNet, InceptionV3, and MobilNetV2 are employed by implementing novel feature selection method in order to classify cancer type on a local binary class dataset and multi-class BACH dataset. The classification performance of the proposed feature selection implemented deep learning methods follows as for the local binary class dataset 98.89% and 92.17% for BACH dataset which is much better than most of the obtained results in literature. The obtained findings on both datasets indicates that the proposed methods can detect and classify the cancerous type of a tissue with high accuracy and efficiency.

Empirical Research on Machine Learning Models and Feature Selection for Traffic Congestion Prediction in Smart Cities

The development of smart cities has occurred over the past ten years. One primary goal of “smart city” initiatives is to lessen vehicle congestion. Several innovative technologies, including vehicular communications, navigation, and traffic control, have been created by Vehicle Networking System to address this problem. The traffic data gathered by smart devices aids in the forecasting of traffic in smart cities. This project created an Intelligent Traffic Congestion Management System (ITCMS) that uses machine learning techniques and traffic data from Kaggle to decrease the amount of time spent stuck in traffic. This study aims to assess feature selection methods and machine learning models for traffic forecasting in smart cities. The feature dimension is reduced using feature selection techniques, such information gain, correlation attribute, and principal component analysis. The recommended model successfully predicted traffic flow, assisting in the alleviation of congestion. The principal component analysis with random forest model outperforms the other machine learning models and has a 95% accuracy rate.

Differences Due to Sex and Sweetener on the Bioavailability of (Poly)phenols in Urine Samples: A Machine Learning Approach.

Metabolic diseases have been related to the overdrinking of high-sugar content beverages. As a result, the demand for alternative formulations based on plant-based ingredients with health-promoting properties has increased during the last few years. Nonetheless, the design and production of effective formulations requires understanding the bioavailability of these compounds. For this purpose, a two-month longitudinal trial with 140 volunteers was conducted to measure the beneficial effects of a maqui-citrus beverage, rich in (poly)phenols. From data obtained by quantifying metabolites present in urine samples, biostatistical and machine learning (data imputation, feature selection, and clustering) methods were applied to assess whether a volunteer's sex and the sweetener added to the beverage (sucrose, sucralose, or stevia) affected the bioavailability of (poly)phenol metabolites. Several metabolites have been described as being differentially influenced: 3,4-dihydroxyphenylacetic acid and naringenin with its derivatives were positively influenced by stevia and men, while eriodictyol sulfate and homoeridictyol glucunoride concentrations were enhanced with stevia and women. By examining groups of volunteers created by clustering analysis, patterns in metabolites' bioavailability distribution as a function of sex and/or sweeteners (or even due to an uncontrolled factor) were also discovered. These results underline the potential of stevia as a (poly)phenol bioavailability enhancer. Furthermore, they also evidence sex affects the bioavailability of (poly)phenols, pointing at a sex-dependent metabolic pathway regulation.

Consensus cluster structure guided multi-view unsupervised feature selection

As the volume of high-dimensional multi-view data continues to grow, there has been a significant development in multi-view unsupervised feature selection methods, particularly those that perform graph learning and feature selection simultaneously. These methods typically begin by constructing a consensus graph, which is then utilized to ensure that the projected samples maintain the local structure of data. However, these methods require data from multiple views to preserve the same manifold structure, which goes against the reality that similarities may vary across different views. On the other hand, despite inconsistencies between heterogeneous features, multiple views share a unique cluster structure. Inspired by this, we propose consensus cluster structure guided multi-view unsupervised feature selection (CCSFS). Specifically, we generate multiple cluster structures and fuse them into a consensus structure to guide feature selection. The proposed method unifies subspace learning, cluster analysis, consensus learning and sparse feature selection into one optimization framework. By leveraging the inherent interactions between these four subtasks, CCSFS can finally select informative and discriminative features. An efficient algorithm is carefully designed to solve the optimization problem of the objective function. We conduct extensive clustering experiments on seven multi-view datasets to demonstrate that the proposed method outperforms some of the latest competitors.