Incremental Feature Selection Research Articles

Machine Learning for Prediction of Resistance Scores in Wheat (Triticum aestivum L.)

ABSTRACTMachine learning methods were shown to improve the prediction accuracies of genomic prediction of resistance scores compared to methods like RR‐BLUP, which were originally designed for metric rather than ordinal response values. We conducted a cross‐validation study with 361 wheat genotypes evaluated for five fungal diseases. Our objective was to compare the prediction accuracy and the ability to identify the most resistant genotypes of 19 genomic prediction approaches. Each approach consisted of a different combination of prediction method (RR‐BLUP, an alternative method with heterogeneous marker variances, Bayesian generalized linear regression with an ordinal response, support vector machine, gradient boosting machine and random forest), predictor (single SNP markers, LD‐based haplotype blocks, 250 variables generated with an autoencoder and SNPs identified with incremental feature selection) and response value (untransformed and logit‐transformed resistance scores). In our dataset, RR‐BLUP was consistently among the methods with the largest prediction accuracies and the best abilities to identify resistant genotypes in four of five investigated traits. However, in P. triticina, using gradient boosting machine and random forest instead of RR‐BLUP increased the prediction accuracy from 0.64 to 0.71, indicating that machine learning methods may have an advantage over linear models in genomic prediction. We also found that even though there was a positive correlation between the prediction accuracy and Cohen's , a measure to judge how well the most resistant genotypes can be identified, the correlation is not perfect and a large value for the prediction accuracy does not necessarily translate into an equally large value.

Identification of Key Genes in Fetal Gut Development at Single-Cell Level by Exploiting Machine Learning Techniques.

The study of fetal gut development is critical due to its substantial influence on immediate neonatal and long-term adult health. Current research largely focuses on microbiome colonization, gut immunity, and barrier function, alongside the impact of external factors on these phenomena. Limited research has been dedicated to the categorization of developing fetal gut cells. Our study aimed to enhance our understanding of fetal gut development by employing advanced machine-learning techniques on single-cell sequencing data. This dataset consisted of 62,849 samples, each characterized by 33,694 distinct gene features. Four feature ranking algorithms were utilized to sort features according to their significance, resulting in four feature lists. Then, these lists were fed into an incremental feature selection method to extract essential genes, classification rules, and build efficient classifiers. Several important genes were recognized by multiple feature ranking algorithms, such as FGG, MDK, RBP1, RBP2, IGFBP7, and SPON2. These features were key in differentiating specific developing intestinal cells, including epithelial, immune, mesenchymal, and vasculature cells of the colon, duo jejunum, and ileum cells. The classification rules showed special gene expression patterns on some intestinal cell types and the efficient classifiers can be useful tools for identifying intestinal cells.

Identification of gene and protein signatures associated with long-term effects of COVID-19 on the immune system after patient recovery by analyzing single-cell multi-omics data using a machine learning approach

Viral infections significantly impact the immune system, and impact will persist until recovery. However, the influence of severe acute respiratory syndrome coronavirus 2 infection on the homeostatic immune status and secondary immune response in recovered patients remains unclear. To investigate these persistent alterations, we employed five feature-ranking algorithms (LASSO, MCFS, RF, CATBoost, and XGBoost), incremental feature selection, synthetic minority oversampling technique and two classification algorithms (decision tree and k-nearest neighbors) to analyze multi-omics data (surface proteins and transcriptome) from coronavirus disease 2019 (COVID-19) recovered patients and healthy controls post-influenza vaccination. The single-cell multi-omics dataset was divided into five subsets corresponding to five immune cell subtypes: B cells, CD4+ T cells, CD8+ T cells, Monocytes, and Natural Killer cells. Each cell was represented by 28,402 scRNA-seq (RNA) features, 3 Hash Tag Oligo (HTO) features, 138 Cellular indexing of transcriptomes and epitopes by sequencing (CITE) features and 23,569 Single Cell Transform (SCT) features. Some multi-omics markers were identified and effective classifiers were constructed. Our findings indicate a distinct immune status in COVID-19 recovered patients, characterized by low expression of ribosomal protein (RPS26) and high expression of immune cell surface proteins (CD33, CD48). Notably, TMEM176B, a membrane protein, was highly expressed in monocytes of COVID-19 convalescent patients. These observations aid in discerning molecular differences among immune cell subtypes and contribute to understanding the prolonged effects of COVID-19 on the immune system, which is valuable for treating infectious diseases like COVID-19.

Machine Learning in Identifying Marker Genes for Congenital Heart Diseases of Different Cardiac Cell Types.

Congenital heart disease (CHD) represents a spectrum of inborn heart defects influenced by genetic and environmental factors. This study advances the field by analyzing gene expression profiles in 21,034 cardiac fibroblasts, 73,296 cardiomyocytes, and 35,673 endothelial cells, utilizing single-cell level analysis and machine learning techniques. Six CHD conditions: dilated cardiomyopathy (DCM), donor hearts (used as healthy controls), hypertrophic cardiomyopathy (HCM), heart failure with hypoplastic left heart syndrome (HF_HLHS), Neonatal Hypoplastic Left Heart Syndrome (Neo_HLHS), and Tetralogy of Fallot (TOF), were investigated for each cardiac cell type. Each cell sample was represented by 29,266 gene features. These features were first analyzed by six feature-ranking algorithms, resulting in several feature lists. Then, these lists were fed into incremental feature selection, containing two classification algorithms, to extract essential gene features and classification rules and build efficient classifiers. The identified essential genes can be potential CHD markers in different cardiac cell types. For instance, the LASSO identified key genes specific to various heart cell types in CHD subtypes. FOXO3 was found to be up-regulated in cardiac fibroblasts for both Dilated and hypertrophic cardiomyopathy. In cardiomyocytes, distinct genes such as TMTC1, ART3, ARHGAP24, SHROOM3, and XIST were linked to dilated cardiomyopathy, Neo-Hypoplastic Left Heart Syndrome, hypertrophic cardiomyopathy, HF-Hypoplastic Left Heart Syndrome, and Tetralogy of Fallot, respectively. Endothelial cell analysis further revealed COL25A1, NFIB, and KLF7 as significant genes for dilated cardiomyopathy, hypertrophic cardiomyopathy, and Tetralogy of Fallot. LightGBM, Catboost, MCFS, RF, and XGBoost further delineated key genes for specific CHD subtypes, demonstrating the efficacy of machine learning in identifying CHD-specific genes. Additionally, this study developed quantitative rules for representing the gene expression patterns related to CHDs. This research underscores the potential of machine learning in unraveling the molecular complexities of CHD and establishes a foundation for future mechanism-based studies.

Matrix-based incremental feature selection method using weight-partitioned multigranulation rough set

Incremental feature selection methods have gained increasing research attention as they improve the efficiency of feature selection for dynamic datasets. Multigranulation rough set, as an extension of rough set theory, allows for a comprehensive and rational analysis of problems from multiple hierarchical and granular perspectives. However, existing research on granularity partitioning relies on the decision maker's subjective experience, which lacks convincing power. In this paper, we propose a generalized multigranulation neighborhood rough set based on weight partition model, using a matrix form. We discuss several properties and define a new entropy measure to evaluate feature importance. A heuristic feature selection algorithm is developed based on this entropy to search for the optimal subset. Furthermore, we discuss dynamic updating mechanism and design two incremental feature selection algorithms. Finally, we conduct experiments on 12 public datasets to evaluate the performance of the proposed algorithms and validate their effectiveness and efficiency in feature selection for both static and dynamic datasets.

Detecting key genes relative expression orderings as biomarkers for machine learning-based intelligent screening and analysis of type 2 diabetes mellitus

Type 2 diabetes (T2D), a prevalent chronic metabolic disease of public health concern, is growing rapidly worldwide. In order to characterize robust transcriptomic differences between T2D and the healthy pancreatic islets, we proposed a novel data analysis approach based on the within-sample relative expression orderings of genes (REOs) and machine learning, which can efficiently integrate data from diverse gene expression datasets and help discover some potential signatures. We first identified the overlapped disease-specific reverse REOs from three bulk islets expression profiles that are substantially distinct. Subsequently, the minimum-Redundancy-Maximum-Relevance (mRMR) combined with incremental feature selection (IFS) strategy was applied to select the optimal reverse REOs by using the support vector machine (SVM). As a result, 7 optimal reverse REOs encompassing 12 genes were found, which could produce encouraging predictive performance on training data, independent testing, and external islets data, showcasing robust prediction capability and superior generalization despite the heterogeneity of data. Furthermore, the analysis of differential expression gene (DEG) in bulk islets data and pseudobulk method in single-cell RNA sequencing (scRNA-seq) islets data reaffirmed the crucial roles of these 7 REOs in islets function and disease progression. In conclusion, these REOs are robust biomarkers in T2D and may also be potential therapeutic targets for T2D. We hope these findings can help elucidate the underlying pathogenic mechanisms of T2D and contribute to the treatment of diabetes. The source code is available at https://github.com/xiexq007/T2D-REO.

Consistency approximation: Incremental feature selection based on fuzzy rough set theory

Fuzzy Rough Set Theory (FRST)-based feature selection has been widely used as a preprocessing step to handle dynamic and large datasets. However, large-scale or high-dimensional datasets remain intractable for FRST-based feature selection approaches due to high space complexity and unsatisfactory classification performance. To overcome these challenges, we propose a Consistency Approximation (CA)-based framework for incremental feature selection. By exploring CA, we introduce a novel significance measure and a tri-accelerator. The CA-based significance measure provides a mechanism for each sample in the universe to keep members with different class labels within its fuzzy neighbourhood as far as possible, while keeping members with the same label as close as possible. Furthermore, our tri-accelerator reduces the search space and decreases the computational space with a theoretical lower bound. The experimental results demonstrate the superiority of our proposed algorithm compared to state-of-the-art methods on efficiency and classification accuracy, especially for large-scale and high-dimensional datasets.

Cost-effective genomic prediction of critical economic traits in sturgeons through low-coverage sequencing

Low-coverage whole-genome sequencing (LCS) offers a cost-effective alternative for sturgeon breeding, especially given the lack of SNP chips and the high costs associated with whole-genome sequencing. In this study, the efficiency of LCS for genotype imputation and genomic prediction was assessed in 643 sequenced Russian sturgeons (∼13.68×). The results showed that using BaseVar+STITCH at a sequencing depth of 2× with a sample size larger than 300 resulted in the highest genotyping accuracy. In addition, when the sequencing depth reached 0.5× and SNP density was reduced to 50 K through linkage disequilibrium pruning, the prediction accuracy was comparable to that of whole sequencing depth. Furthermore, an incremental feature selection method has the potential to improve prediction accuracy. This study suggests that the combination of LCS and imputation can be a cost-effective strategy, contributing to the genetic improvement of economic traits and promoting genetic gains in aquaculture species.

Identification of RNA‐dependent liquid‐liquid phase separation proteins using an artificial intelligence strategy

RNA-dependent liquid-liquid phase separation (LLPS) proteins play critical roles in cellular processes such as stress granule formation, DNA repair, RNA metabolism, germ cell development, and protein translation regulation. The abnormal behavior of these proteins is associated with various diseases, particularly neurodegenerative disorders like amyotrophic lateral sclerosis and frontotemporal dementia, making their identification crucial. However, conventional biochemistry-based methods for identifying these proteins are time-consuming and costly. Addressing this challenge, our study developed a robust computational model for their identification. We constructed a comprehensive dataset containing 137 RNA-dependent and 606 non-RNA-dependent LLPS protein sequences, which were then encoded using amino acid composition, composition of K-spaced amino acid pairs, Geary autocorrelation, and conjoined triad methods. Through a combination of correlation analysis, mutual information scoring, and incremental feature selection, we identified an optimal feature subset. This subset was used to train a random forest model, which achieved an accuracy of 90% when tested against an independent dataset. This study demonstrates the potential of computational methods as efficient alternatives for the identification of RNA-dependent LLPS proteins. To enhance the accessibility of the model, a user-centric web server has been established and can be accessed via the link: http://rpp.lin-group.cn.

Rough set Theory-Based group incremental approach to feature selection

Feature selection in a dynamic learning system often encounters challenges from sample variation. Incremental selection techniques using rough set theory (RST) address this challenge, to a certain degree, but two issues remain: (1) high redundancy and (2) low efficiency in processing high-dimensional datasets. To address these issues and further improve feature selection, we develop triple nested equivalence class (TNEC) RST for a group incremental approach to feature selection. In particular, we construct a group TNEC to enable universe-reduction learning to help consistently filter out inconsequential samples and incrementally update dependencies. Furthermore, with TNEC-based incremental partitioning, we develop a novel dependency computing technique to reduce redundant features and avoid repeated learning. We conducted experimental verification using 18 dynamic datasets, including ultra-high-dimensional datasets. Tests on five incremental datasets from each of the 18 original datasets validated that the group TNEC approach significantly outperformed the state-of-the-art methods in terms of classification efficiency, selection accuracy, feature significance, and suitability for ultra-high-dimensional data. In both incremental and static feature selection cases, the TNEC approach extended the ability of evolutionary and other types of heuristic learning algorithms in handling sample variations.

Machine Learning Reveals Impacts of Smoking on Gene Profiles of Different Cell Types in Lung.

Smoking significantly elevates the risk of lung diseases such as chronic obstructive pulmonary disease (COPD) and lung cancer. This risk is attributed to the harmful chemicals in tobacco smoke that damage lung tissue and impair lung function. Current research on the impact of smoking on gene expression in specific lung cells is limited. This study addresses this gap by analyzing gene expression profiles at the single-cell level from 43,539 lung endothelial cells, 234,349 lung epithelial cells, 189,843 lung immune cells, and 16,031 lung stromal cells using advanced machine learning techniques. The data, categorized by different lung cell types, were classified into three smoking states: active smoker, former smoker, and never smoker. Each cell sample encompassed 28,024 feature genes. Employing an incremental feature selection method within a computational framework, several specific genes have been identified as potential markers of smoking status in different lung cell types. These include B2M, EEF1A1, and TPT1 in lung endothelial cells; FTL and MT-ATP8 in lung epithelial cells; HLA-B and HLA-C in lung immune cells; and HSP90B1 and LCN2 in lung stroma cells. Additionally, this study developed quantitative rules for representing the gene expression patterns related to smoking. This research highlights the potential of machine learning in oncology, enhancing our molecular understanding of smoking's harm and laying the groundwork for future mechanism-based studies.

Incremental feature selection approach to multi-dimensional variation based on matrix dominance conditional entropy for ordered data set

Incremental feature selection approach to multi-dimensional variation based on matrix dominance conditional entropy for ordered data set

Excavation of gene markers associated with pancreatic ductal adenocarcinoma based on interrelationships of gene expression.

Pancreatic ductal adenocarcinoma (PDAC) accounts for 95% of all pancreatic cancer cases, posing grave challenges to its diagnosis and treatment. Timely diagnosis is pivotal for improving patient survival, necessitating the discovery of precise biomarkers. An innovative approach was introduced to identify gene markers for precision PDAC detection. The core idea of our method is to discover gene pairs that display consistent opposite relative expression and differential co-expression patterns between PDAC and normal samples. Reversal gene pair analysis and differential partial correlation analysis were performed to determine reversal differential partial correlation (RDC) gene pairs. Using incremental feature selection, the authors refined the selected gene set and constructed a machine-learning model for PDAC recognition. As a result, the approach identified 10 RDC gene pairs. And the model could achieve a remarkable accuracy of 96.1% during cross-validation, surpassing gene expression-based models. The experiment on independent validation data confirmed the model's performance. Enrichment analysis revealed the involvement of these genes in essential biological processes and shed light on their potential roles in PDAC pathogenesis. Overall, the findings highlight the potential of these 10 RDC gene pairs as effective diagnostic markers for early PDAC detection, bringing hope for improving patient prognosis and survival.

Identifying Flare-indicative Photospheric Magnetic Field Parameters from Multivariate Time-series Data of Solar Active Regions

Photospheric magnetic field parameters are frequently used to analyze and predict solar events. Observation of these parameters over time, i.e., representing solar events by multivariate time-series (MVTS) data, can determine relationships between magnetic field states in active regions and extreme solar events, e.g., solar flares. We can improve our understanding of these events by selecting the most relevant parameters that give the highest predictive performance. In this study, we propose a two-step incremental feature selection method for MVTS data using a deep-learning model based on long short-term memory (LSTM) networks. First, each MVTS feature (magnetic field parameter) is evaluated individually by a univariate sequence classifier utilizing an LSTM network. Then, the top performing features are combined to produce input for an LSTM-based multivariate sequence classifier. Finally, we tested the discrimination ability of the selected features by training downstream classifiers, e.g., Minimally Random Convolutional Kernel Transform and support vector machine. We performed our experiments using a benchmark data set for flare prediction known as Space Weather Analytics for Solar Flares. We compared our proposed method with three other baseline feature selection methods and demonstrated that our method selects more discriminatory features compared to other methods. Due to the imbalanced nature of the data, primarily caused by the rarity of minority flare classes (e.g., the X and M classes), we used the true skill statistic as the evaluation metric. Finally, we reported the set of photospheric magnetic field parameters that give the highest discrimination performance in predicting flare classes.

Incremental feature selection for large-scale hierarchical classification with the arrival of new samples

Incremental feature selection for large-scale hierarchical classification with the arrival of new samples

Exploring Prognostic Gene Factors in Breast Cancer via Machine Learning.

Breast cancer remains the most prevalent cancer in women. To date, its underlying molecular mechanisms have not been fully uncovered. The determination of gene factors is important to improve our understanding on breast cancer, which can correlate the specific gene expression and tumor staging. However, the knowledge in this regard is still far from complete. Thus, this study aimed to explore these knowledge gaps by analyzing existing gene expression profile data from 3149 breast cancer samples, where each sample was represented by the expression of 19,644 genes and classified into Nottingham histological grade (NHG) classes (Grade 1, 2, and 3). To this end, a machine learning-based framework was designed. First, the profile data were analyzed by using seven feature ranking algorithms to evaluate the importance of features (genes). Seven feature lists were generated, each of which sorted features in accordance with feature importance evaluated from a special aspect. Then, the incremental feature selection method was applied to each list to determine essential features for classification and building efficient classifiers. Consequently, overlapping genes, such as AURKA, CBX2, and MYBL2, were deemed as potentially related to breast cancer malignancy and prognosis, indicating that such genes were identified to be important by multiple feature ranking algorithms. In addition, the study formulated classification rules to reflect special gene expression patterns for three NHG classes. Some genes and rules were analyzed and supported by recent literature, providing new references for studying breast cancer.

Deep-STP: a deep learning-based approach to predict snake toxin proteins by using word embeddings.

Snake venom contains many toxic proteins that can destroy the circulatory system or nervous system of prey. Studies have found that these snake venom proteins have the potential to treat cardiovascular and nervous system diseases. Therefore, the study of snake venom protein is conducive to the development of related drugs. The research technologies based on traditional biochemistry can accurately identify these proteins, but the experimental cost is high and the time is long. Artificial intelligence technology provides a new means and strategy for large-scale screening of snake venom proteins from the perspective of computing. In this paper, we developed a sequence-based computational method to recognize snake toxin proteins. Specially, we utilized three different feature descriptors, namely g-gap, natural vector and word 2 vector, to encode snake toxin protein sequences. The analysis of variance (ANOVA), gradient-boost decision tree algorithm (GBDT) combined with incremental feature selection (IFS) were used to optimize the features, and then the optimized features were input into the deep learning model for model training. The results show that our model can achieve a prediction performance with an accuracy of 82.00% in 10-fold cross-validation. The model is further verified on independent data, and the accuracy rate reaches to 81.14%, which demonstrated that our model has excellent prediction performance and robustness.

Predicting Critical Path of Labor Dispute Resolution in Legal Domain by Machine Learning Models Based on SHapley Additive exPlanations and Soft Voting Strategy

The labor dispute is one of the most common civil disputes. It can be resolved in the order of the following steps, which include mediation in arbitration, arbitration award, first-instance mediation, first-instance judgment, and second-instance judgment. The process can cease at any step when it is successfully resolved. In recent years, due to the increasing rights awareness of employees, the number of labor disputes has been rising annually. However, resolving labor disputes is time-consuming and labor-intensive, which brings a heavy burden to employees and dispute resolution institutions. Using artificial intelligence algorithms to identify and predict the critical path of labor dispute resolution is helpful for saving resources and improving the efficiency of, and reducing the cost of dispute resolution. In this study, a machine learning approach based on Shapley Additive exPlanations (SHAP) and a soft voting strategy is applied to predict the critical path of labor dispute resolution. We name our approach LDMLSV (stands for Labor Dispute Machine Learning based on SHapley additive exPlanations and Voting). This approach employs three machine learning models (Random Forest, Extra Trees, and CatBoost) and then integrates them using a soft voting strategy. Additionally, SHAP is used to explain the model and analyze the feature contribution. Based on the ranking of feature importance obtained from SHAP and an incremental feature selection method, we obtained an optimal feature subset comprising 33 features. The LDMLSV achieves an accuracy of 0.90 on this optimal feature subset. Therefore, the proposed approach is a highly effective method for predicting the critical path of labor dispute resolution.

Promoter Prediction in Agrobacterium tumefaciens Strain C58 by Using Artificial Intelligence Strategies.

Promoters are the genomic regions upstream of genes that RNA polymerase binds in order to initiate gene transcription. Understanding the regulation of gene expression depends on being able to identify promoters, because they are the most important component of gene expression. Agrobacterium tumefaciens (A. tumefaciens) strain C58 was the subject of this study with the goal of creating a machine learning-based model to predict promoters. In this study, nucleotide density (ND), k-mer, and one-hot were used to encode the promoter sequence. Support vector machine (SVM) on fivefold cross-validation with incremental feature selection (IFS) was used to optimize the generated features. These improved characteristics were then used to distinguish promoter sequences by feeding them into the random forest (RF) classifier. Tenfold cross-validation (CV) analysis revealed that the projected model has the ability to produce an accuracy of 84.22%.

Identification of key genes associated with persistent immune changes and secondary immune activation responses induced by influenza vaccination after COVID-19 recovery by machine learning methods

COVID-19 is hypothesized to exert enduring effects on the immune systems of patients, leading to alterations in immune-related gene expression. This study aimed to scrutinize the persistent implications of SARS-CoV-2 infection on gene expression and its influence on subsequent immune activation responses. We designed a machine learning-based approach to analyze transcriptomic data from both healthy individuals and patients who had recovered from COVID-19. Patients were categorized based on their influenza vaccination status and then compared with healthy controls. The initial sample set encompassed 86 blood samples from healthy controls and 72 blood samples from recuperated COVID-19 patients prior to influenza vaccination. The second sample set included 123 blood samples from healthy controls and 106 blood samples from recovered COVID-19 patients who had been vaccinated against influenza. For each sample, the dataset captured expression levels of 17,060 genes. Above two sample sets were first analyzed by seven feature ranking algorithms, yielding seven feature lists for each dataset. Then, each list was fed into the incremental feature selection method, incorporating three classic classification algorithms, to extract essential genes, classification rules and build efficient classifiers. The genes and rules were analyzed in this study. The main findings included that NEXN and ZNF354A were highly expressed in recovered COVID-19 patients, whereas MKI67 and GZMB were highly expressed in patients with secondary immune activation post-COVID-19 recovery. These pivotal genes could provide valuable insights for future health monitoring of COVID-19 patients and guide the creation of continued treatment regimens.