Perform Feature Selection Research Articles

Supervised spectral feature selection with neighborhood rough set

Spectral feature selection, an excellent dimensionality reduction method, is extensively used in knowledge mining and protein sequence analysis. However, the graph representation derived from data with potential noises significantly impacts feature selection performance. Similarly, its performance deteriorates when label information is ignored, as the selected features have poor discrimination ability. To overcome these drawbacks, this article optimizes neighborhood structure and further constructs the conditional entropy based on neighborhood purity to generate a precise graph representation incorporating label information to enhance the compactness of intra-class samples and increase the dispersion between inter-class samples. Then this article proposes a novel spectral feature selection method. Specifically, the method dynamically learns the precise graph representation from low-dimension space, which can effectively preserve the local structure of data, and the global structure is preserved by introducing a constraint term that maintains the graph representation before and after dimension reduction. Additionally, ℓ2,1-norm is introduced to ensure row sparsity and select the valuable features. The alternative optimization algorithm is employed to solve the optimization problem and its convergence is theoretically analyzed. Finally, to assess the performance of the proposed method, a series of comprehensive experiments are performed on several real-world datasets.

NJmat: Data-Driven Machine Learning Interface to Accelerate Material Design.

Machine learning techniques have significantly transformed the way materials scientists conduct research. However, the widespread deployment of machine learning software in daily experimental and simulation research for materials and chemical design has been limited. This is partly due to the substantial time investment and learning curve associated with mastering the necessary codes and computational environments. In this paper, we introduce a user-friendly, data-driven machine learning interface featuring multiple "button-clicking" functionalities to streamline the design of materials and chemicals. This interface automates the processes of transforming materials and molecules, performing feature selection, constructing machine learning models, making virtual predictions, and visualizing results. Such automation accelerates materials prediction and analysis in the inverse design process, aligning with the time criteria outlined by the Materials Genome Initiative. With simple button clicks, researchers can build machine learning models and predict new materials once they have gathered experimental or simulation data. Beyond the ease of use, NJmat offers three additional features for data-driven materials design: (1) automatic feature generation for both inorganic materials (from chemical formulas) and organic molecules (from SMILES), (2) automatic generation of Shapley plots, and (3) automatic construction of "white-box" genetic models and decision trees to provide scientific insights. We present case studies on surface design for halide perovskite materials encompassing both inorganic and organic species. These case studies illustrate general machine learning models for virtual predictions as well as the automatic featurization and Shapley/genetic model construction capabilities. We anticipate that this software tool will expedite materials and molecular design within the scope of the Materials Genome Initiative, particularly benefiting experimentalists.

Design of a precise ensemble expert system for crop yield prediction using machine learning analytics

AbstractAgriculture is facing significant challenges in the development of crop yield forecasts, which are important aspects of decision‐making at the international, regional, and local levels. The area of agriculture is attracting growing attention because of increasing the demand for food supplies. To ensure future food supplies, crop yield prediction (CYP) provides the best decision‐making to assist farmers in agricultural yield forecasting efficiently. Nevertheless, CYP is a difficult endeavor because of the intricacy of the underlying mechanisms and the effect of numerous factors, including weather patterns, soil characteristics, and crop management techniques. In today's era, ensemble learning (EL) approaches have recently demonstrated significant promise for enhancing the reliability and accuracy of CYP. The success of the EL techniques depends on several facts, including how the base learner models are trained and how these are combined. This study provides important insights into the EL techniques for CYP. This paper proposes an expert system model named precise ensemble expert system for crop yield prediction (PEESCYP) to predict the best crop for agricultural land. The proposed PEESCYP model employs multiple imputation by chained equation (MICE) data imputation technique to treat the missing values of the collected dataset, the isolation forest (IF) technique for outlier detection, the ant colony optimization (ACO) technique to perform feature selection, robust scaling (RS) technique to perform data normalization, and the extra tree (ET) is used for classification to overcome the variance and overfitting problem of the single classifiers. The measurements of the proposed PEESCYP model have been collected by means of accuracy, precision, recall, and F‐1 score using a prepared dataset, which is collected from International Crops Research Institute for the Semi‐Arid Tropics (ICRISAT), and the proposed model is compared with different single‐classifier based ML models, EL models, and various existing models available in the literature. The results of this experiment underline that the proposed PEESCYP model outperforms the others.

The artificial intelligence advantage: Supercharging exploratory data analysis.

Explorative data analysis (EDA) is a critical step in scientific projects, aiming to uncover valuable insights and patterns within data. Traditionally, EDA involves manual inspection, visualization, and various statistical methods. The advent of artificial intelligence (AI) and machine learning (ML) has the potential to improve EDA, offering more sophisticated approaches that enhance its efficacy. This review explores how AI and ML algorithms can improve feature engineering and selection during EDA, leading to more robust predictive models and data-driven decisions. Tree-based models, regularized regression, and clustering algorithms were identified as key techniques. These methods automate feature importance ranking, handle complex interactions, perform feature selection, reveal hidden groupings, and detect anomalies. Real-world applications include risk prediction in total hip arthroplasty and subgroup identification in scoliosis patients. Recent advances in explainable AI and EDA automation show potential for further improvement. The integration of AI and ML into EDA accelerates tasks and uncovers sophisticated insights. However, effective utilization requires a deep understanding of the algorithms, their assumptions, and limitations, along with domain knowledge for proper interpretation. As data continues to grow, AI will play an increasingly pivotal role in EDA when combined with human expertise, driving more informed, data-driven decision-making across various scientific domains. Level of Evidence: Level V - Expert opinion.

Feature selection followed by a novel residuals-based normalization that includes variance stabilization simplifies and improves single-cell gene expression analysis

Normalization is a crucial step in the analysis of single-cell RNA-sequencing (scRNA-seq) counts data. Its principal objectives are reduction of systematic biases primarily introduced through technical sources and transformation of counts to make them more amenable for the application of established statistical frameworks. In the standard workflows, normalization is followed by feature selection to identify highly variable genes (HVGs) that capture most of the biologically meaningful variation across the cells. Here, we make the case for a revised workflow by proposing a simple feature selection method and showing that we can perform feature selection before normalization by relying on observed counts. We highlight that the feature selection step can be used to not only select HVGs but to also identify stable genes. We further propose a novel variance stabilization transformation inclusive residuals-based normalization method that in fact relies on the stable genes to inform the reduction of systematic biases. We demonstrate significant improvements in downstream clustering analyses through the application of our proposed methods on biological truth-known as well as simulated counts datasets. We have implemented this novel workflow for analyzing high-throughput scRNA-seq data in an R package called Piccolo.

A Regularized Cox Hierarchical Model for Incorporating Annotation Information in Predictive Omic Studies.

Associated with high-dimensional omics data there are often "meta-features" such as biological pathways and functional annotations, summary statistics from similar studies that can be informative for predicting an outcome of interest. We introduce a regularized hierarchical framework for integrating meta-features, with the goal of improving prediction and feature selection performance with time-to-event outcomes. A hierarchical framework is deployed to incorporate meta-features. Regularization is applied to the omic features as well as the meta-features so that high-dimensional data can be handled at both levels. The proposed hierarchical Cox model can be efficiently fitted by a combination of iterative reweighted least squares and cyclic coordinate descent. In a simulation study we show that when the external meta-features are informative, the regularized hierarchical model can substantially improve prediction performance over standard regularized Cox regression. We illustrate the proposed model with applications to breast cancer and melanoma survival based on gene expression profiles, which show the improvement in prediction performance by applying meta-features, as well as the discovery of important omic feature sets with sparse regularization at meta-feature level. The proposed hierarchical regularized regression model enables integration of external meta-feature information directly into the modeling process for time-to-event outcomes, improves prediction performance when the external meta-feature data is informative. Importantly, when the external meta-features are uninformative, the prediction performance based on the regularized hierarchical model is on par with standard regularized Cox regression, indicating robustness of the framework. In addition to developing predictive signatures, the model can also be deployed in discovery applications where the main goal is to identify important features associated with the outcome rather than developing a predictive model.

A New Performance Metric to Evaluate Filter Feature Selection Methods in Text Classification

High dimensionality and sparsity are the primary issues in text classification. Using feature selection approaches, the most effective way to solve the problem is to select a subset of features. The most common and effective methods used for this process are filter techniques. Various performance metrics such as Micro-F1, Macro-F1, and Accuracy are used to evaluate the performance of filter methods used for feature selection on datasets  Such methods work depending on a classification algorithm. However, when selecting features in filter techniques, the information on the individual features is evaluated without considering the relationship between the features. In such an approach, the actual performance of the filter technique used in feature selection may not be determined. In such a case, it causes the existing methods to be insufficient in testing the validity of the proposed method. For this purpose, this study suggests a novel performance metric called Selection Error (SE) to determine the actual performance evaluation of filter techniques. The Selection Error metric allows us to analyze the information value of the selected features more accurately than existing methods without relying on a classifier. The feature selection performance of the filtering approaches was performed on six different datasets with both The Selection Error and traditional performance metrics. When the results are examined, it is seen that there is a strong relationship between the proposed performance metric and the classification performance metric results. The Selection Error aims to significantly contribute to the literature by demonstrating the success of filtering feature selection methods, regardless of classifier performance. 

Pairwise-Constraint-Guided Multi-View Feature Selection by Joint Sparse Regularization and Similarity Learning

Feature selection is a basic and important step in real applications, such as face recognition and image segmentation. In this paper, we propose a new weakly supervised multi-view feature selection method by utilizing pairwise constraints, i.e., the pairwise constraint-guided multi-view feature selection (PCFS for short) method. In this method, linear projections of all views and a consistent similarity graph with pairwise constraints are jointly optimized to learning discriminative projections. Meanwhile, the l2,0-norm-based row sparsity constraint is imposed on the concatenation of projections for discriminative feature selection. Then, an iterative algorithm with theoretically guaranteed convergence is developed for the optimization of PCFS. The performance of the proposed PCFS method was evaluated by comprehensive experiments on six benchmark datasets and applications on cancer clustering. The experimental results demonstrate that PCFS exhibited competitive performance in feature selection in comparison with related models.

Research on virtual entertainment robots based on machine learning algorithms providing psychological health services for college students

In modern society, college students are facing increasing psychological pressure and mental health problems. In this context, virtual entertainment robots have become a promising form of mental health services, which can utilize machine learning algorithms to provide personalized psychological support and guidance by analyzing a large amount of psychological data and user information. Study the use of sample calculation and screening methods to determine the number of samples and perform feature selection to improve algorithm performance. Then analyze the detection effect and evaluate the effectiveness of the algorithm. By designing the architecture of a virtual entertainment robot and adopting anti-interference strategies to ensure that the robot can accurately recognize mental health information, text recognition technology was implemented, its effectiveness was evaluated, and further multi-source information recognition was carried out to improve recognition accuracy. Finally, a psychological health evaluation system for college students was constructed, and corresponding psychological health service strategies were proposed to meet the needs of college students. The results of this study indicate that virtual entertainment robots based on machine learning algorithms can effectively provide mental health services, providing support and guidance for the mental health problems of college students.

Intelligent Recommendation Systems Powered by Consensus Neural Networks: The Ultimate Solution for Finding Suitable Chiral Chromatographic Systems?

The selection of suitable combinations of chiral stationary phases (CSPs) and mobile phases (MPs) for the enantioresolution of chiral compounds is a complex issue that often requires considerable experimental effort and can lead to significant waste. Linking the structure of a chiral compound to a CSP/MP system suitable for its enantioseparation can be an effective solution to this problem. In this study, we evaluate algorithmic tools for this purpose. Our proposed consensus model, which uses multiple optimized artificial neural networks (ANNs), shows potential as an intelligent recommendation system (IRS) for ranking chromatographic systems suitable for the enantioresolution of chiral compounds with different molecular structures. To evaluate the IRS potential in a proof-of-concept stage, 56 structural descriptors for 56 structurally unrelated chiral compounds across 14 different families are considered. Chromatographic systems under study comprise 7 cellulose and amylose derivative CSPs and acetonitrile or methanol aqueous MPs (14 chromatographic systems in all). The ANNs are optimized using a fit-for-purpose version of the chaotic neural network algorithm with competitive learning (CCLNNA), a novel approach not previously applied in the chemical domain. CCLNNA is adapted to define the inner ANN complexity and perform feature selection of the structural descriptors. A customized target function evaluates the correctness of recommending the appropriate CSP/MP system. The ANN-consensus model exhibits no advisory failures and requires only an experimental attempt to verify the IRS recommendation for complete enantioresolution. This outstanding performance highlights its potential to effectively resolve this problem.

AO-SVM: a machine learning model for predicting water quality in the cauvery river

Water pollution is a significant cause of death globally, resulting in 1.8 million deaths annually due to waterborne diseases. Assessing water quality is a complex process that involves identifying contaminants in water sources and determining whether it is safe for human consumption. In this study, we utilized the Cauvery River dataset to develop a model for evaluating water quality. The aim of our research was to proficiently perform feature selection and classification tasks. We introduced a novel technique called the Aquila Optimization Support Vector Machine (AO-SVM), an advanced and effective machine learning system for predicting water quality. Here SVM is used for the classification, and the Aquila algorithm is used for optimizing SVM. The results show that the proposed method achieved a maximum accuracy rate of 96.3%, an execution time of 0.75 s, a precision of 93.9%, a recall rate of 95.1%, and an F1-Score value of 94.7%. The suggested AO-SVM model outperformed all other existing classification models regarding classification accuracy and other parameters.

An Approach for Efficient and Accurate Phishing Website Prediction Using Improved ML Classifier Performance for Feature Selection

The article discusses the use of machine learning (ML) to combat phishing websites, which are deceptive sites that mimic trusted entities to steal sensitive information. This is why the continued invention of methods of identifying and counteracting phishing threats is beneficial. Such attacks pose significant risks to the integrity of online security. To enhance the success rate and specificity of predicting phishing websites, this study proposes a new approach that utilizes machine learning algorithms. To enhance the methods mentioned above and achieve better results in classification and better prediction of customer behaviour, the main points exposed to further transformations are increasing classifier accuracy and selecting an optimal feature space. Traditional anti-phishing strategies like blacklisting and heuristic searches often have slow detection times and high false positive rates. The article introduces a novel feature selection method to extract highly correlated features from datasets, thereby enhancing classifier accuracy. Using six feature selection techniques on a phishing dataset, it evaluates eight classifiers, including SVM, Logistic Regression, Random Forest, and others. The study finds that the Random Forest classifier combined with the Chi-2 feature selection method significantly improves model accuracy, achieving up to 96.99%.

Prediction of glass transition temperature of oxide glasses based on interpretable machine learning and sparse data sets

The glass transition temperature (Tg) is a crucial characteristic of oxide glasses, exerting significant influence on their properties and applications. In this study, we utilized a sparse dataset and machine learning techniques to establish a predictive model for the relationship between the composition of oxide glasses and Tg. Among four machine learning algorithms compared, the Extreme Tree Regressor (ETR) algorithm demonstrated superior performance, exhibiting robust generalization capabilities on the validation set. We performed feature selection on the original features using Pearson Correlation Coefficient (PCC) and Maximal Information Coefficient (MIC), resulting in a subset of 38 dimensions. Subsequently, we employed Sparse Principal Component Analysis (PCA) and Recursive Feature Elimination (RFE) for dimensionality reduction, yielding a final subset of 12 dimensions. Introducing SHapley Additive exPlanations (SHAP) values for interpretability analysis of the predictive model, we obtained important feature rankings and analyzed how variations in features affect the target variable. Finally, using the CaO-SiO2-Al2O3 system as an example, we demonstrated how the model predicts Tg based on composition, facilitating rational design of oxide glass compositions.

Integrative deep learning with prior assisted feature selection.

Integrative analysis has emerged as a prominent tool in biomedical research, offering a solution to the "small and large " challenge. Leveraging the powerful capabilities of deep learning in extracting complex relationship between genes and diseases, our objective in this study is to incorporate deep learning into the framework of integrative analysis. Recognizing the redundancy within candidate features, we introduce a dedicated feature selection layer in the proposed integrative deep learning method. To further improve the performance of feature selection, the rich previous researches are utilized by an ensemble learning method to identify "prior information". This leads to the proposed prior assisted integrative deep learning (PANDA) method. We demonstrate the superiority of the PANDA method through a series of simulation studies, showing its clear advantages over competing approaches in both feature selection and outcome prediction. Finally, a skin cutaneous melanoma (SKCM) dataset is extensively analyzed by the PANDA method to show its practical application.

Solar Power Generation Forecast Using Multivariate Convolution Gated Recurrent Unit Network

For the advancement of smart grids, solar power generation predictions have become an important research topic. In the case of using traditional modeling methods, excessive computational costs may be incurred and it is difficult for these methods to learn the multi-variable dependencies of the data. Therefore, in this paper, a deep learning model was used to combine convolutional neural networks and long short-term memory recurrent network predictions. This method enables hourly power generation one day into the future. Convolutional neural networks are used to extract the features of multiple time series, while long short-term memory neural networks predict multivariate outcomes simultaneously. In order to obtain more accurate prediction results, we performed feature selection on meteorological features and combined the selected weather features to train the prediction model. We further distinguished sunny- and rainy-day models according to the predicted daily rainfall conditions. In the experiment, it was shown that the method of combining meteorological features further reduced the error. Finally, taking into account the differences in climate conditions between the northern and southern regions of Taiwan, the experimental results of case studies involving multiple regions were evaluated to verify the proposed method. The results showed that training combined with selected meteorological features can be widely used in regions with different climates in Taiwan.

Outcome-guided Bayesian clustering for disease subtype discovery using high-dimensional transcriptomic data

Due to the tremendous heterogeneity of disease manifestations, many complex diseases that were once thought to be single diseases are now considered to have disease subtypes. Disease subtyping analysis, that is the identification of subgroups of patients with similar characteristics, is the first step to accomplish precision medicine. With the advancement of high-throughput technologies, omics data offers unprecedented opportunity to reveal disease subtypes. As a result, unsupervised clustering analysis has been widely used for this purpose. Though promising, the subtypes obtained from traditional quantitative approaches may not always be clinically meaningful (i.e. correlate with clinical outcomes). On the other hand, the collection of rich clinical data in modern epidemiology studies has the great potential to facilitate the disease subtyping process via omics data and to discovery clinically meaningful disease subtypes. Thus, we developed an outcome-guided Bayesian clustering (GuidedBayesianClustering) method to fully integrate the clinical data and the high-dimensional omics data. A Gaussian mixed model framework was applied to perform sample clustering; a spike-and-slab prior was utilized to perform gene selection; a mixture model prior was employed to incorporate the guidance from a clinical outcome variable; and a decision framework was adopted to infer the false discovery rate of the selected genes. We deployed conjugate priors to facilitate efficient Gibbs sampling. Our proposed full Bayesian method is capable of simultaneously (i) obtaining sample clustering (disease subtype discovery); (ii) performing feature selection (select genes related to the disease subtype); and (iii) utilizing clinical outcome variable to guide the disease subtype discovery. The superior performance of the GuidedBayesianClustering was demonstrated through simulations and applications of breast cancer expression data and Alzheimer's disease. An R package has been made publicly available on GitHub to improve the applicability of our method.

A two-stage clonal selection algorithm for local feature selection on high-dimensional data

Various evolutionary computation algorithms have shown their excellent performance for high-dimensional feature selection (FS). However, most of current FS methods choose a global feature subset and ignore the correlations between feature subsets and sample subspaces, which limits the performance of methods in the sample space with different probability distributions. To solve the issue, we propose a two-stage clonal selection algorithm for filter-based local feature selection (TSCSA-LFS) that integrates symmetric uncertainty and a discrete clonal selection algorithm with three contributions. First, unlike conventional feature selection methods, TSCSA-LFS introduces local sample behaviors and assigns subsets of features for different sample regions. Second, an improved discrete clonal selection algorithm is developed for searching relevant features, which contains mutual information-based individual initialization, a differential evolution-based mutation strategy pool and a local search technique. Third, a two-part antibody representation is employed for automatical adjustment of the weight-related parameter. Our method shows the promising experimental results compared with well-known global FS and clonal selection-based local FS methods on fourteen high-dimensional datasets. For instance, our method can obviously outperform local FS, filter-based and hybrid methods on at least nine datasets

A deep learning method for classification of HNSCC and HPV patients using single-cell transcriptomics.

Head and Neck Squamous Cell Carcinoma (HNSCC) is the seventh most highly prevalent cancer type worldwide. Early detection of HNSCC is one of the important challenges in managing the treatment of the cancer patients. Existing techniques for detecting HNSCC are costly, expensive, and invasive in nature. In this study, we aimed to address this issue by developing classification models using machine learning and deep learning techniques, focusing on single-cell transcriptomics to distinguish between HNSCC and normal samples. Furthermore, we built models to classify HNSCC samples into HPV-positive (HPV+) and HPV-negative (HPV-) categories. In this study, we have used GSE181919 dataset, we have extracted 20 primary cancer (HNSCC) samples, and 9 normal tissues samples. The primary cancer samples contained 13 HPV- and 7 HPV+ samples. The models developed in this study have been trained on 80% of the dataset and validated on the remaining 20%. To develop an efficient model, we performed feature selection using mRMR method to shortlist a small number of genes from a plethora of genes. We also performed Gene Ontology (GO) enrichment analysis on the 100 shortlisted genes. Artificial Neural Network based model trained on 100 genes outperformed the other classifiers with an AUROC of 0.91 for HNSCC classification for the validation set. The same algorithm achieved an AUROC of 0.83 for the classification of HPV+ and HPV- patients on the validation set. In GO enrichment analysis, it was found that most genes were involved in binding and catalytic activities. A software package has been developed in Python which allows users to identify HNSCC in patients along with their HPV status. It is available at https://webs.iiitd.edu.in/raghava/hnscpred/.

FogNet: Custom CNN with optimal feature selection-based combat model for secured fog computing environment

In recent years, the increasing number of connected devices and the exponential growth of data generated by these devices have posed significant challenges for intrusion detection systems (IDS) in fog computing environments. However, conventional IDS failed to provide better attack detection performance in fog computing networks. So, this work proposes a custom convolutional neural network (CCNN) with optimal feature selection-based combat model for secured fog computing environment. Initially, preprocessing of the dataset aims to enhance the quality of the input data by removing attack labels, reducing dimensionality, and extracting relevant features. Then, comprehensive learning particle swarm-based effective seeker optimization (CLPS-ESO) is employed to extract features by leveraging a comprehensive learning strategy and the powerful optimization capabilities of effective seeker particle swarm optimization. To further enhance the feature selection performance, the random forest-dependent African buffalo optimization with weighted genetic algorithm, here after referred as RF-IABO-WGA, is utilized to identify the most discriminative features from the CLPS-ESO dataset. By selecting a subset of informative features, the computational complexity is reduced, and the detection accuracy is improved. Finally, the selected features are fed into a custom convolutional neural network (CCNN) classifier, which leverages the power of deep learning to accurately classify network traffic as either normal or malicious. The CCNN architecture can effectively learn complex patterns and capture the underlying characteristics of network data, enabling accurate intrusion detection in fog computing environments. Experimental results on a UNSW-NB dataset demonstrate that the proposed FogNet resulted in accuracy of 99.771878 %, precision of 99.770460 %, recall of 99.538589 %, and F1-score of 99.653512 %, which are superior performances compared to existing approaches.

Diagnosing the degree of differentiation between types of oral cancer based on extreme deep neural network model and Raman spectroscopy

As a highly prevalent and recurrent cancer, detecting the degree of differentiation in oral cancer is crucial. Current methods rely on biopsies in the presence of significant lesions, which are time-consuming. This study introduces an efficient and rapid approach for mass determination of oral cancer differentiation levels. By leveraging serum Raman spectroscopy combined with deep neural networks and extreme gradient boosting, we performed feature selection and interaction on oral cancer samples of varying differentiation levels, achieving the most accurate and reliable classification of oral cancer differentiation stages.