Sequential Backward Selection Method Research Articles

Interpretable Machine Learning Prediction of Voltage and Specific Capacity for Electrode Materials

AbstractThe application of machine learning (ML) is becoming widespread and playing an important role in the property prediction and design of battery. Average voltage and specific capacity are two important indicators of electrode materials, which determine suitability of electrode materials in batteries. In this study, 4351 data are collected from Material Project and 324 features are extracted from Material Project or calculated by Matminer. Four ML models are discussed for prediction of average voltage and specific capacity. In order to ensure the prediction accuracy and efficiency of ML models, sequential backward selection (SBS) method is introduced to select optimal feature set, which can reduce redundancy features. The combination of Deep Neural Network model and optimal feature set selected by the SBS method achieves accurate prediction and outperforms Crystal Graph Convolutional Neural Network and Graph‐attention Graph Neural Network. The interpretability analysis provides insight into the relationship between features and two target properties. To cope with the poor prediction performance for metal‐ion battery with scarce data, transfer learning is adopted and an excellent improvement is achieved in the prediction in Na‐ion, Mg‐ion and Ca‐ion electrode material. It can be concluded that ML is an effective approach to battery property prediction and design.

Efficient prediction of SOC and aggregate OC components by continuous wavelet transform spectra under different feature selection methods

Soil Organic Carbon (SOC) fixation plays a vital role in reducing greenhouse gas emissions and the formation of soil aggregates is recognized as a crucial process in SOC fixation. In the current study, we characterized spectral information on SOC and aggregate Organic carbon (OC) components by using a rapid and robust method. Soil samples were collected from different layers to obtain the SOC and aggregate OC components to record corresponding spectra. The Continuous Wavelet Transform (CWT) spectra were then combined with various characteristic wavelength selection methods to build Multivariate Linear Regression (MLR), BP neural network (BP), and Random Forest (RF) models. Finally, the optimal combination of characteristic wavelengths based on the best-performing model for each index was determined. The results demonstrated that the model's accuracy, which was built using the set of characteristic wavelengths of the best model for all metrics, was higher than that of the best model for a single metric. This was due to the correlation between SOC and aggregate OC components. The best set of characteristic wavelengths were those selected by the Sequential Backward Selection method with wavelet decomposition scales of 23, 26, and 27, and those selected by the Minimum Redundancy Maximum Relevance method with wavelet decomposition scale 27. The RF model for Free silt and clay particles organic carbon (FSP-OC) showed the highest accuracy among the four SOC prediction models. In addition, it was discovered that as the wavelet decomposition scales become more similar, the extracted characteristic wavelengths also become more similar, resulting in similar model predictions. In summary, the accuracy of the predictive model for SOC and its constituents can be demonstrably improved by reasonable combination of characteristic wavelengths selection method and wavelet decomposition scale.

Unraveling the mechanisms underlying drug-induced cholestatic liver injury: identifying key genes using machine learning techniques on human in vitro data sets

Drug-induced intrahepatic cholestasis (DIC) is a main type of hepatic toxicity that is challenging to predict in early drug development stages. Preclinical animal studies often fail to detect DIC in humans. In vitro toxicogenomics assays using human liver cells have become a practical approach to predict human-relevant DIC. The present study was set up to identify transcriptomic signatures of DIC by applying machine learning algorithms to the Open TG-GATEs database. A total of nine DIC compounds and nine non-DIC compounds were selected, and supervised classification algorithms were applied to develop prediction models using differentially expressed features. Feature selection techniques identified 13 genes that achieved optimal prediction performance using logistic regression combined with a sequential backward selection method. The internal validation of the best-performing model showed accuracy of 0.958, sensitivity of 0.941, specificity of 0.978, and F1-score of 0.956. Applying the model to an external validation set resulted in an average prediction accuracy of 0.71. The identified genes were mechanistically linked to the adverse outcome pathway network of DIC, providing insights into cellular and molecular processes during response to chemical toxicity. Our findings provide valuable insights into toxicological responses and enhance the predictive accuracy of DIC prediction, thereby advancing the application of transcriptome profiling in designing new approach methodologies for hazard identification.

Autonomic nervous pattern analysis of sleep deprivation

The cumulative long-term effects of sleep deprivation (SD) have been associated with a wide range of deleterious health consequences. In order to detect SD with machine learning methods, this paper analyzed the autonomic nervous patterns (ANP) of the normal sleep (NS) populations and those of the acute sleep deprivation (ASD) individuals in the morning. Besides, we also analyzed the ANP of people before sleep deprivation (BSD) and during sleep deprivation (DSD) at night. Electrocardiogram (ECG) data were acquired from 120 subjects. Twenty-five heart rate variability (HRV) features were extracted for statistical analysis and pattern recognition. Sequential backward selection method was applied to select the critical feature subset of SD pattern recognition. Three classical classifiers were trained with BSD and DSD samples, and their performance were validated and compared with each other, in order to obtain the best parameters of the SD recognition models. The results of statistical analysis showed that the ANP of NS and ASD groups were significantly different. Besides, the ANP of NS and ASD showed significant within-group sex difference. The results of SD pattern recognition showed that, with 3-dimension critical HRV features of the night ECG data, support vector machine classifier obtained validation accuracies of 76.92% for males and 83.02% for females. It is worth noting that, the low-frequency power of RR interval series increases with the increase of SD time, showing that SD leads to persistent activation of sympathetic nervous system.

Diagnosing Parkinson’s disease by means of ensemble classification of patients’ voice samples

This paper proposes a method for diagnosing Parkinson’s disease using ensemble classification of patient voice samples. Conducted research concerned testing the parameters of the ensemble of classifiers, in terms of types and numbers of classifiers included in it. More than a dozen popular classifiers were considered in the study. Additionally, for each of the tested classifiers, a set of features of voice samples were selected, for which a given classifier showed the highest classification efficiency. The Sequential Backward Selection (SBS), which belongs to the wrapper methods, was used for feature selection. The ensemble of classifiers was then tested in two cases, with all features considered and including only those indicated by the SBS method. The obtained results were compared with each other.All experiments were performed on a publicly available database containing voice samples of Parkinson’s patients and healthy patients. This database can be found in the UCI archives.

Relevant SMS Spam Feature Selection Using Wrapper Approach and XGBoost Algorithm

In recent years with the widely usage of mobile devices, the problem of SMS Spam increased dramatically. Receiving those undesired messages continuously can cause frustration to users. And sometimes it can be harmful, by sending SMS messages containing fake web pages in order to steal users’ confidential information. Besides spasm number of hazardous actions, there is a limited number of spam filtering software. According to this paper, XGBoost algorithm used for handling SMS spam detection problem. Number of structural features was collected from previous studies. 15 structural features were extracted from Tiago’s dataset, which is the most frequently used dataset by researchers. For selecting the optimal relevant features, two different types of wrapper feature selection algorithms were used in order to reduce and select best relevant features. The accuracy and performance obtained by the selected features via sequential backward selection method was better comparing to sequential forward selection method. The extracted nine optimal features can be a good representation of a spam SMS message. Additionally, the classification accuracy obtained by the proposed method using nine optimal features with XGBoost algorithm is 98.64 using 10-fold cross validation.

Embedded binary PSO integrating classical methods for multilevel improved feature selection in liver and kidney disease diagnosis

Feature selection is an important pre-processing technique in the field of data mining. This process removes irrelevant data thereby reduces the number of features. This paper presents a new algorithm called embedded binary particle swarm optimisation (BPSO) to improve the performance of BPSO for feature selection. Embedded BPSO incorporates classical methods to select elite feature subset. The population is refined or extended at regular intervals using the best features from sequential forward selection and sequential backward selection methods. In this study, probabilistic neural network and support vector machine with threefold cross-validation are used to evaluate the particles. The embedded algorithm is verified in the feature selection module of the liver and kidney cancer diagnostic system. The elite features extracted from wrapper based embedded algorithm are used to characterise diseases using the classifier. Findings show that the proposed system is proficient in selecting the best features with minimum error rate.

Computational identification of deleterious synonymous variants in human genomes using a feature-based approach

BackgroundAlthough synonymous single nucleotide variants (sSNVs) do not alter the protein sequences, they have been shown to play an important role in human disease. Distinguishing pathogenic sSNVs from neutral ones is challenging because pathogenic sSNVs tend to have low prevalence. Although many methods have been developed for predicting the functional impact of single nucleotide variants, only a few have been specifically designed for identifying pathogenic sSNVs.ResultsIn this work, we describe a computational model, IDSV (Identification of Deleterious Synonymous Variants), which uses random forest (RF) to detect deleterious sSNVs in human genomes. We systematically investigate a total of 74 multifaceted features across seven categories: splicing, conservation, codon usage, sequence, pre-mRNA folding energy, translation efficiency, and function regions annotation features. Then, to remove redundant and irrelevant features and improve the prediction performance, feature selection is employed using the sequential backward selection method. Based on the optimized 10 features, a RF classifier is developed to identify deleterious sSNVs. The results on benchmark datasets show that IDSV outperforms other state-of-the-art methods in identifying sSNVs that are pathogenic.ConclusionsWe have developed an efficient feature-based prediction approach (IDSV) for deleterious sSNVs by using a wide variety of features. Among all the features, a compact and useful feature subset that has an important implication for identifying deleterious sSNVs is identified. Our results indicate that besides splicing and conservation features, a new translation efficiency feature is also an informative feature for identifying deleterious sSNVs. While the function regions annotation and sequence features are weakly informative, they may have the ability to discriminate deleterious sSNVs from benign ones when combined with other features. The data and source code are available on website http://bioinfo.ahu.edu.cn:8080/IDSV.

Variable selection for Fisher linear discriminant analysis using the modified sequential backward selection algorithm for the microarray data

One of the major challenges is small sample size as compared to large features number for microarray data. Variable selection is an important step for improving diagnostics of cancer or the classification according to the phenotypes via gene expression data. In this study, we propose a modified sequential backward selection (SBS) algorithm to deal with the case where the covariance matrix is singular. Then we propose a variable selection algorithm based on the weighted Mahalanobis distance and modified SBS methods. Furthermore, based on the proposed variable selection algorithm, a Fisher linear discriminant method is proposed to improve the accuracy of tumor classification through simultaneously taking into account genes’ joint discriminatory power. To validate the efficiency, we apply the proposed discriminant method to two different DNA microarray data sets for experiment investigation. The empirical results show that our method for tumor classification can obtain better classification effectiveness than Markov random field method and independent variable group analysis I methods, which demonstrates that the proposed variable selection method can obtain more correct and informative gene subset if taking into account the joint discriminatory power of genes for tumor classification.

Neuromuscular disease classification system

Diagnosis of neuromuscular diseases is based on subjective visual assessment of biopsies from patients by the pathologist specialist. A system for objective analysis and classification of muscular dystrophies and neurogenic atrophies through muscle biopsy images of fluorescence microscopy is presented. The procedure starts with an accurate segmentation of the muscle fibers using mathematical morphology and a watershed transform. A feature extraction step is carried out in two parts: 24 features that pathologists take into account to diagnose the diseases and 58 structural features that the human eye cannot see, based on the assumption that the biopsy is considered as a graph, where the nodes are represented by each fiber, and two nodes are connected if two fibers are adjacent. A feature selection using sequential forward selection and sequential backward selection methods, a classification using a Fuzzy ARTMAP neural network, and a study of grading the severity are performed on these two sets of features. A database consisting of 91 images was used: 71 images for the training step and 20 as the test. A classification error of 0% was obtained. It is concluded that the addition of features undetectable by the human visual inspection improves the categorization of atrophic patterns.

Swfoldrate: Predicting protein folding rates from amino acid sequence with sliding window method

Protein folding is the process by which a protein processes from its denatured state to its specific biologically active conformation. Understanding the relationship between sequences and the folding rates of proteins remains an important challenge. Most previous methods of predicting protein folding rate require the tertiary structure of a protein as an input. In this study, the long-range and short-range contact in protein were used to derive extended version of the pseudo amino acid composition based on sliding window method. This method is capable of predicting the protein folding rates just from the amino acid sequence without the aid of any structural class information. We systematically studied the contributions of individual features to folding rate prediction. The optimal feature selection procedures are adopted by means of combining the forward feature selection and sequential backward selection method. Using the jackknife cross validation test, the method was demonstrated on the large dataset. The predictor was achieved on the basis of multitudinous physicochemical features and statistical features from protein using nonlinear support vector machine (SVM) regression model, the method obtained an excellent agreement between predicted and experimentally observed folding rates of proteins. The correlation coefficient is 0.9313 and the standard error is 2.2692. The prediction server is freely available at http://www.jci-bioinfo.cn/swfrate/input.jsp.

A CAD system for the automatic detection of clustered microcalcifications in digitized mammogram films.

Clusters of microcalcifications in mammograms are an important early sign of breast cancer. This paper presents a computer-aided diagnosis (CAD) system for the automatic detection of clustered microcalcifications in digitized mammograms. The proposed system consists of two main steps. First, potential microcalcification pixels in the mammograms are segmented out by using mixed features consisting of wavelet features and gray level statistical features, and labeled into potential individual microcalcification objects by their spatial connectivity. Second, individual microcalcifications are detected by using a set of 31 features extracted from the potential individual microcalcification objects. The discriminatory power of these features is analyzed using general regression neural networks via sequential forward and sequential backward selection methods. The classifiers used in these two steps are both multilayer feedforward neural networks. The method is applied to a database of 40 mammograms (Nijmegen database) containing 105 clusters of microcalcifications. A free-response operating characteristics (FROC) curve is used to evaluate the performance. Results show that the proposed system gives quite satisfactory detection performance. In particular, a 90% mean true positive detection rate is achieved at the cost of 0.5 false positive per image when mixed features are used in the first step and 15 features selected by the sequential backward selection method are used in the second step. However, we must be cautious when interpreting the results, since the 20 training samples are also used in the testing step.

Designing classifier fusion systems by genetic algorithms

We suggest two simple ways to use a genetic algorithm (GA) to design a multiple-classifier system. The first GA version selects disjoint feature subsets to be used by the individual classifiers, whereas the second version selects (possibly) overlapping feature subsets, and also the types of the individual classifiers. The two GAs have been tested with four real data sets: heart, Satimage, letters, and forensic glasses. We used three-classifier systems and basic types of individual classifiers (the linear and quadratic discriminant classifiers and the logistic classifier). The multiple-classifier systems designed with the two GAs were compared against classifiers using: all features; the best feature subset found by the sequential backward selection method; and the best feature subset found by a CA. The GA design can be made less prone to overtraining by including penalty terms in the fitness function accounting for the number of features used.