Tenfold CV Research Articles

PHisPred: a tool for the identification of histidine phosphorylation sites by integrating amino acid patterns and properties

BackgroundProtein histidine phosphorylation (pHis) plays critical roles in prokaryotic signal transduction pathways and various eukaryotic cellular processes. It is estimated to account for 6–10% of the phosphoproteome, however only hundreds of pHis sites have been discovered to date. Due to the inherent disadvantages of experimental methods, it is an urgent task for developing efficient computational approaches to identify pHis sites.ResultsHere, we present a novel tool, pHisPred, for accurately identifying pHis sites from protein sequences. We manually collected the largest number of experimental validated pHis sites to build benchmark datasets. Using randomized tenfold CV, the weighted SVM-RBF model shows the best performance than other four commonly used classification models (LR, KNN, RF, and MLP). From ten thousands of features, 140 and 150 most informative features were individually selected out for eukaryotic and prokaryotic models. The average AUC and F1-score values of pHisPred were (0.81, 0.40) and (0.78, 0.46) for tenfold CV on the eukaryotic and prokaryotic training datasets, respectively. In addition, pHisPred significantly outperforms other tools on testing datasets, in particular on the eukaryotic one.ConclusionWe implemented a python program of pHisPred, which is freely available for non-commercial use at https://github.com/xiaofengsong/pHisPred. Moreover, users can use it to train new models with their own data.

Decoding motor imagery with a simplified distributed dipoles model at source level.

Motor imagery (MI) based brain computer interface significantly oriented the development of neuro-rehabilitation, and the crucial issue is how to accurately detect the changes of cerebral cortex for MI decoding. The brain activity can be calculated based on the head model and observed scalp EEG, providing insights regarding cortical dynamics by using equivalent current dipoles with high spatial and temporal resolution. Now, all the dipoles within entire cortex or partial regions of interest are directly applied to data representation, this may make the key information weakened or lost, and it is worth studying how to choose the most important from numerous dipoles. In this paper, we devote to building a simplified distributed dipoles model (SDDM), which is combined with convolutional neural network (CNN), generating a MI decoding method at source level (called SDDM-CNN). First, all channels of raw MI-EEG signals are subdivided by a series of bandpass filters with width of 1Hz, the average energies associated with any sub-band signals are calculated and ranked in a descending order to screen the top n sub-bands; then, the MI-EEG signals over each selected sub-band are mapped into source space by using EEG source imaging technology, and for each scout of neuroanatomical Desikan-Killiany partition, a centered dipole is selected as the most relevant dipole and put together to build a SDDM to reflect the neuroelectric activity of entire cerebral cortex; finally, the 4 dimensional (4D) magnitude matrix is constructed for each SDDM and fused into a novel data representation, which is further input to a well-designed 3DCNN with n parallel branches (nB3DCNN) to extract and classify the comprehensive features from time-frequency-space dimensions. Experiments are carried out on three public datasets, and the average ten-fold CV decoding accuracies achieve 95.09%, 97.98% and 94.53% respectively, and the statistical analysis is fulfilled by standard deviation, kappa value and confusion matrix. Experiment results suggest that it is beneficial to pick out the most sensitive sub-bands in sensor domain, and SDDM can sufficiently describe the dynamic changing of entire cortex, improving decoding performance while greatly reducing number of source signals. Also, nB3DCNN is capable of exploring spatial-temporal features from multi sub-bands.

Spatial prediction of soil organic carbon stocks in an arid rangeland using machine learning algorithms.

Assessing the role of machine learning (ML) models concerning environmental predictors on spatial variation of soil organic carbon stocks (SOCS) in arid rangelands is very necessary. This study was conducted to explore the variability of surface SOCS in rangeland in the west of Iran using ML approaches. A number of 33 environmental predictors derived from Sentinel-2B and DEM were used. The optimal soil sampling (n = 80) position was determined by Latin hypercube sampling (cLHS) method. Robust and popular random Forest (RF), cubist (CB) along with random forest-ordinary kriging (RF-OK), and cubist-ordinary kriging (CB-OK) hybrid ML models were applied to the prediction of SOCS. Ten-fold CV was implemented for modeling performance and uncertainty map. According to data analysis, the maximum, minimum, and average values of SOCS are 44.50, 10.50, and 20.50 (ton. ha-1) at the surface depth (0-30cm), respectively. In general, normalized and standardized height covariates had a higher effect related to other predictors. On the other hand, two remote sensing (RS) indices, including salinity ratio (salinity) and GNDVI index, had a better impact on SOCS variability. The external validation of model performance indicated that RF-OK with (R2 = 0.75, RMSE = 6.33 ton. ha-1) with the high and low uncertainty range (3.33-9.50 ton. ha-1) was the outperformed ML model in compare with other models as RF (R2 = 0.65, RMSE = 7.38 ton. ha-1), CB-OK (R2 = 0.56, RMSE = 9.22 ton. ha-1), and CB (R2 = 0.33, RMSE = 10.42 ton. ha-1). In general, the hybrid models improved the accuracy of RF and CB with increased 0.11 until 0.23 of R2, and 1.05 to 1.2 (ton. ha-1) decreased RMSE of model's prediction. Hence, we conclude that the topographic attributes (especially normalized and standardized height) were the most critical factors in controlling surface SOCS in arid rangelands when combining with robust RF ML model, and optimized soil sampling methods like RF-cLHS can prepare acceptable soil properties maps.

RETRACTED: Breast cancer diagnosis using multiple activation deep neural network

Deep Neural Network (DNN) stands for multilayered Neural Network (NN) that is capable of progressively learn the more abstract and composite representations of the raw features of the input data received, with no need for any feature engineering. They are advanced NNs having repetitious hidden layers between the initial input and the final layer. The working principle of such a standard deep classifier is based on a hierarchy formed by the composition of linear functions and a defined nonlinear Activation Function (AF). It remains uncertain (not clear) how the DNN classifier can function so well. But it is clear from many studies that within DNN, the AF choice has a notable impact on the kinetics of training and the success of tasks. In the past few years, different AFs have been formulated. The choice of AF is still an area of active study. Hence, in this study, a novel deep Feed forward NN model with four AFs has been proposed for breast cancer classification: hidden layer 1: Swish, hidden layer, 2:-LeakyReLU, hidden layer 3: ReLU, and final output layer: naturally Sigmoidal. The purpose of the study is twofold. Firstly, this study is a step toward a more profound understanding of DNN with layer-wise different AFs. Secondly, research is also aimed to explore better DNN-based systems to build predictive models for breast cancer data with improved accuracy. Therefore, the benchmark UCI dataset WDBC was used for the validation of the framework and evaluated using a ten-fold CV method and various performance indicators. Multiple simulations and outcomes of the experimentations have shown that the proposed solution performs in a better way than the Sigmoid, ReLU, and LeakyReLU and Swish activation DNN in terms of different parameters. This analysis contributes to producing an expert and precise clinical dataset classification method for breast cancer. Furthermore, the model also achieved improved performance compared to many established state-of-the-art algorithms/models.

A novel decoding method for motor imagery tasks with 4D data representation and 3D convolutional neural networks

Objective. Motor imagery electroencephalography (MI-EEG) produces one of the most commonly used biosignals in intelligent rehabilitation systems. The newly developed 3D convolutional neural network (3DCNN) is gaining increasing attention for its ability to recognize MI tasks. The key to successful identification of movement intention is dependent on whether the data representation can faithfully reflect the cortical activity induced by MI. However, the present data representation, which is often generated from partial source signals with time-frequency analysis, contains incomplete information. Therefore, it would be beneficial to explore a new type of data representation using raw spatiotemporal dipole information as well as the possible development of a matching 3DCNN. Approach. Based on EEG source imaging and 3DCNN, a novel decoding method for identifying MI tasks is proposed, called ESICNND. MI-EEG is mapped to the cerebral cortex by the standardized low resolution electromagnetic tomography algorithm, and the optimal sampling points of the dipoles are selected as the time of interest to best reveal the difference between any two MI tasks. Then, the initial subject coordinate system is converted to a magnetic resonance imaging coordinate system, followed by dipole interpolation and volume down-sampling; the resulting 3D dipole amplitude matrices are merged at the selected sampling points to obtain 4D dipole feature matrices (4DDFMs). These matrices are augmented by sliding window technology and input into a 3DCNN with a cascading architecture of three modules (3M3DCNN) to perform the extraction and classification of comprehensive features. Main results. Experiments are carried out on two public datasets; the average ten-fold CV classification accuracies reach 88.73% and 96.25%, respectively, and the statistical analysis demonstrates outstanding consistency and stability. Significance. The 4DDFMs reveals the variation of cortical activation in a 3D spatial cube with a temporal dimension and matches the 3M3DCNN well, making full use of the high-resolution spatiotemporal information from all dipoles.

A novel computational model for predicting potential LncRNA-disease associations based on both direct and indirect features of LncRNA-disease pairs

BackgroundAccumulating evidence has demonstrated that long non-coding RNAs (lncRNAs) are closely associated with human diseases, and it is useful for the diagnosis and treatment of diseases to get the relationships between lncRNAs and diseases. Due to the high costs and time complexity of traditional bio-experiments, in recent years, more and more computational methods have been proposed by researchers to infer potential lncRNA-disease associations. However, there exist all kinds of limitations in these state-of-the-art prediction methods as well.ResultsIn this manuscript, a novel computational model named FVTLDA is proposed to infer potential lncRNA-disease associations. In FVTLDA, its major novelty lies in the integration of direct and indirect features related to lncRNA-disease associations such as the feature vectors of lncRNA-disease pairs and their corresponding association probability fractions, which guarantees that FVTLDA can be utilized to predict diseases without known related-lncRNAs and lncRNAs without known related-diseases. Moreover, FVTLDA neither relies solely on known lncRNA-disease nor requires any negative samples, which guarantee that it can infer potential lncRNA-disease associations more equitably and effectively than traditional state-of-the-art prediction methods. Additionally, to avoid the limitations of single model prediction techniques, we combine FVTLDA with the Multiple Linear Regression (MLR) and the Artificial Neural Network (ANN) for data analysis respectively. Simulation experiment results show that FVTLDA with MLR can achieve reliable AUCs of 0.8909, 0.8936 and 0.8970 in 5-Fold Cross Validation (fivefold CV), 10-Fold Cross Validation (tenfold CV) and Leave-One-Out Cross Validation (LOOCV), separately, while FVTLDA with ANN can achieve reliable AUCs of 0.8766, 0.8830 and 0.8807 in fivefold CV, tenfold CV, and LOOCV respectively. Furthermore, in case studies of gastric cancer, leukemia and lung cancer, experiment results show that there are 8, 8 and 8 out of top 10 candidate lncRNAs predicted by FVTLDA with MLR, and 8, 7 and 8 out of top 10 candidate lncRNAs predicted by FVTLDA with ANN, having been verified by recent literature. Comparing with the representative prediction model of KATZLDA, comparison results illustrate that FVTLDA with MLR and FVTLDA with ANN can achieve the average case study contrast scores of 0.8429 and 0.8515 respectively, which are both notably higher than the average case study contrast score of 0.6375 achieved by KATZLDA.ConclusionThe simulation results show that FVTLDA has good prediction performance, which is a good supplement to future bioinformatics research.

Improved cytokine\u2013receptor interaction prediction by exploiting the negative sample space

BackgroundCytokines act by binding to specific receptors in the plasma membrane of target cells. Knowledge of cytokine–receptor interaction (CRI) is very important for understanding the pathogenesis of various human diseases—notably autoimmune, inflammatory and infectious diseases—and identifying potential therapeutic targets. Recently, machine learning algorithms have been used to predict CRIs. “Gold Standard” negative datasets are still lacking and strong biases in negative datasets can significantly affect the training of learning algorithms and their evaluation. To mitigate the unrepresentativeness and bias inherent in the negative sample selection (non-interacting proteins), we propose a clustering-based approach for representative negative sample selection.ResultsWe used deep autoencoders to investigate the effect of different sampling approaches for non-interacting pairs on the training and the performance of machine learning classifiers. By using the anomaly detection capabilities of deep autoencoders we deduced the effects of different categories of negative samples on the training of learning algorithms. Random sampling for selecting non-interacting pairs results in either over- or under-representation of hard or easy to classify instances. When K-means based sampling of negative datasets is applied to mitigate the inadequacies of random sampling, random forest (RF) together with the combined feature set of atomic composition, physicochemical-2grams and two different representations of evolutionary information performs best. Average model performances based on leave-one-out cross validation (loocv) over ten different negative sample sets that each model was trained with, show that RF models significantly outperform the previous best CRI predictor in terms of accuracy (+ 5.1%), specificity (+ 13%), mcc (+ 0.1) and g-means value (+ 5.1). Evaluations using tenfold cv and training/testing splits confirm the competitive performance.ConclusionsA comparative analysis was performed to assess the effect of three different sampling methods (random, K-means and uniform sampling) on the training of learning algorithms using different evaluation methods. Models trained on K-means sampled datasets generally show a significantly improved performance compared to those trained on random selections—with RF seemingly benefiting most in our particular setting. Our findings on the sampling are highly relevant and apply to many applications of supervised learning approaches in bioinformatics.

A Method of Classification Performance Improvement Via a Strategy of Clustering-Based Data Elimination Integrated with k-Fold Cross-Validation

Non-system errors that occur during data entry or data collection create noisy data that reduce the success of classification systems. To eliminate this data, a classification system with a new data reduction method consisting of a modified k-means algorithm using relief algorithm coefficients named MKMA-RAC was developed. The main theme of this article is the elimination of noisy data and its consistent application to the classification system using the k-fold cross-validation method. By means of the developed system, the training data became free from noisy data by integrating the support vector machine, linear discriminant analysis (LDA) and decision tree classifiers with MKMA-RAC-based data reduction for every fold. The data reduction process was not applied for the test data. Datasets used in the proposed method were the Hepatitis, Liver Disorders, SPECT images and Statlog (Heart) dataset taken from the UCI database. Classification performance values obtained both from the proposed method and without the proposed method with tenfold CV were given for these datasets. For Hepatitis, Liver Disorders, SPECT images and Statlog (Heart) datasets, and classification successes of the proposed system with SVM classifier were 96.88%, 74.56%, 87.24%, and 90.00%, classification successes of the proposed system with LDA classifier were 94.91%, 69.05%, 82.38%, and 88.52%, classification successes of the proposed system with decision tree classifier were 96.25%, 77.73%, 88.77% and 89.63%, respectively. The test results have shown that the proposed system generally achieved higher classification performance than other literature results. Therefore, the performance is very encouraging for pattern recognition applications.

Comparative Study on Variable Selection Approaches in Establishment of Remote Sensing Model for Forest Biomass Estimation

In the field of quantitative remote sensing of forest biomass, a prominent phenomenon is the increasing number of explanatory variables. Then how to effectively select explanatory variables has become an important issue. Linear regression model is one of the commonly used remote sensing models. In the process of establishing the linear regression model, a vital step is to select explanatory variables. Focusing on variable selection and model stability, this paper conducts a comparative study on the performance of eight linear regression parameter estimation methods (Stepwise Regression Method (SR), Criterions Based on The Bayes Method (BIC), Criterions Based on The Bayes Method (AIC), Criterions Based on Prediction Error (Cp), Least Absolute Shrinkage and Selection Operator (Lasso), Adaptive Lasso, Smoothly Clipped Absolute Deviation (SCAD), Non-negative garrote (NNG)) in the subtropical forest biomass remote sensing model development. For the purpose of comparison, OLS and RR, are commonly used as methods with no variable selection ability, and are also compared and discussed. The performance of five aspects are evaluated in this paper: (i) Determination coefficient, prediction error, model error, etc., (ii) significance test about the difference between determination coefficients, (iii) parameter stability, (iv) variable selection stability and (v) variable selection ability of the methods. All the results are obtained through a five ten-fold CV. Some evaluation indexes are calculated with or without degrees of freedom. The results show that BIC performs best in comprehensive evaluation, while NNG, Cp and AIC perform poorly as a whole. Other methods show a great difference in the performance on each index. SR has a strong capability in variable selection, although it is poor in commonly used indexes. The short-wave infrared band and the texture features derived from it are selected most frequently by various methods, indicating that these variables play an important role in forest biomass estimation. Some of the conclusions in this paper are likely to change as the study object changes. The ultimate goal of this paper is to introduce various model establishment methods with variable selection capability, so that we can have more choices when establishing similar models, and we can know how to select the most appropriate and effective method for specific problems.

Estimation of personal PM2.5 and BC exposure by a modeling approach – Results of a panel study in Shanghai, China

BackgroundEpidemiologic studies of PM2.5 (particulate matter with aerodynamic diameter ≤2.5 μm) and black carbon (BC) typically use ambient measurements as exposure proxies given that individual measurement is infeasible among large populations. Failure to account for variation in exposure will bias epidemiologic study results. The ability of ambient measurement as a proxy of exposure in regions with heavy pollution is untested. ObjectiveWe aimed to investigate effects of potential determinants and to estimate PM2.5 and BC exposure by a modeling approach. MethodsWe collected 417 24 h personal PM2.5 and 130 72 h personal BC measurements from a panel of 36 nonsmoking college students in Shanghai, China. Each participant underwent 4 rounds of three consecutive 24-h sampling sessions through December 2014 to July 2015. We applied backwards regression to construct mixed effect models incorporating all accessible variables of ambient pollution, climate and time-location information for exposure prediction. All models were evaluated by marginal R2 and root mean square error (RMSE) from a leave-one-out-cross-validation (LOOCV) and a 10-fold cross-validation (10-fold CV). ResultsPersonal PM2.5 was 47.6% lower than ambient level, with mean (±Standard Deviation, SD) level of 39.9 (±32.1) μg/m3; whereas personal BC (6.1 (±2.8) μg/m3) was about one-fold higher than the corresponding ambient concentrations. Ambient levels were the most significant determinants of PM2.5 and BC exposure. Meteorological and season indicators were also important predictors. Our final models predicted 75% of the variance in 24 h personal PM2.5 and 72 h personal BC. LOOCV analysis showed an R2 (RMSE) of 0.73 (0.40) for PM2.5 and 0.66 (0.27) for BC. Ten-fold CV analysis showed a R2 (RMSE) of 0.73 (0.41) for PM2.5 and 0.68 (0.26) for BC. ConclusionWe used readily accessible data and established intuitive models that can predict PM2.5 and BC exposure. This modeling approach can be a feasible solution for PM exposure estimation in epidemiological studies.

An improved gSVM-SCADL2 with firefly algorithm for identification of informative genes and pathways

Incorporation of pathway knowledge into microarray analysis has been favoured by researchers owing to the improved biological interpretation of the analysis outcome. However, most of the pathway data are manually curated without specific biological context. Inclusion of non-informative genes in the analysis of context specific microarray data could lead to classifier with poor discriminative power. Thus, one of the main challenges is how to effectively identify informative genes from the pathway data. This paper proposes a firefly optimised penalised support vector machine with SCADL2 penalty function SVM-SCADL2-FFA in optimising tuning parameters for each pathway for efficient identification of informative genes and pathways. Experiments are done on lung cancer and gender data sets. Tenfold CV is used to evaluate the performance in terms of accuracy, specificity, sensitivity and F-score. The identified informative genes are validated through online databases. Our proposed method shows consistent improvements compared to previous works.

AN INCREMENTAL FRAMEWORK BASED ON CROSS-VALIDATION FOR ESTIMATING THE ARCHITECTURE OF A MULTILAYER PERCEPTRON

We define the problem of optimizing the architecture of a multilayer perceptron (MLP) as a state space search and propose the MOST (Multiple Operators using Statistical Tests) framework that incrementally modifies the structure and checks for improvement using cross-validation. We consider five variants that implement forward/backward search, using single/multiple operators and searching depth-first/breadth-first. On 44 classification and 30 regression datasets, we exhaustively search for the optimal and evaluate the goodness based on: (1) Order, the accuracy with respect to the optimal and (2) Rank, the computational complexity. We check for the effect of two resampling methods (5 × 2, ten-fold cv), four statistical tests (5 × 2 cv t, ten-fold cv t, Wilcoxon, sign) and two corrections for multiple comparisons (Bonferroni, Holm). We also compare with Dynamic Node Creation (DNC) and Cascade Correlation (CC). Our results show that: (1) On most datasets, networks with few hidden units are optimal, (2) forward searching finds simpler architectures, (3) variants using single node additions (deletions) generally stop early and get stuck in simple (complex) networks, (4) choosing the best of multiple operators finds networks closer to the optimal, (5) MOST variants generally find simpler networks having lower or comparable error rates than DNC and CC.