Ten-fold Cross Validation Test Research Articles

Ensemble classification approach for screening of obstructive sleep apnoea using ECG

Sleep is essential for healthy human life. Obstructive Sleep Apnoea (OSA) is a commonly occurring sleep breathing disorder and manifests itself as pauses in nocturnal breathing. Screening is traditionally done at sleep labs, but is considered as labour intensive and costly procedure. Hence, reduced complexity methods for screening of OSA are necessary. Heart Rate Variability (HRV) is widely accepted as a characteristic of OSA. This paper proposes an ensemble learning based classifier for the detection of OSA through HRV. Sleep signals were acquired from benchmarked databases such as Apnoea-ECG and UCCD. Several linear and non-linear features that includes time domain, frequency domain and statistical features are utilised. Dimensionality reduction was done for deriving the optimal feature set. Ten-fold cross validation test was performed and a maximum accuracy of 94% was achieved in detecting OSA occurrences. The results of the proposed study are comparable to the gold standard of OSA screening.

Predicting the Presence of Learning Motivation in Electronic Learning: A New Rules to Predict

Research affirms that electronic learning (e-learning) has a great deal of advantages to learning process, it’s provides learners with flexibility in terms of time, place, and pace. That easiness was not be a guarantee of a good learning outcomes though they got the same learning material and course, because the fact is the outcomes was not the same from one to another and even not satisfy enough. One of prerequisite to the successful of e-learning process is the presence of learning motivation and it was not easy to identify. We propose a novel model to predicting the presence of learning motivation in e-learning using those attributes that have been identified in previous research. This model has been built using WEKA toolkit by comparing fourteen algorithms for Tree Classifier and ten-fold cross validation testing methods to process 3.200 of data sets. The best accuracy reached at 91.1% and identified four parameters to predict the presence of learning motivation in e-learning, and aim to assist teachers in identify whether student needs motivation or does not. This study also confirmed that Tree Classifier still has the best accuracy to predict and classify academic performance, it was reached average 90.48% for fourteen algorithm for accuracy value.

2D-SAR and 3D-QSAR analyses for acetylcholinesterase inhibitors.

Alzheimer's disease (AD) accounts for almost three quarters of dementia patients and interferes people's normal life. Great progress has been made recently in the study of Acetylcholinesterase (AChE), known as one of AD's biomarkers. In this study, acetylcholinesterase inhibitors (AChEI) were collected to build a two-dimensional structure-activity relationship (2D-SAR) model and three-dimensional quantitative structure-activity relationship (3D-QSAR) model based on feature selection method combined with random forest. After calculation, the prediction accuracy of the 2D-SAR model was 89.63% by using the tenfold cross-validation test and 87.27% for the independent test set. Three cutting ways were employed to build 3D-QSAR models. A model with the highest [Formula: see text] (cross-validated correlation coefficient) and [Formula: see text](non-cross-validated correlation coefficient) was obtained to predict AChEI activity. The mean absolute error (MAE) of the training set and the test set was 0.0689 and 0.5273, respectively. In addition, molecular docking was also employed to reveal that the ionization state of the compounds had an impact upon their interaction with AChE. Molecular docking results indicate that Ser124 might be one of the active site residues.

RESTORING TONE-MARKS IN STANDARD YORÙBÁ ELECTRONIC TEXT: IMPROVED MODEL

Diacritic Restoration is a necessity in the processing of languages with Latinbased scripts that utilizes letters outside the basic Latin alphabet used by English language. Yoruba is one such languages, marking underdot (dot-below)on three characters and tone marks on all seven vowels and two syllabic nasals. The problem of restoring underdotted characters has been fairly addressed using character as linguistic units for restoration. However, the existing characterbased approaches and word-based approach has not been able to sufficiently address restoration of tone marks in Yoruba. We address in this study tone marks restoration as a subset of diacritic restoration. We proposed using the syllable (derived from word) as the linguistic token for tone marks restoration. In our experimental setup, we used Yoruba text collected from various sources as data with total word count of 250,336 words. These words, on syllabification, yielded 464,274 syllables. The syllables were divided into training and testing data in different proportions ranging from 99% used for training and 1% used for testing to 70% used for training and 30% used for testing. The aim of evaluation different proportions was to determine how the ratio of training-to-test data affect the variations that may occur in the result. We applied Memory-based learning to train the models. We also set up a similar experiment using character token to be able to compare the performance. The result showed that using syllable was able to increase accuracy at word level to 96.23% and an average of almost 15% over that gotten from using character. We also found out that using 75% of data for training and the remaining 25% for testing gives the results with the least variation in a ten-fold cross validation test. Hybridizing the syllable „based approach with other methods like lexicon lookup might likely lead to improvement over the current result.

2D-QSAR and 3D-QSAR Analyses for EGFR Inhibitors.

Epidermal growth factor receptor (EGFR) is an important target for cancer therapy. In this study, EGFR inhibitors were investigated to build a two-dimensional quantitative structure-activity relationship (2D-QSAR) model and a three-dimensional quantitative structure-activity relationship (3D-QSAR) model. In the 2D-QSAR model, the support vector machine (SVM) classifier combined with the feature selection method was applied to predict whether a compound was an EGFR inhibitor. As a result, the prediction accuracy of the 2D-QSAR model was 98.99% by using tenfold cross-validation test and 97.67% by using independent set test. Then, in the 3D-QSAR model, the model with q2 = 0.565 (cross-validated correlation coefficient) and r2 = 0.888 (non-cross-validated correlation coefficient) was built to predict the activity of EGFR inhibitors. The mean absolute error (MAE) of the training set and test set was 0.308 log units and 0.526 log units, respectively. In addition, molecular docking was also employed to investigate the interaction between EGFR inhibitors and EGFR.

Comparative Analysis of Classification Algorithms on Endometrial Cancer Data

Objective: To expose the Performance of classification algorithms on endometrial cancer data. The best algorithms are listed based on the result of various test options and ranked based on their accuracies. Methods and Analysis: Classification is one of the data mining techniques used to find a model that describes the data classes or concepts. The class-label of strange instance is predicted with the help of classification. It compares the classification algorithms by measuring accuracies, speed and strength of algorithms using WEKA tool. Accuracies of classification algorithms are calculated by means of four different options. The error rate and time taken to build the model also measured. Findings: The accuracies of sixteen algorithms are measured by training set, test set, tenfold cross validation and percentage split testing options. The average accuracies are calculated, then compared and ranked with highest accuracy first. The best five algorithms are taken for final performance on endometrial cancer dataset. The accuracy of Random Forest algorithm is high, but it took 0.16 sec to build the model, whereas the IBK, Random Tree and KStar algorithms’ performs well with 0sec to build the model. Bagging algorithm takes more time to build the model. In terms of time and accuracy IBK produces better results as compared to other algorithms. Random Forest algorithm is most excellent in provisos of correctly classified occurrence. Novelty/Improvement: With the 315 instances of endometrial cancer data, the time taken to build the model is zero for IBK, KStar and Random Tree algorithms. If the number of instance increases then time also will increase.

PredPhos: an ensemble framework for structure-based prediction of phosphorylation sites.

BackgroundPost-translational modifications (PTMs) occur on almost all proteins and often strongly affect the functions of modified proteins. Phosphorylation is a crucial PTM mechanism with important regulatory functions in biological systems. Identifying the potential phosphorylation sites of a target protein may increase our understanding of the molecular processes in which it takes part.ResultsIn this paper, we propose PredPhos, a computational method that can accurately predict both kinase-specific and non-kinase-specific phosphorylation sites by using optimally selected properties. The optimal combination of features was selected from a set of 153 novel structural neighborhood properties by a two-step feature selection method consisting of a random forest algorithm and a sequential backward elimination method. To overcome the imbalanced problem, we adopt an ensemble method, which combines bootstrap resampling technique, support vector machine-based fusion classifiers and majority voting strategy. We evaluate the proposed method using both tenfold cross validation and independent test. Results show that our method achieves a significant improvement on the prediction performance for both kinase-specific and non-kinase-specific phosphorylation sites.ConclusionsThe experimental results demonstrate that the proposed method is quite effective in predicting phosphorylation sites. Promising results are derived from the new structural neighborhood properties, the novel way of feature selection, as well as the ensemble method.

SPAR: a random forest-based predictor for self-interacting proteins with fine-grained domain information

Protein self-interaction, i.e. the interaction between two or more identical proteins expressed by one gene, plays an important role in the regulation of cellular functions. Considering the limitations of experimental self-interaction identification, it is necessary to design specific bioinformatics tools for self-interacting protein (SIP) prediction from protein sequence information. In this study, we proposed an improved computational approach for SIP prediction, termed SPAR (Self-interacting Protein Analysis serveR). Firstly, we developed an improved encoding scheme named critical residues substitution (CRS), in which the fine-grained domain-domain interaction information was taken into account. Then, by employing the Random Forest algorithm, the performance of CRS was evaluated and compared with several other encoding schemes commonly used for sequence-based protein-protein interaction prediction. Through the tenfold cross-validation tests on a balanced training dataset, CRS performed the best, with the average accuracy up to 72.01%. We further integrated CRS with other encoding schemes and identified the most important features using the mRMR (the minimum redundancy maximum relevance) feature selection method. Our SPAR model with selected features achieved an average accuracy of 92.09% on the human-independent test set (the ratio of positives to negatives was about 1:11). Besides, we also evaluated the performance of SPAR on an independent yeast test set (the ratio of positives to negatives was about 1:8) and obtained an average accuracy of 76.96%. The results demonstrate that SPAR is capable of achieving a reasonable performance in cross-species application. The SPAR server is freely available for academic use at http://systbio.cau.edu.cn/zzdlab/spar/ .

An Approach for Predicting Essential Genes Using Multiple Homology Mapping and Machine Learning Algorithms.

Investigation of essential genes is significant to comprehend the minimal gene sets of cell and discover potential drug targets. In this study, a novel approach based on multiple homology mapping and machine learning method was introduced to predict essential genes. We focused on 25 bacteria which have characterized essential genes. The predictions yielded the highest area under receiver operating characteristic (ROC) curve (AUC) of 0.9716 through tenfold cross-validation test. Proper features were utilized to construct models to make predictions in distantly related bacteria. The accuracy of predictions was evaluated via the consistency of predictions and known essential genes of target species. The highest AUC of 0.9552 and average AUC of 0.8314 were achieved when making predictions across organisms. An independent dataset from Synechococcus elongatus, which was released recently, was obtained for further assessment of the performance of our model. The AUC score of predictions is 0.7855, which is higher than other methods. This research presents that features obtained by homology mapping uniquely can achieve quite great or even better results than those integrated features. Meanwhile, the work indicates that machine learning-based method can assign more efficient weight coefficients than using empirical formula based on biological knowledge.

Unsupervised learning assisted robust prediction of bioluminescent proteins

Bioluminescence plays an important role in nature, for example, it is used for intracellular chemical signalling in bacteria. It is also used as a useful reagent for various analytical research methods ranging from cellular imaging to gene expression analysis. However, identification and annotation of bioluminescent proteins is a difficult task as they share poor sequence similarities among them. In this paper, we present a novel approach for within-class and between-class balancing as well as diversifying of a training dataset by effectively combining unsupervised K-Means algorithm with Synthetic Minority Oversampling Technique (SMOTE) in order to achieve the true performance of the prediction model. Further, we experimented by varying different levels of balancing ratio of positive data to negative data in the training dataset in order to probe for an optimal class distribution which produces the best prediction accuracy. The appropriately balanced and diversified training set resulted in near complete learning with greater generalization on the blind test datasets. The obtained results strongly justify the fact that optimal class distribution with a high degree of diversity is an essential factor to achieve near perfect learning. Using random forest as the weak learners in boosting and training it on the optimally balanced and diversified training dataset, we achieved an overall accuracy of 95.3% on a tenfold cross validation test, and an accuracy of 91.7%, sensitivity of 89. 3% and specificity of 91.8% on a holdout test set. It is quite possible that the general framework discussed in the current work can be successfully applied to other biological datasets to deal with imbalance and incomplete learning problems effectively.

Prediction of O-glycosylation sites based on multi-scale composition of amino acids and feature selection.

Protein glycosylation is one of the most important and complex post-translational modification that provides greater proteomic diversity than any other post-translational modification. Fast and reliable computational methods to identify glycosylation sites are in great demand. Two key issues, feature encoding and feature selection, can critically affect the accuracy of a computational method. We present a new O-glycosylation sites prediction method using only amino acid sequence information. The method includes the following components: (1) on the basis of multi-scale theory, features based on multi-scale composition of amino acids were extracted from the training sequences with identified glycosylation sites; (2) perform a two-stage feature selection to remove features that had adverse effects on the prediction, including a stage one preliminary filtering with Student's t test, and a second stage screening through iterative elimination using novel pairwise comparisons conducted in random subspace using support vector machine. Important features retained are used to build prediction model. The method is evaluated with sequence-based tenfold cross-validation tests on balanced datasets. The results of our experiments show that our method significantly outperforms those reported in the literature in terms of sensitivity, specificity, accuracy, Matthew's correlation coefficient. The prediction accuracy of serine and threonine residues sites reached 95.7 and 92.7%. The Matthew correlation coefficient of our method for S and T sites is 0.914 and 0.873, respectively. This method can evaluate each feature with the interactions of the rest of the features, which are still included in the model and have the advantage of high efficiency.

Identification of human drug targets using machine-learning algorithms

Identification of potential drug targets is a crucial task in the drug-discovery pipeline. Successful identification of candidate drug targets in entire genomes is very useful, and computational prediction methods can speed up this process. In the current work we have developed a sequence-based prediction method for the successful identification and discrimination of human drug target proteins, from human non-drug target proteins. The training features include sequence-based features, such as amino acid composition, amino acid property group composition, and dipeptide composition for generating predictive models. The classification of human drug target proteins presents a classic example of class imbalance. We have addressed this issue by using SMOTE (Synthetic Minority Over-sampling Technique) as a preprocessing step, for balancing the training data with a ratio of 1:1 between drug targets (minority samples) and non-drug targets (majority samples). Using ensemble classification learning method-Rotation Forest and ReliefF feature-selection technique for selecting the optimal subset of salient features, the best model with selected features can achieve 87.1% sensitivity, 83.6% specificity, and 85.3% accuracy, with 0.71 Matthews correlation coefficient (mcc) on a tenfold stratified cross-validation test. The subset of identified optimal features may help in assessing the compositional patterns in human drug targets. For further validation, using a rigorous leave-one-out cross-validation test, the model achieved 88.1% sensitivity, 83.0% specificity, 85.5% accuracy, and 0.712 mcc. The proposed method was tested on a second dataset, for which the current pipeline gave promising results. We suggest that the present approach can be applied successfully as a complementary tool to existing methods for novel drug target prediction.

Combination use of protein-protein interaction network topological features improves the predictive scores of deleterious non-synonymous single-nucleotide polymorphisms.

Single-nucleotide polymorphisms (SNPs) are the most frequent form of genetic variations. Non-synonymous SNPs (nsSNPs) occurring in coding region result in single amino acid substitutions that associate with human hereditary diseases. Plenty of approaches were designed for distinguishing deleterious from neutral nsSNPs based on sequence level information. Novel in this work, combinations of protein-protein interaction (PPI) network topological features were introduced in predicting disease-related nsSNPs. Based on a dataset that was compiled from Swiss-Prot, a random forest model was constructed with an average accuracy value of 80.43% and an MCC value of 0.60 in a rigorous tenfold crossvalidation test. For an independent dataset, our model achieved an accuracy of 88.05% and an MCC of 0.67. Compared with previous studies, our approach presented superior prediction ability. Results showed that the incorporated PPI network topological features outperform conventional features. Our further analysis indicated that disease-related proteins are topologically different from other proteins. This study suggested that nsSNPs may share some topological information of proteins and the change of topological attributes could provide clues in illustrating functional shift due to nsSNPs.

Detecting concept mentions in biomedical text using hidden Markov model: multiple concept types at once or one at a time?

BackgroundIdentifying phrases that refer to particular concept types is a critical step in extracting information from documents. Provided with annotated documents as training data, supervised machine learning can automate this process. When building a machine learning model for this task, the model may be built to detect all types simultaneously (all-types-at-once) or it may be built for one or a few selected types at a time (one-type- or a-few-types-at-a-time). It is of interest to investigate which strategy yields better detection performance.ResultsHidden Markov models using the different strategies were evaluated on a clinical corpus annotated with three concept types (i2b2/VA corpus) and a biology literature corpus annotated with five concept types (JNLPBA corpus). Ten-fold cross-validation tests were conducted and the experimental results showed that models trained for multiple concept types consistently yielded better performance than those trained for a single concept type. F-scores observed for the former strategies were higher than those observed for the latter by 0.9 to 2.6% on the i2b2/VA corpus and 1.4 to 10.1% on the JNLPBA corpus, depending on the target concept types. Improved boundary detection and reduced type confusion were observed for the all-types-at-once strategy.ConclusionsThe current results suggest that detection of concept phrases could be improved by simultaneously tackling multiple concept types. This also suggests that we should annotate multiple concept types in developing a new corpus for machine learning models. Further investigation is expected to gain insights in the underlying mechanism to achieve good performance when multiple concept types are considered.

APSLAP: An Adaptive Boosting Technique for Predicting Subcellular Localization of Apoptosis Protein

Apoptotic proteins play key roles in understanding the mechanism of programmed cell death. Knowledge about the subcellular localization of apoptotic protein is constructive in understanding the mechanism of programmed cell death, determining the functional characterization of the protein, screening candidates in drug design, and selecting protein for relevant studies. It is also proclaimed that the information required for determining the subcellular localization of protein resides in their corresponding amino acid sequence. In this work, a new biological feature, class pattern frequency of physiochemical descriptor, was effectively used in accordance with the amino acid composition, protein similarity measure, CTD (composition, translation, and distribution) of physiochemical descriptors, and sequence similarity to predict the subcellular localization of apoptosis protein. AdaBoost with the weak learner as Random-Forest was designed for the five modules and prediction is made based on the weighted voting system. Bench mark dataset of 317 apoptosis proteins were subjected to prediction by our system and the accuracy was found to be 100.0 and 92.4%, and 90.1% for self-consistency test, jack-knife test, and tenfold cross validation test respectively, which is 0.9% higher than that of other existing methods. Beside this, the independent data (N151 and ZW98) set prediction resulted in the accuracy of 90.7 and 87.7%, respectively. These results show that the protein feature represented by a combined feature vector along with AdaBoost algorithm holds well in effective prediction of subcellular localization of apoptosis proteins. The user friendly web interface "APSLAP" has been constructed, which is freely available at http://apslap.bicpu.edu.in and it is anticipated that this tool will play a significant role in determining the specific role of apoptosis proteins with reliability.

Identifying Areas of the Visual Field Important for Quality of Life in Patients with Glaucoma

PurposeThe purpose of this study was to create a vision-related quality of life (VRQoL) prediction system to identify visual field (VF) test points associated with decreased VRQoL in patients with glaucoma.MethodVRQoL score was surveyed in 164 patients with glaucoma using the ‘Sumi questionnaire’. A binocular VF was created from monocular VFs by using the integrated VF (IVF) method. VRQoL score was predicted using the ‘Random Forest’ method, based on visual acuity (VA) of better and worse eyes (better-eye and worse-eye VA) and total deviation (TD) values from the IVF. For comparison, VRQoL scores were regressed (linear regression) against: (i) mean of TD (IVF MD); (ii) better-eye VA; (iii) worse-eye VA; and (iv) IVF MD and better- and worse-eye VAs. The rank of importance of IVF test points was identified using the Random Forest method.ResultsThe root mean of squared prediction error associated with the Random Forest method (0.30 to 1.97) was significantly smaller than those with linear regression models (0.34 to 3.38, p<0.05, ten-fold cross validation test). Worse-eye VA was the most important variable in all VRQoL tasks. In general, important VF test points were concentrated along the horizontal meridian. Particular areas of the IVF were important for different tasks: peripheral superior and inferior areas in the left hemifield for the ‘letters and sentences’ task, peripheral, mid-peripheral and para-central inferior regions for the ‘walking’ task, the peripheral superior region for the ‘going out’ task, and a broad scattered area across the IVF for the ‘dining’ task.ConclusionThe VRQoL prediction model with the Random Forest method enables clinicians to better understand patients’ VRQoL based on standard clinical measurements of VA and VF.

A cost-sensitive extension of AdaBoost with markov random field priors for automated segmentation of breast tumors in ultrasonic images

A cost-sensitive extension of AdaBoost based on Markov random field (MRF) priors was developed to train an ensemble segmentation process which can avoid irregular shape, isolated points and holes, leading to lower error rate. The method was applied to breast tumor segmentation in ultrasonic images. A cost function was introduced into the AdaBoost algorithm that penalizes dissimilar adjacent labels in MRF regularization. The extended AdaBoost algorithm generates a series of weak segmentation processes by sequentially selecting a process whose error rate weighted by the cost is minimum. The method was tested by generation of an ensemble segmentation process for breast tumors in ultrasonic images. This was followed by a active contour to refine the extracted tumor boundary. The segmentation performance was evaluated by tenfold cross validation test, where 300 carcinomas, 50 fibroadenomas, and 50 cysts were used. The experimental results revealed that the error rate of the proposed ensemble segmentation was two-thirds the error rate of the segmentation trained by AdaBoost without MRF. By combining the ensemble segmentation with a geodesic active contour, the average Jaccard index between the extracted tumors and the manually segmented true regions was 93.41%, significantly higher than the conventional segmentation process. A cost-sensitive extension of AdaBoost based on MRF priors provides an efficient and accurate means for the segmentation of tumors in breast ultrasound images.

Voting with a parameterized veto strategy

This paper presents our winner solution for the KDD Cup 2006 problem. It is based on the results of three different supervised learning techniques which are then combined in a classifier committee, and finally a single solution is obtained with a voting procedure. The voting procedure assigns weights to each member of the committee according to their average performance on a ten-fold cross-validation test and it also takes into account the confidence values returned by the three algorithms. The final decision of the committee is determined by means of a parameterized veto strategy, which takes into consideration the maximal allowed error rate beside the confidence values of the committee members. The solution presented here won Task 2 and became runner-up at Task 1 in the competition.

Gene Recognition from Questionable ORFs in Bacterial and Archaeal Genomes

The ORFs of microbial genomes in annotation files are usually classified into two groups: the first corresponds to known genes; whereas the second includes ‘putative’, ‘probable’, ‘conserved hypothetical’, ‘hypothetical’, ‘unknown’ and ‘predicted’ ORFs etc. Since the annotation is not 100% accurate, it is essential to confirm which ORF of the latter group is coding and which is not. Starting from known genes in the former, this paper describes an improved Z curve method to recognize genes in the latter. Ten-fold cross-validation tests show that the average accuracy of the algorithm is greater than 99% for recognizing the known genes in 57 bacterial and archaeal genomes. The method is then applied to recognize genes of the latter group. The likely non-coding ORFs in each of the 57 bacterial or archaeal genomes studied here are recognized and listed at the website http://tubic.tju.edu.cn/ZCURVE_C_html/non-coding.html. The working mechanism of the algorithm has been discussed in details. A computer program, called ZCURVE_C, was written to calculate a coding score called Z-curve score for ORFs in the above 57 bacterial and archaeal genomes. Coding/non-coding is simply determined by the criterion of Z-curve score > 0/Z-curve score lt; 0. A website has been set up to provide the service to calculate the Z-curve score. A user may submit the DNA sequence of an ORF to the server at http://tubic.tju.edu.cn/ZCURVE_C/Default.cgi, and the Z-curve score of the ORF is calculated and returned to the user immediately.