Test Set Accuracy Research Articles

Protein binding site prediction using an empirical scoring function

Most biological processes are mediated by interactions between proteins and their interacting partners including proteins, nucleic acids and small molecules. This work establishes a method called PINUP for binding site prediction of monomeric proteins. With only two weight parameters to optimize, PINUP produces not only 42.2% coverage of actual interfaces (percentage of correctly predicted interface residues in actual interface residues) but also 44.5% accuracy in predicted interfaces (percentage of correctly predicted interface residues in the predicted interface residues) in a cross validation using a 57-protein dataset. By comparison, the expected accuracy via random prediction (percentage of actual interface residues in surface residues) is only 15%. The binding sites of the 57-protein set are found to be easier to predict than that of an independent test set of 68 proteins. The average coverage and accuracy for this independent test set are 30.5 and 29.4%, respectively. The significant gain of PINUP over expected random prediction is attributed to (i) effective residue-energy score and accessible-surface-area-dependent interface-propensity, (ii) isolation of functional constraints contained in the conservation score from the structural constraints through the combination of residue-energy score (for structural constraints) and conservation score and (iii) a consensus region built on top-ranked initial patches.

Expression signature of p53 status (p53 signature) for the prediction of TP53 mutation and prognosis in breast cancer

652 Background: The current risk evaluation criteria, such as the St. Gallen and NIH, are not sufficient for predicting which BC patients are at high risk for recurrence and mortality. The TP53 mutations are associated with worse RFS and OS, independent other risk factors in BC, but the p53 status is not simply determined and often determined incorrectly (Soussi et al. Nat Rev Cancer 2006). Our purpose was to identify the gene sets that determine expression signature of the p53 status and to correlate the gene expression profile (GEP) with clinical outcome. Methods: Comprehensive expression analysis (Agilent 41K human genome oligo microarray) and DNA sequencing of the TP53 gene in 38 Japanese BC (30 Stage I-II, 8 stage III-IV; 20 TP53 wt, 20 TP53 mt) were performed using RNA and DNA obtained from the microdissected frozen tumor samples. We determined a gene set consisted of 33 genes from differentially expressed genes depending on the p53 status in the 26 BC (training set) and validated the ability to predict p53 status in the remaining 12 BC (test set). Prognostic value of the gene set was analyzed in 29 BC (stage I or II), and were validated using publicly available dataset of the independent 295 BC (stage I or II), van de Vijver et al. NEJM 2002). Results: The GEP using the 33 genes, many of which related in cell cycle and cell division, predicted the p53 status (wt or mt p53 signature) accurately in the test set (accuracy: 83%, overall accuracy: 95%). The p53 signature has the ability to predict RFS of the 29 early BC (Log Rank, P=0.032). It divided the 295 early BC into 176 wt and 119 mt group, and also has the ability to predict both RFS and OS (Log Rank, both P<0.0001). This was also true when the cohort were stratified by ER status, LN status, St. Gallen criteria and NIH criteria, and retained the predicted value in ER+, LN+, LN-, St. Gallen (intermediate/high) and NIH (high) subgroups (Log Rank, P<0.0001). Conclusions: The p53 signature is a powerful and independent predictor of the outcome of disease in early breast cancer than standard systems based on clinical and histologic criteria. No significant financial relationships to disclose.

Prediction of Milk/Plasma Drug Concentration (M/P) Ratio Using Support Vector Machine (SVM) Method

Development of reliable computational models to predict/classify milk-to-plasma (M/P) drug concentration ratio remains a challenging object. Support vector machine (SVM) method, as a new algorithm, was constructed to distinguish the potential risk of drugs to nursing infants. Each drug was represented by a large pool of descriptors, of which five were found to be most important for constructing the predictive models. Next, two classification models, linear discriminant analysis (LDA) and SVM, were developed with bootstrapping validation based on the selected molecular descriptors. The classification accuracy of training set and test set for SVM was 90.63 and 90.00%, respectively. The total accuracy for SVM was 90.48%, which was higher than that of LDA (77.78%). Comparison of the two methods shows that the performance of SVM was better than that of LDA, which implies that the SVM method is an effective tool in evaluating the risk of drugs when experimental M/P ratios have not been investigated.

Quantitative structure-activity relationship models for prediction of sensory irritants (log RD 50) of volatile organic chemicals

Quantitative classification and regression models for prediction of sensory irritants (log RD 50) of volatile organic chemicals (VOCs) have been developed. Each compound was represented by the calculated structural descriptors to encode constitutional, topological, geometrical, electrostatic, and quantum–chemical features. The heuristic method (HM) was then used to search the descriptor space and select the descriptors responsible for activity. The best classification results were found using support vector machine (SVM): the accuracy for training, test and overall data set is 96.5%, 85.7% and 94.4%, respectively. The nonlinear regression models were built by radial basis function neural networks (RNFNN) and SVM, respectively. The root mean squared errors (RMS) in prediction for the training, test and overall data set are 0.4755, 0.6322 and 0.5009 for reactive group, 0.2430, 0.4798 and 0.3064 for nonreactive group by RBFNN. The comparative results obtained by SVM are 0.4415, 0.7430 and 0.5140 for reactive group, 0.3920, 0.4520 and 0.4050 for nonreactive group, respectively. This paper proposes an effective method for poisonous chemicals screening and considering.

A Pilot Application of SELDI Mass Spectrometry To Study Proteomic Patterns in Plasma from Patients with T-Cell Large Granular Lymphocyte Leukemia.

T-cell large granular lymphocyte (T-LGL) leukemia is a rare chronic lymphoproliferative disorder of cytotoxic T cells of unknown etiology. Chronic antigenic stimulation of T-lymphocytes in autoimmune diseases or viral infection has been postulated to play a role in the pathogenesis of this condition. Surface-enhanced laser desorption/ionization (SELDI) mass spectrometry of plasma and serum specimens was applied as a useful screening method to determine pathophysiologic protein/peptide markers, which may be useful in the diagnosis of occult malignancies. In contrast to traditional diagnostic principles, the SELDI method enables screening of protein patterns without knowing the identity of targets. Patterns of up- and down-modulation of peptides and proteins (resulting from the presence of abnormal products of diseased cells, pathogens or immunogenetic factors) may signal various clinical features and diagnostic clues. We used SELDI mass spectrometry to study proteomic patterns in the plasma of patients with T-LGL leukemia. After initial screening and optimization we selected a hydrophobic H4 ProteinChip surface for further systematic screening. Deviation of mass values and peak intensities were <0.5% and <20%, respectively.Reproducibility was monitored by random duplication of samples with an average of 70 peaks per spectrum (mass/charge ratio from 1500–50,000 Da). Our study included plasma samples from 28 LGL patients, 9 patients with idiopathic neutropenia (without the presence of LGL clone) and 36 healthy blood donors. The diagnosis of T-LGL leukemia included the presence of monoclonal TCR rearrangement and flow cytometric detection of abnormal CD8+CD3+CD57+ cell population. Spectra were analyzed using Biomarker Patterns Software, which allows for internal cross validation for the prediction of the accuracy in a blinded test set. To separate LGL patients from patients with chronic neutropenia, intensity cut-off for 2050 Da peak was found in 22/28 LGL patients, resulting in 100% specificity and 67% sensitivity. Based on the intensities of 3 mass peaks (2274, 5870, 2660 Da), the sensitivity and specificity of LGL distinction from healthy controls was 96% and 61%, respectively. By direct digestion with carboxypeptidase Y on the ProteinChip surface, the C-terminal sequence of the 2050 Da mass peak has been identified. From the sequencing analysis and the search in the ExPASy protein database, we concluded that the 2050 Da mass peak corresponded to the fragment of complement C3 precursor. Additional 2 kDa mass peaks may represent various other C3 fragments. Interestingly, the intensity of the mass peak corresponding to the C3 fragments decreased with the application of immunosuppressive therapy when plasma samples from 3 LGL patients were serially analyzed. The pathophysiologic significance of the identified peak remains unclear, but increased expression of the C3 receptor on LGL cells has been reported. Our data in LGL leukemia indicate that SELDI mass spectrometry can be used to identify immune biomarkers for malignant lymphoproliferations.

Detection of Scab‐Damaged Hard Red Spring Wheat Kernels by Near‐Infrared Reflectance

ABSTRACTScab (Fusarium head blight) is a fungal disease that has become increasingly prevalent in North American wheat during the past 15 years. It is of concern to growers, processors, and the consumers because of depressed yields, poor flour quality, and the potential for elevated concentrations of the mycotoxin, deoxynivalenol (DON). Both wheat breeder and wheat inspector must currently deal with the assessment of scab in harvested wheat by manual human inspection. The study described herein examined the accuracy of a semi‐automated wheat scab inspection system that is based on near‐infrared (NIR) reflectance (1,000–1,700 nm) of individual kernels. Using statistical classification techniques such as linear discriminant analysis and nonparametric (k‐nearest‐neighbor) classification, upper limits of accuracy for NIR‐based classification schemes of ≈88% (cross‐validation) and 97% (test) were determined. An exhaustive search of the most suitable wavelength pairs for the spectral difference, [log(1/R)λ1 ‐ log(1/R)λ2], revealed that the slope of the low‐wavelength side of a broad carbohydrate absorption band (centered at ≈1,200 nm) was very effective at discriminating between healthy and scab‐damaged kernels with test set accuracies of 95%. The achieved accuracy levels demonstrate the potential for the use of NIR spectroscopy in commercial sorting and inspection operations for wheat scab.

Combinatorial QSAR of Ambergris Fragrance Compounds.

ChemInformVolume 35, Issue 25 Other Subjects Combinatorial QSAR of Ambergris Fragrance Compounds. Assia Kovatcheva, Assia Kovatcheva Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorAlexander Golbraikh, Alexander Golbraikh Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorScott Oloff, Scott Oloff Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorYun-De Xiao, Yun-De Xiao Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorWeifan Zheng, Weifan Zheng Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorPeter Wolschann, Peter Wolschann Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorGerhard Buchbauer, Gerhard Buchbauer Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorAlexander Tropsha, Alexander Tropsha Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this author Assia Kovatcheva, Assia Kovatcheva Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorAlexander Golbraikh, Alexander Golbraikh Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorScott Oloff, Scott Oloff Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorYun-De Xiao, Yun-De Xiao Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorWeifan Zheng, Weifan Zheng Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorPeter Wolschann, Peter Wolschann Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorGerhard Buchbauer, Gerhard Buchbauer Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this authorAlexander Tropsha, Alexander Tropsha Div. Med. Chem. Nat. Prod., Sch. Pharm., Univ. N. C., Chapel Hill, NC 27599, USASearch for more papers by this author First published: 02 June 2004 https://doi.org/10.1002/chin.200425232AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat No abstract is available for this article. Volume35, Issue25June 22, 2004 RelatedInformation

Singing Voice Identification Using Spectral Envelope Estimation

In this paper, we present a spectrum-based system for singer identification that operates for the ideal case in which audio samples contain only the singer's voice. Our method begins with the computation of a robust estimate of the spectral envelope called the composite transfer function (CTF). The CTF is derived from the instantaneous amplitude and frequency of the sinusoidal partials which make up the vocal signal. Unlike traditional source-filter theory , the CTF does not explicitly separate the spectral characteristics of the vocal source and the vocal tract filter. The principal components of the CTFs are used as features for a quadratic classifier to identify singers. The approach is validated on a database containing samples from twelve classically trained singers. In cross validation experiments, test set accuracies of approximately 95% are found for a baseline case. The classifier's performance is not degraded when different vowels are included in classifier training and evaluation. Restricting the frequency range of the CTFs and using a test set containing samples extracted from solo performances of Italian arias reduces the test set accuracy to 70-80%.

Combinatorial QSAR of ambergris fragrance compounds.

A combinatorial quantitative structure-activity relationships (Combi-QSAR) approach has been developed and applied to a data set of 98 ambergris fragrance compounds with complex stereochemistry. The Combi-QSAR approach explores all possible combinations of different independent descriptor collections and various individual correlation methods to obtain statistically significant models with high internal (for the training set) and external (for the test set) accuracy. Seven different descriptor collections were generated with commercially available MOE, CoMFA, CoMMA, Dragon, VolSurf, and MolconnZ programs; we also included chirality topological descriptors recently developed in our laboratory (Golbraikh, A.; Bonchev, D.; Tropsha, A. J. Chem. Inf. Comput. Sci. 2001, 41, 147-158). CoMMA descriptors were used in combination with MOE descriptors. MolconnZ descriptors were used in combination with chirality descriptors. Each descriptor collection was combined individually with four correlation methods, including k-nearest neighbors (kNN) classification, Support Vector Machines (SVM), decision trees, and binary QSAR, giving rise to 28 different types of QSAR models. Multiple diverse and representative training and test sets were generated by the divisions of the original data set in two. Each model with high values of leave-one-out cross-validated correct classification rate for the training set was subjected to extensive internal and external validation to avoid overfitting and achieve reliable predictive power. Two validation techniques were employed, i.e., the randomization of the target property (in this case, odor intensity) also known as the Y-randomization test and the assessment of external prediction accuracy using test sets. We demonstrate that not every combination of the data modeling technique and the descriptor collection yields a validated and predictive QSAR model. kNN classification in combination with CoMFA descriptors was found to be the best QSAR approach overall since predictive models with correct classification rates for both training and test sets of 0.7 and higher were obtained for all divisions of the ambergris data set into the training and test sets. Many predictive QSAR models were also found using a combination of kNN classification method with other collections of descriptors. The combinatorial QSAR affords automation, computational efficiency, and higher probability of identifying significant QSAR models for experimental data sets than the traditional approaches that rely on a single QSAR method.

High-Performance Commercial Data Mining: A Multistrategy Machine Learning Application

We present an application of inductive concept learning and interactive visualization techniques to a large-scale commercial data mining project. This paper focuses on design and configuration of high-level optimization systems (wrappers) for relevance determination and constructive induction, and on integrating these wrappers with elicited knowledge on attribute relevance and synthesis. In particular, we discuss decision support issues for the application (cost prediction for automobile insurance markets in several states) and report experiments using i>D2K, a Java-based visual programming system for data mining and information visualization, and several commercial and research tools. We describe exploratory clustering, descriptive statistics, and supervised decision tree learning in this application, focusing on a parallel genetic algorithm (GA) system, i>Jenesis, which is used to implement relevance determination (attribute subset selection). Deployed on several high-performance network-of-workstation systems (Beowulf clusters), i>Jenesis achieves a linear speedup, due to a high degree of task parallelism. Its test set accuracy is significantly higher than that of decision tree inducers alone and is comparable to that of the best extant search-space based wrappers.

NEURAL NETWORK WITH PRINCIPAL COMPONENT ANALYSIS FOR POULTRY CARCASS CLASSIFICATION

ABSTRACTThis paper reports the results of applying principal component analyses (PCA) of spectral reflectance data to reduce the number of input nodes for neural networks for classification of wholesome and unwholesome poultry carcasses. The results showed that the models with PCA pretreatment of input data performed better than those models without pretreatment. When sensing moving poultry carcasses in an environment without room light with a visible/near‐infrared spectrophotometer, the neural network classification models with PCA pretreatment achieved 100% accuracies for training, validating, and testing. For carcasses moving at 60 birds/min, 50 factors were required for perfect classification, while for 90 birds/min 30 factors were required. When sensing in room light, the best model was generated with 30 factors for a shackle speed of 60 birds/min, with a test set accuracy of 95.8%. For 90 birds/min, the best model with a test set accuracy of 96.8% was obtained when 15 factors were used. This study showed that PCA reduced the number of input nodes to the neural network classifiers and, in most cases, improved the model's classification accuracy. It also required fewer training samples and reduced training time.

Using percentage accuracy to measure neural network predictions in Stock Market movements

A speculator on a Stock Market, aside from having money to spare, needs at least one other thing — a means of producing accurate and understandable predictions ahead of others in the Market, so that a tactical and price advantage can be gained. This work demonstrates that it is possible to predict one such Market to a high degree of accuracy. For the purpose, the Financial Times — Stock Exchange (F.T.-S.E.) 100 Share Index in the UK, known as ‘The Footsie’, was selected. Neural network predictions were obtained for the daily Market close 5 days ahead, and 25 days ahead, as measured in mean square error and in root mean square error. To measure percentage accuracy, each individual test case prediction was compared with the actual market outcome, and total percentage accuracy for the whole test set was similarly calculated. Comparisons were also drawn with predictions for the same test cases using four types of Multiple Linear Regression. The neural network results indicated that predictions based upon the lowest mean square error bear little relationship to the same test cases, when measured in terms of overall percentage accuracy. For the lay person, or a Stock-Market speculator, it was also shown that predictions can be produced to a high level of accuracy, in a readily understandable format.

Refining PID Controllers Using Neural Networks

The KBANN (Knowledge-Based Artificial Neural Networks) approach uses neural networks to refine knowledge that can be written in the form of simple propositional rules. We extend this idea further by presenting the MANNCON (Multivariable Artificial Neural Network Control) algorithm by which the mathematical equations governing a PID (Proportional-Integral-Derivative) controller determine the topology and initial weights of a network, which is further trained using backpropagation. We apply this method to the task of controlling the outflow and temperature of a water tank, producing statistically significant gains in accuracy over both a standard neural network approach and a nonlearning PID controller. Furthermore, using the PID knowledge to initialize the weights of the network produces statistically less variation in test set accuracy when compared to networks initialized with small random numbers.

Total Parenteral Nutrition

<h3>Purpose</h3> To describe an artificial intelligence platform that detects thyroid eye disease (TED). <h3>Design</h3> Development of a deep learning model. <h3>Methods</h3> 1944 photographs from a clinical database were used to train a deep learning model. 344 additional images (‘test set’) were used to calculate performance metrics. Receiver operating characteristic, precision–recall curves and heatmaps were generated. From the test set, 50 images were randomly selected (‘survey set’) and used to compare model performance with ophthalmologist performance. 222 images obtained from a separate clinical database were used to assess model recall and to quantitate model performance with respect to disease stage and grade. <h3>Results</h3> The model achieved test set accuracy of 89.2%, specificity 86.9%, recall 93.4%, precision 79.7% and an F1 score of 86.0%. Heatmaps demonstrated that the model identified pixels corresponding to clinical features of TED. On the survey set, the ensemble model achieved accuracy, specificity, recall, precision and F1 score of 86%, 84%, 89%, 77% and 82%, respectively. 27 ophthalmologists achieved mean performance of 75%, 82%, 63%, 72% and 66%, respectively. On the second test set, the model achieved recall of 91.9%, with higher recall for moderate to severe (98.2%, n=55) and active disease (98.3%, n=60), as compared with mild (86.8%, n=68) or stable disease (85.7%, n=63). <h3>Conclusions</h3> The deep learning classifier is a novel approach to identify TED and is a first step in the development of tools to improve diagnostic accuracy and lower barriers to specialist evaluation.