Consensus-based Methods Research Articles

Evaluation of the Consensus of Four Peptide Identification Algorithms for Tandem Mass Spectrometry Based Proteomics

The availability of different scoring schemes and filter settings of protein database search algorithms has greatly expanded the number of search methods for identifying candidate peptides from MS/MS spectra. We have previously shown that consensus-based methods that combine three search algorithms yield higher sensitivity and specificity compared to the use of a single search engine (individual method). We hypothesized that union of four search engines (Sequest, Mascot, X!Tandem and Phenyx) can further enhance sensitivity and specificity. ROC plots were generated to measure the sensitivity and specificity of 5460 consensus methods derived from the same dataset. We found that Mascot outperformed individual methods for sensitivity and specificity, while Phenyx performed the worst. The union consensus methods generally produced much higher sensitivity, while the intersection consensus methods gave much higher specificity. The union methods from four search algorithms modestly improved sensitivity, but not specificity, compared to union methods that used three search engines. This suggests that a strategy based on specific combination of search algorithms, instead of merely 'as many search engines as possible', may be key strategy for success with peptide identification. Lastly, we provide strategies for optimizing sensitivity or specificity of peptide identification in MS/MS spectra for different user-specific conditions.

An empirically based tool for analyzing mortality associated with congenital heart surgery

Analysis of congenital heart surgery results requires a reliable method of estimating the risk of adverse outcomes. Two major systems in current use are based on projections of risk or complexity that were predominantly subjectively derived. Our goal was to create an objective, empirically based index that can be used to identify the statistically estimated risk of in-hospital mortality by procedure and to group procedures into risk categories. Mortality risk was estimated for 148 types of operative procedures using data from 77,294 operations entered into the European Association for Cardiothoracic Surgery (EACTS) Congenital Heart Surgery Database (33,360 operations) and the Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database (43,934 patients) between 2002 and 2007. Procedure-specific mortality rate estimates were calculated using a Bayesian model that adjusted for small denominators. Each procedure was assigned a numeric score (the STS-EACTS Congenital Heart Surgery Mortality Score [2009]) ranging from 0.1 to 5.0 based on the estimated mortality rate. Procedures were also sorted by increasing risk and grouped into 5 categories (the STS-EACTS Congenital Heart Surgery Mortality Categories [2009]) that were chosen to be optimal with respect to minimizing within-category variation and maximizing between-category variation. Model performance was subsequently assessed in an independent validation sample (n = 27,700) and compared with 2 existing methods: Risk Adjustment for Congenital Heart Surgery (RACHS-1) categories and Aristotle Basis Complexity scores. Estimated mortality rates ranged across procedure types from 0.3% (atrial septal defect repair with patch) to 29.8% (truncus plus interrupted aortic arch repair). The proposed STS-EACTS score and STS-EACTS categories demonstrated good discrimination for predicting mortality in the validation sample (C-index = 0.784 and 0.773, respectively). For procedures with more than 40 occurrences, the Pearson correlation coefficient between a procedure's STS-EACTS score and its actual mortality rate in the validation sample was 0.80. In the subset of procedures for which RACHS-1 and Aristotle Basic Complexity scores are defined, discrimination was highest for the STS-EACTS score (C-index = 0.787), followed by STS-EACTS categories (C-index = 0.778), RACHS-1 categories (C-index = 0.745), and Aristotle Basic Complexity scores (C-index = 0.687). When patient covariates were added to each model, the C-index improved: STS-EACTS score (C-index = 0.816), STS-EACTS categories (C-index = 0.812), RACHS-1 categories (C-index = 0.802), and Aristotle Basic Complexity scores (C-index = 0.795). The proposed risk scores and categories have a high degree of discrimination for predicting mortality and represent an improvement over existing consensus-based methods. Risk models incorporating these measures may be used to compare mortality outcomes across institutions with differing case mixes.

Evaluation of CASP8 model quality predictions

The model quality assessment problem consists in the a priori estimation of the overall and per-residue accuracy of protein structure predictions. Over the past years, a number of methods have been developed to address this issue and CASP established a prediction category to evaluate their performance in 2006. In 2008 the experiment was repeated and its results are reported here. Participants were invited to infer the correctness of the protein models submitted by the registered automatic servers. Estimates could apply to both whole models and individual amino acids. Groups involved in the tertiary structure prediction categories were also asked to assign local error estimates to each predicted residue in their own models and their results are also discussed here. The correlation between the predicted and observed correctness measures was the basis of the assessment of the results. We observe that consensus-based methods still perform significantly better than those accepting single models, similarly to what was concluded in the previous edition of the experiment.

QMEANclust: estimation of protein model quality by combining a composite scoring function with structural density information

BackgroundThe selection of the most accurate protein model from a set of alternatives is a crucial step in protein structure prediction both in template-based and ab initio approaches. Scoring functions have been developed which can either return a quality estimate for a single model or derive a score from the information contained in the ensemble of models for a given sequence. Local structural features occurring more frequently in the ensemble have a greater probability of being correct. Within the context of the CASP experiment, these so called consensus methods have been shown to perform considerably better in selecting good candidate models, but tend to fail if the best models are far from the dominant structural cluster. In this paper we show that model selection can be improved if both approaches are combined by pre-filtering the models used during the calculation of the structural consensus.ResultsOur recently published QMEAN composite scoring function has been improved by including an all-atom interaction potential term. The preliminary model ranking based on the new QMEAN score is used to select a subset of reliable models against which the structural consensus score is calculated. This scoring function called QMEANclust achieves a correlation coefficient of predicted quality score and GDT_TS of 0.9 averaged over the 98 CASP7 targets and perform significantly better in selecting good models from the ensemble of server models than any other groups participating in the quality estimation category of CASP7. Both scoring functions are also benchmarked on the MOULDER test set consisting of 20 target proteins each with 300 alternatives models generated by MODELLER. QMEAN outperforms all other tested scoring functions operating on individual models, while the consensus method QMEANclust only works properly on decoy sets containing a certain fraction of near-native conformations. We also present a local version of QMEAN for the per-residue estimation of model quality (QMEANlocal) and compare it to a new local consensus-based approach.ConclusionImproved model selection is obtained by using a composite scoring function operating on single models in order to enrich higher quality models which are subsequently used to calculate the structural consensus. The performance of consensus-based methods such as QMEANclust highly depends on the composition and quality of the model ensemble to be analysed. Therefore, performance estimates for consensus methods based on large meta-datasets (e.g. CASP) might overrate their applicability in more realistic modelling situations with smaller sets of models based on individual methods.

The treatment of aggression using arts therapies in forensic psychiatry: Results of a qualitative inquiry

The article describes the body of knowledge of arts therapies in forensic psychiatry based on recent practice, theory and research. The first part gives an overview of observational details, interventions, effects and rationales of drama therapy, music therapy, art therapy and dance-movement therapy in general and more specifically in the Netherlands. It shows that arts therapies can help to decrease recidivism. In the second part the results are presented of a qualitative naturalistic inquiry with 31 experienced arts therapists working in 12 institutions in the Netherlands and Germany. The arts therapists have been involved by means of semi-structured questionnaires, interviews and focus groups. Their implicit knowledge about indications, goals, interventions, effects and rationales have been compared and integrated into consensus-based treatment methods. The research reflects the Dutch tradition where all arts therapies are developed and researched within the same methodological formats. The results of one of the problem areas that have been researched, destructive aggression, are presented. Finally a comparison has been made between all arts therapies for the treatment of destructive aggression.

Using consensus methods to construct adaptive interfaces in multimodal web-based systems

This paper presents a concept of adaptive development of user interfaces in multimodal web-based systems. Today, it is crucial for general access web-based systems that the user interface is properly designed and adjusted to user needs and capabilities. It is believed that adaptive interfaces could offer a possible solution to this problem. Here, we introduce the notion of the user profile for classification, the interface profile for describing the system interface, and the compound usability measure for evaluation of the interface. Consensus-based methods are applied for constructing the interface profiles appropriate to classes of users.