Gold-standard Benchmark Research Articles

Automatic Classification of Protein Structure Using the Maximum Contact Map Overlap Metric

In this work, we propose a new distance measure for comparing two protein structures based on their contact map representations. We show that our novel measure, which we refer to as the maximum contact map overlap (max-CMO) metric, satisfies all properties of a metric on the space of protein representations. Having a metric in that space allows one to avoid pairwise comparisons on the entire database and, thus, to significantly accelerate exploring the protein space compared to no-metric spaces. We show on a gold standard superfamily classification benchmark set of 6759 proteins that our exact k-nearest neighbor (k-NN) scheme classifies up to 224 out of 236 queries correctly and on a larger, extended version of the benchmark with 60; 850 additional structures, up to 1361 out of 1369 queries. Our k-NN classification thus provides a promising approach for the automatic classification of protein structures based on flexible contact map overlap alignments.

FV-137 candidate drug: Novel and selective ion channel modulation for epilepsy and pain

Broad spectrum Anti-Epileptic Drugs (AED) are valued for their potential to treat refractory epilepsy and as general ‘neurostabilizers’ have been employed for a variety of neurological diseases including migrane, biopolar disorder and neuropathic pain. However, the benefits of historical AEDs have been constrained by unintended CNS side effects of these non-selective medicines. Combining proprietary breakthrough chemistry with Fluorinov Pharma and NIH’s in vivo phenotypic screening strategy identified three completely novel candidate drugs, including FV-137 with superior safety and efficacy profiles targeting selective ion channels. FV-137 is a novel, orally administered AED, structurally distinct than any known AED, displaying promising preclinical efficacy and safety. FV-137 emerged from a program whose goal was to identify broad-spectrum anti-epileptic drugs (AED) using gold standard benchmarks such as Depakote ® , Neurontin ® and Keppra ® as primary comparators. FV-137 was extensively screened across all available in vitro and rodent seizure models at ASP, and was found to display an excellent spectrum of anti-seizure activity and safety profiles superior to that produced by Depakote, Neurontin and Keppra. FV-137 has an oral ED50 of 22.7 ± 0.6 and 104.9±4.3 mg/kg in the rat MES (po), mouse PTZ (ip) assays, respectively. FV-137 also produced a dose related protection of a psychomotor seizure syndrome induced by a 6-Hz electrical stimulus with an oral ED50 of 48.5 ± 1mg/kg, and prevented seizures induced by sound in the Audiogenic Seizure (AGS)-susceptible Frings mouse model (i.p) with an ED50 value of 28.9 ± 3.0 mg/kg. In addition, FV-137 displayed significant efficacy in the electrically induced corneally kindled mouse, kindled rat amygdala, kindled rat hippocampal and lamotrigineresistant kindled rat seizures models. Additionally, FV-137 also increased seizure threshold in a dose– dependent manner, therefore, it has the potential to be effective in refractory, or pharmacoresistant, epilepsy. In-vivo pharmacokinetic studies in rat revealed that FV-137 had excellent oral bioavailability (F% = 85%), moderate oral half-life (~3h), and predicted plasma drug levels in the range 5-50μM across multiple models of epilepsy. In addition, drug exposure as measured by Cmax and AUC was shown to be linear with dose. Safety assessment of FV-137 suggests no interaction with the major CYP isoenzymes when tested up to 500 μM, except for 1A2 and 2D6 with IC50 values of 19 uM and 156 uM respectively and no functional hERG activity when tested up to 50 μM. Irwin and acute toxicity studies suggest a maximally tolerated oral dose of greater than 500 mg/kg in the mice and rats. Mechanistically, in whole-cell patch-clamp recordings FV-137 inhibited P/Q-type Ca 2+ Channels, Cav2.1/β4/α2δ1 and Cav2.2/β3/α2δ1, and voltage-gated Na + channels, Nav1.6 and Nav1.7 and no inhibition up to 100 μM in other related ion channels. Importantly, no significant affinities were observed in a CEREP broad profiling screen (10 μM) across 90 known GPCR, ion channel, transporter and enzyme targets. The anticonvulsant profile of FV-137 suggests that it may be useful for treating multiple forms of epilepsy (generalized tonic-clonic, complex partial, absence seizures), including refractory (or pharmacoresistant) epilepsy, at dose levels that confer a good safety margin. Furthermore, in models of neuropathic pain (spinal nerve injury [SNI] and spinal nerve ligation models [SNL]) and formalin inflammatory pain, administration of a single dose showed robust efficacy (SNI: 23 mg/kg ip; SNL: 60mg/kg and formalin: 41 mg/kg ip). On the basis of its pharmacology and superior drug-like properties, FV-137 was identified as a promising AED candidate.

GR-Align: fast and flexible alignment of protein 3D structures using graphlet degree similarity

Protein structure alignment is key for transferring information from well-studied proteins to less studied ones. Structural alignment identifies the most precise mapping of equivalent residues, as structures are more conserved during evolution than sequences. Among the methods for aligning protein structures, maximum Contact Map Overlap (CMO) has received sustained attention during the past decade. Yet, known algorithms exhibit modest performance and are not applicable for large-scale comparison. Graphlets are small induced subgraphs that are used to design sensitive topological similarity measures between nodes and networks. By generalizing graphlets to ordered graphs, we introduce GR-Align, a CMO heuristic that is suited for database searches. On the Proteus_300 set (44 850 protein domain pairs), GR-Align is several orders of magnitude faster than the state-of-the-art CMO solvers Apurva, MSVNS and AlEigen7, and its similarity score is in better agreement with the structural classification of proteins. On a large-scale experiment on the Gold-standard benchmark dataset (3 207 270 protein domain pairs), GR-Align is several orders of magnitude faster than the state-of-the-art protein structure comparison tools TM-Align, DaliLite, MATT and Yakusa, while achieving similar classification performances. Finally, we illustrate the difference between GR-Align's flexible alignments and the traditional ones by querying a flexible protein in the Astral-40 database (11 154 protein domains). In this experiment, GR-Align's top scoring alignments are not only in better agreement with structural classification of proteins, but also that they allow transferring more information across proteins.

Plasma and Brain Pharmacokinetics of Previously Unexplored Lithium Salts.

Despite its narrow therapeutic window, lithium is still regarded as the gold standard comparator and benchmark treatment for mania. Recent attempts to find new drugs with similar therapeutic activities have yielded new chemical entities. However, these potential new drugs have yet to match the many bioactivities attributable to lithium's efficacy for the treatment of neuropsychiatric diseases. Consequently, an intense effort for re-engineering lithium therapeutics using crystal engineering is currently underway. We sought to improve the likelihood of success of these endeavors by evaluating the pharmacokinetics of previously unexplored lithium salts with organic anions (lithium salicylate and lithium lactate). We report that these lithium salts exhibit profoundly different pharmacokinetics compared to the more common FDA approved salt, lithium carbonate, in rats. Remarkably, lithium salicylate produced elevated plasma and brain levels of lithium beyond 48 hours post-dose without the sharp peak that contributes to the toxicity problems of current lithium therapeutics. These findings could be important for the development of the next generation of lithium therapeutics.

Characterising ChIP-seq binding patterns by model-based peak shape deconvolution

BackgroundChromatin immunoprecipitation combined with massive parallel sequencing (ChIP-seq) is widely used to study protein-chromatin interactions or chromatin modifications at genome-wide level. Sequence reads that accumulate locally at the genome (peaks) reveal loci of selectively modified chromatin or specific sites of chromatin-binding factors. Computational approaches (peak callers) have been developed to identify the global pattern of these sites, most of which assess the deviation from background by applying distribution statistics.ResultsWe have implemented MeDiChISeq, a regression-based approach, which - by following a learning process - defines a representative binding pattern from the investigated ChIP-seq dataset. Using this model MeDiChISeq identifies significant genome-wide patterns of chromatin-bound factors or chromatin modification. MeDiChISeq has been validated for various publicly available ChIP-seq datasets and extensively compared with other peak callers.ConclusionsMeDiChI-Seq has a high resolution when identifying binding events, a high degree of peak-assessment reproducibility in biological replicates, a low level of false calls and a high true discovery rate when evaluated in the context of gold-standard benchmark datasets. Importantly, this approach can be applied not only to ‘sharp’ binding patterns - like those retrieved for transcription factors (TFs) - but also to the broad binding patterns seen for several histone modifications. Notably, we show that at high sequencing depths, MeDiChISeq outperforms other algorithms due to its powerful peak shape recognition capacity which facilitates discerning significant binding events from spurious background enrichment patterns that are enhanced with increased sequencing depths.Electronic supplementary materialThe online version of this article (doi:10.1186/1471-2164-14-834) contains supplementary material, which is available to authorized users.

The significance of the ProtDeform score for structure prediction and alignment.

BackgroundWhen a researcher uses a program to align two proteins and gets a score, one of her main concerns is how often the program gives a similar score to pairs that are or are not in the same fold. This issue was analysed in detail recently for the program TM-align with its associated TM-score. It was shown that because the TM-score is length independent, it allows a P-value and a hit probability to be defined depending only on the score. Also, it was found that the TM-scores of gapless alignments closely follow an Extreme Value Distribution (EVD).The program ProtDeform for structural protein alignment was developed recently and is characterised by the ability to propose different transformations of different protein regions. Our goal is to analyse its associated score to allow a researcher to have objective reasons to prefer one aligner over another, and carry out a better interpretation of the output.ResultsThe study on the ProtDeform score reveals that it is length independent in a wider score range than TM-scores and that PD-scores of gapless (random) alignments also approximately follow an EVD. On the CASP8 predictions, PD-scores and TM-scores, with respect to native structures, are highly correlated (0.95), and show that around a fifth of the predictions have a quality as low as 99.5% of the random scores. Using the Gold Standard benchmark, ProtDeform has lower probabilities of error than TM-align both at a similar speed. The analysis is extended to homology discrimination showing that, again, ProtDeform offers higher hit probabilities than TM-align. Finally, we suggest using three different P-values according to the three different contexts: Gapless alignments, optimised alignments for fold discrimination and that for superfamily discrimination.In conclusion, PD-scores are at the very least as valuable for prediction scoring as TM-scores, and on the protein classification problem, even more reliable.

MetaDBSite: a meta approach to improve protein DNA-binding sites prediction

BackgroundProtein-DNA interactions play an important role in many fundamental biological activities such as DNA replication, transcription and repair. Identification of amino acid residues involved in DNA binding site is critical for understanding of the mechanism of gene regulations. In the last decade, there have been a number of computational approaches developed to predict protein-DNA binding sites based on protein sequence and/or structural information.ResultsIn this article, we present metaDBSite, a meta web server to predict DNA-binding residues for DNA-binding proteins. MetaDBSite integrates the prediction results from six available online web servers: DISIS, DNABindR, BindN, BindN-rf, DP-Bind and DBS-PRED and it solely uses sequence information of proteins. A large dataset of DNA-binding proteins is constructed from the Protein Data Bank and it serves as a gold-standard benchmark to evaluate the metaDBSite approach and the other six predictors.ConclusionsThe comparison results show that metaDBSite outperforms single individual approach. We believe that metaDBSite will become a useful and integrative tool for protein DNA-binding residues prediction. The MetaDBSite web-server is freely available at http://projects.biotec.tu-dresden.de/metadbsite/ and http://sysbio.zju.edu.cn/metadbsite.

Benchmarking short text semantic similarity

Short text semantic similarity measurement is a new and rapidly growing field of research. 'Short texts' are typically sentence length but are not required to be grammatically correct. There is great potential for applying these measures in fields such as information retrieval, dialogue management and question answering. A dataset of 65 sentence pairs, with similarity ratings, produced in 2006 has become adopted as a de facto gold standard benchmark. This paper discusses the adoption of the 2006 dataset, lays down a number of criteria that can be used to determine whether a dataset should be awarded a 'gold standard' accolade and illustrates its use as a benchmark. Procedures for the generation of further gold standard datasets in this field are recommended.

SU-FF-T-629: XBeam: An Accurate, Easily Commissioned Radiotherapy X-Ray Beam Model

Purpose: To develop a simple x-ray beam model which is accurate in both fluence and dose and easy to commission, validating against recently developed gold standard Monte Carlo fluence benchmarks for Siemens and Varian clinical beams. Method and Materials: Dose distributions and relative output factors were measured in water for 6 and 18 MV clinical beams for 1–40 cm wide square fields. A compact ion chamber and photon diode was used to measure the profiles, a parallel-plate chamber for percentage depth dose (PDD). The beam model consists of three sources: a spot and a ring for photons and a ring for electrons. The free parameters were adjusted based on select measurements. The central axis energy distribution was sampled from the thin-target Schiff bremsstrahlung spectrum, and attenuated through a flattener of thickness adjusted to match the measured PDD in small fields. Particles statistical weights were adjusted at different positions to match the measured profiles. Analytical sampling procedures were developed to generate the full phase space above the jaws. Results: Spectral reconstruction resulted in a good match to the measured small field depth dose curves for the Siemens beams. The energy distribution is in good agreement with the fluence benchmark. XBeam fluence and dose along the diagonal of the largest field for the Siemens beams are in good agreement with measurements out to 25 cm. An overestimate outside of the penumbral region will be improved by adjusting the spot size. Comprehensive comparisons to fluence benchmarks for Siemens and Varian beams are in progress and will be presented. Conclusion: XBeam has a minimum number of free parameters, which are fitted to dose measurements within realistic limits. Preliminary results indicate the model will be easy to commission and accurate in both fluence and dose. The fluence benchmarks greatly facilitated development of the beam model.

Short‐ and Long‐Term Effects of Training on Dental Hygiene Faculty Members’ Capacity to Write SOAP Notes

Calibration among faculty is challenging to achieve and maintain. In this study, calibration refers to the training process by which standardization of chart documentation in a SOAP note format was achieved. In the SOAP format, chart entries by health care providers are written in the following categories: Subjective data, Objective data, Assessment, and Plans. The primary training "effect" or outcome that was measured in this study was the capacity of faculty members to write a SOAP note that adhered to prescribed standards for chart documentation. This study was conducted to assess the short-term effects of training and determine whether faculty members' capacity to write appropriately constructed SOAP notes could be sustained for one year. Eight dental hygiene faculty members at the University of Minnesota participated in a pre-training assessment in which they prepared a SOAP note based on a patient case, completed a training session on writing SOAP notes, and completed a post-training test shortly after training that also consisted of writing a SOAP note based on a patient's case. One year later, a follow-up test, similar to the pre- and post-tests, was conducted. Each component of the SOAP note was compared and scored against a gold standard benchmark score of 29 that represented the number of items that should have been included in an ideal SOAP note in the estimation of the investigators, based on chart documentation guidelines of the University of Minnesota Dental Hygiene Division. The mean score for the pre-test was 18.25 (SD=2.82), which represented 63 percent of the benchmark gold standard score of 29. The post-test mean score immediately after training was 24.63 (SD=2.13; 84.9 percent of the benchmark score), and the one-year follow-up mean score was 22.75 (SD=1.83; 78.4 percent of the gold standard benchmark). From the pre-test to the post-test administered in close approximation to the SOAP note training, faculty members' approximation of the gold standard benchmark increased by 35 percent, or 6.28 points, and from the post-test to the follow-up test one year subsequently, approximation of the benchmark score decreased by approximately 1 percent or 1.88 points. Friedman's test indicated that the differences in mean scores for the pre-test, post-test, and follow-up test were significant. The Sign test was used for post hoc tests; alpha was adjusted using Bonferroni's procedure. Conclusions support a hypothesis that faculty capacity to write a SOAP note that adheres to standards can be increased through training and that the effects can be maintained over a period of approximately one year.

Claims Reserving When There Are Negative Values in the Runoff Triangle

This paper is concerned with the situation that occurs in claims reserving when there are negative values in the development triangle of incremental claim amounts. Typically these negative values will be the result of salvage recoveries, payments from third parties, total or partial cancellation of outstanding claims due to initial overestimation of the loss or to a possible favorable jury decision in favor of the insurer, rejection by the insurer, or just plain errors. Some of the traditional methods of claims reserving, such as the chain-ladder technique, may produce estimates of the reserves even when there are negative values. However, many methods can break down in the presence of enough (in number and/or size) negative incremental claims if certain constraints are not met. Historically the chain-ladder method has been used as a gold standard (benchmark) because of its generalized use and ease of application. A method that improves on the gold standard is one that can handle situations where there are many negative incremental claims and/or some of these are large. This paper presents a Bayesian model to consider negative incremental values, based on a three-parameter log-normal distribution. The model presented here allows the actuary to provide point estimates and measures of dispersion, as well as the complete distribution for outstanding claims from which the reserves can be derived. It is concluded that the method has a clear advantage over other existing methods. A Markov chain Monte Carlo simulation is applied using the package WinBUGS.

SU‐FF‐T‐293: Inter‐Linac and Intra‐Linac Variability of X‐Ray and Electron Dose Distributions

Purpose: To determine the maximum expected variation in fluence, differential in energy and position, with an accuracy of 1% for a single linac model, and eventually, combining with Monte Carlo beam models, to develop gold standard beam model benchmarks. Method and Materials: The 6 and 18 MV beams of the Siemens Primus linear accelerator (linac) were tuned through the manufacturer acceptance range in field flatness by adjusting beam energy. Dose profiles were measured at the low, high and average energy in this range for small, medium and large field sizes to quantify the intra‐linac variations caused by energy adjustment. The 6 and 18 MV x‐ray beams, and 6–21 MeV electron beams on three Primus machines, with the same treatment head, were tuned to give the same central axis depth dose curves to better than 0.5% for the largest fields available: 40×40 cm for photons, and 25×25 cm for electrons. Profiles were measured on these linacs to investigate the inter‐linac variations. Results: For the intra‐linac variations, the energy tuning has the largest effect of 7% at 10 cm on beam penetration for small fields. Scattering in the field reduces this effect for larger fields. Wedged‐field profiles are affected more than the open field, due to the dependence of transmission on energy. For inter‐linac variation, profiles measured from three different machines vary more than 2% due to beam tuning differences, lateral displacement of flatteners or scattering foils, and differences in position of components relative to the source. Conclusion: The range of inter‐linac and intra‐linac variability has been measured on a Primus linac and is clinically significant. This data will be used with Monte Carlo treatment head simulation to calculate gold standard beam model benchmarks for the development of tunable beam models of high accuracy and detail in the dose‐critical components of fluence.

Benchmarking the Effectiveness of Psychotherapy: Program Evaluation as a Component of Evidence-Based Practice

The medical director of a child guidance center is starting a new treatment program. Following intense media coverage of adverse events associated with selective serotonin reuptake inhibitors, depressed teens and their families are refusing to accept pharmacotherapy. The director has paid for three social work therapists to attend a cognitivebehavioral therapy (CBT) workshop, and these clinicians will begin seeing depressed teens next week. The director, however, is worried. From what she has heard and read, CBT has a good track record in treating depression although it may work best in combination with the medications that her patients are refusing to take. In addition, she wonders how well will CBT work when delivered by her social work therapists to the population of poor, Spanishspeaking teens served by the clinic? In this example, the medical director struggles with how to bring the principles of evidence-based practice (EBP) to bear on the problem of program evaluation. She is asking a critical, pragmatic question: Does therapy work, here, now, and with my patients? This column discusses a method for addressing this query that uses the results of published psychotherapy clinical trials as a gold-standard benchmark against which the outcomes of practice can be measured (McFall, 1996). This methodology views research evidence on the effects of psychotherapy as, literally, a base—solid ground that researchers and practitioners can use as a foundation when trying to solve problems, whether theoretical or applied.