Gold Standard Set Research Articles

Machine learning for phenotyping opioid overdose events.

To develop machine learning models for classifying the severity of opioid overdose events from clinical data. Opioid overdoses were identified by diagnoses codes from the Marshfield Clinic population and assigned a severity score via chart review to form a gold standard set of labels. Three primary feature sets were constructed from disparate data sources surrounding each event and used to train machine learning models for phenotyping. Random forest and penalized logistic regression models gave the best performance with cross-validated mean areas under the ROC curves (AUCs) for all severity classes of 0.893 and 0.882 respectively. Features derived from a common data model outperformed features collected from disparate data sources for the same cohort of patients (AUCs 0.893 versus 0.837, p value = 0.002). The addition of features extracted from free text to machine learning models also increased AUCs from 0.827 to 0.893 (p value < 0.0001). Key word features extracted using natural language processing (NLP) such as 'Narcan' and 'Endotracheal Tube' are important for classifying overdose event severity. Random forest models using features derived from a common data model and free text can be effective for classifying opioid overdose events.

The Embase UK filter: validation of a geographic search filter to retrieve research about the UK from OVID Embase.

The authors developed a validated geographic search filter to retrieve research about the United Kingdom (UK) from OVID Embase. It was created to be used alongside their previously published OVID MEDLINE UK filter in systematic literature searches for context-sensitive topics. To develop a validated geographic search filter to retrieve research about the UK from OVID Embase. The Embase UK filter was translated from the MEDLINE UK filter. A gold standard set of references was generated using the relative recall method. The set contained references to publications about the UK that had informed National Institute for Health and Care Excellence (NICE) guidance and it was used to validate the filter. Recall, precision and number-needed-to-read (NNR) were calculated using a case study. The validated Embase UK filter demonstrated 99.8% recall against the references with UK identifiers in the gold standard set. In the case study, the Embase UK filter demonstrated 98.5% recall, 7.6% precision and a NNR of 13. The Embase UK filter can be used alongside the MEDLINE UK filter. The filters have the potential to save time and associated resource costs when they are used for context-sensitive topics that require research about UK settings.

Powerful gene set analysis in GWAS with the Generalized Berk-Jones statistic.

A common complementary strategy in Genome-Wide Association Studies (GWAS) is to perform Gene Set Analysis (GSA), which tests for the association between one phenotype of interest and an entire set of Single Nucleotide Polymorphisms (SNPs) residing in selected genes. While there exist many tools for performing GSA, popular methods often include a number of ad-hoc steps that are difficult to justify statistically, provide complicated interpretations based on permutation inference, and demonstrate poor operating characteristics. Additionally, the lack of gold standard gene set lists can produce misleading results and create difficulties in comparing analyses even across the same phenotype. We introduce the Generalized Berk-Jones (GBJ) statistic for GSA, a permutation-free parametric framework that offers asymptotic power guarantees in certain set-based testing settings. To adjust for confounding introduced by different gene set lists, we further develop a GBJ step-down inference technique that can discriminate between gene sets driven to significance by single genes and those demonstrating group-level effects. We compare GBJ to popular alternatives through simulation and re-analysis of summary statistics from a large breast cancer GWAS, and we show how GBJ can increase power by incorporating information from multiple signals in the same gene. In addition, we illustrate how breast cancer pathway analysis can be confounded by the frequency of FGFR2 in pathway lists. Our approach is further validated on two other datasets of summary statistics generated from GWAS of height and schizophrenia.

Adapting Community Detection Algorithms for Disease Module Identification in Heterogeneous Biological Networks.

Biological networks catalog the complex web of interactions happening between different molecules, typically proteins, within a cell. These networks are known to be highly modular, with groups of proteins associated with specific biological functions. Human diseases often arise from the dysfunction of one or more such proteins of the biological functional group. The ability, to identify and automatically extract these modules has implications for understanding the etiology of different diseases as well as the functional roles of different protein modules in disease. The recent DREAM challenge posed the problem of identifying disease modules from six heterogeneous networks of proteins/genes. There exist many community detection algorithms, but all of them are not adaptable to the biological context, as these networks are densely connected and the size of biologically relevant modules is quite small. The contribution of this study is 3-fold: first, we present a comprehensive assessment of many classic community detection algorithms for biological networks to identify non-overlapping communities, and propose heuristics to identify small and structurally well-defined communities—core modules. We evaluated our performance over 180 GWAS datasets. In comparison to traditional approaches, with our proposed approach we could identify 50% more number of disease-relevant modules. Thus, we show that it is important to identify more compact modules for better performance. Next, we sought to understand the peculiar characteristics of disease-enriched modules and what causes standard community detection algorithms to detect so few of them. We performed a comprehensive analysis of the interaction patterns of known disease genes to understand the structure of disease modules and show that merely considering the known disease genes set as a module does not give good quality clusters, as measured by typical metrics such as modularity and conductance. We go on to present a methodology leveraging these known disease genes, to also include the neighboring nodes of these genes into a module, to form good quality clusters and subsequently extract a “gold-standard set” of disease modules. Lastly, we demonstrate, with justification, that “overlapping” community detection algorithms should be the preferred choice for disease module identification since several genes participate in multiple biological functions.

Using SNOMED CT-encoded problems to improve ICD-10-CM coding—A randomized controlled experiment

ObjectiveClinical problems in the Electronic Health Record that are encoded in SNOMED CT can be translated into ICD-10-CM codes through the NLM’s SNOMED CT to ICD-10-CM map (NLM Map). This study evaluates the potential benefits of using the map-generated codes to assist manual ICD-10-CM coding. MethodsDe-identified clinic notes taken by the physician during an outpatient encounter were made available on a secure web server and randomly assigned for coding by professional coders with usual coding or map-assisted coding. Map-assisted coding made use of the problem list maintained by the physician and the NLM Map to suggest candidate ICD-10-CM codes to the coder. A gold standard set of codes for each note was established by the coders using a Delphi consensus process. Outcomes included coding time, coding reliability as measured by the Jaccard coefficients between codes from two coders with the same method of coding, and coding accuracy as measured by recall, precision and F-score according to the gold standard. ResultsWith map-assisted coding, the average coding time per note reduced by 1.5 min (p = 0.006). There was a small increase in coding reliability and accuracy (not statistical significant). The benefits were more pronounced in the more experienced than less experienced coders. Detailed analysis of cases in which the correct ICD-10-CM codes were not found by the NLM Map showed that most failures were related to omission in the problem list and suboptimal mapping of the problem list terms to SNOMED CT. Only 12% of the failures was caused by errors in the NLM Map. ConclusionMap-assisted coding reduces coding time and can potentially improve coding reliability and accuracy, especially for more experienced coders. More effort is needed to improve the accuracy of the map-suggested ICD-10-CM codes.

Integrating genomic resources to present full gene and putative promoter capture probe sets for bread wheat

BackgroundWhole-genome shotgun resequencing of wheat is expensive because of its large, repetitive genome. Moreover, sequence data can fail to map uniquely to the reference genome, making it difficult to unambiguously assign variation. Resequencing using target capture enables sequencing of large numbers of individuals at high coverage to reliably identify variants associated with important agronomic traits. Previous studies have implemented complementary DNA/exon or gene-based probe sets in which the promoter and intron sequence is largely missing alongside newly characterized genes from the recent improved reference sequences.ResultsWe present and validate 2 gold standard capture probe sets for hexaploid bread wheat, a gene and a putative promoter capture, which are designed using recently developed genome sequence and annotation resources. The captures can be combined or used independently. We demonstrate that the capture probe sets effectively enrich the high-confidence genes and putative promoter regions that were identified in the genome alongside a large proportion of the low-confidence genes and associated promoters. Finally, we demonstrate successful sample multiplexing that allows generation of adequate sequence coverage for single-nucleotide polymorphism calling while significantly reducing cost per sample for gene and putative promoter capture.ConclusionsWe show that a capture design employing an “island strategy” can enable analysis of the large gene/putative promoter space of wheat with only 2 × 160 Mbp probe sets. Furthermore, these assays extend the regions of the wheat genome that are amenable to analyses beyond its exome, providing tools for detailed characterization of these regulatory regions in large populations.

Automatic identification of relevant chemical compounds from patents.

In commercial research and development projects, public disclosure of new chemical compounds often takes place in patents. Only a small proportion of these compounds are published in journals, usually a few years after the patent. Patent authorities make available the patents but do not provide systematic continuous chemical annotations. Content databases such as Elsevier’s Reaxys provide such services mostly based on manual excerptions, which are time-consuming and costly. Automatic text-mining approaches help overcome some of the limitations of the manual process. Different text-mining approaches exist to extract chemical entities from patents. The majority of them have been developed using sub-sections of patent documents and focus on mentions of compounds. Less attention has been given to relevancy of a compound in a patent. Relevancy of a compound to a patent is based on the patent’s context. A relevant compound plays a major role within a patent. Identification of relevant compounds reduces the size of the extracted data and improves the usefulness of patent resources (e.g. supports identifying the main compounds). Annotators of databases like Reaxys only annotate relevant compounds. In this study, we design an automated system that extracts chemical entities from patents and classifies their relevance. The gold-standard set contained 18 789 chemical entity annotations. Of these, 10% were relevant compounds, 88% were irrelevant and 2% were equivocal. Our compound recognition system was based on proprietary tools. The performance (F-score) of the system on compound recognition was 84% on the development set and 86% on the test set. The relevancy classification system had an F-score of 86% on the development set and 82% on the test set. Our system can extract chemical compounds from patents and classify their relevance with high performance. This enables the extension of the Reaxys database by means of automation.

Interrogation of genome-wide networks in biology: comparison of knowledge-based and statistical methods

Networks are used extensively in the study of biological systems to address a wide range of questions such as understanding the complex behaviour of a given system or identifying key alterations leading to a disease phenotype. Numerous network-based methods have been developed for inferring molecular interactions using transcriptomic and proteomic data. Different network methods come with their own advantages and limitations, and often give different results for the same data. A systematic study is essential to understand how the methods fare in terms of correctly predicting known biological processes and yielding testable biological hypotheses. To address this, we have carried out a comparison of four different methods to derive context-specific perturbations for two different case studies and evaluated their performance. The methods can be broadly classified into statistical inference and knowledge-based methods. Two of the four methods, WGCNA and ARACNE, belong to the broad class of data-driven approaches which do not rely on prior network information. On the other hand, ResponseNet and jActiveModules utilise knowledge-based protein–protein interaction networks and integrate condition-specific transcriptome or proteome data. We evaluated the interactions inferred through all the approaches and assessed their biological relevance based on three criteria: (1) enrichment of the gold standard gene sets, (2) comparison to gold standard pathways and (3) recovery of hub genes from the context-specific perturbed network, known to be related to the given condition. Comparing the performance of these four methods in two different cases, tuberculosis and melanoma, showed superior performance by ResponseNet, based on all three criteria.

Estimating post-editing time using a gold-standard set of machine translation errors

With the improved quality of Machine Translation (MT) systems in the last decades, post-editing (the correction of MT errors) has gained importance in Computer-Assisted Translation (CAT) workflows. Depending on the number and the severity of the errors in the MT output, the effort required to post-edit varies from sentence to sentence. The existing Quality Estimation (QE) systems provide quality scores that reflect the quality of an MT output at sentence level or word level. However, they fail to explain the relationship between different types of MT errors and the required post-editing effort to correct them. We suggest a more informative approach to QE in which different types of MT errors are detected in a first step, which are then used to estimate post-editing effort in a second step. In this paper we define the upper boundary of such a system. We use different machine learning methods to estimate Post-Editing Time (PET) by using a gold-standard set of MT errors as features. We show that post-editing time can be estimated with high accuracy when all the translation errors in the MT output are known. Furthermore, we apply feature selection methods and investigate the predictive power of different MT error types on PET. Our results show that the same prediction performance can be achieved by only using a small subset of MT error types, indicating that successful two-step QE systems can be built with less effort in the future, by detecting only the error types with highest predictive power.

Development and validation of the PEPPER framework (Prenatal Exposure PubMed ParsER) with applications to food additives.

BackgroundGlobally, 36% of deaths among children can be attributed to environmental factors. However, no comprehensive list of environmental exposures exists. We seek to address this gap by developing a literature-mining algorithm to catalog prenatal environmental exposures.MethodsWe designed a framework called PEPPER Prenatal Exposure PubMed ParsER to a) catalog prenatal exposures studied in the literature and b) identify study type. Using PubMed Central, PEPPER classifies article type (methodology, systematic review) and catalogs prenatal exposures. We coupled PEPPER with the FDA’s food additive database to form a master set of exposures.ResultsWe found that of 31 764 prenatal exposure studies only 53.0% were methodology studies. PEPPER consists of 219 prenatal exposures, including a common set of 43 exposures. PEPPER captured prenatal exposures from 56.4% of methodology studies (9492/16 832 studies). Two raters independently reviewed 50 randomly selected articles and annotated presence of exposures and study methodology type. Error rates for PEPPER’s exposure assignment ranged from 0.56% to 1.30% depending on the rater. Evaluation of the study type assignment showed agreement ranging from 96% to 100% (kappa = 0.909, p < .001). Using a gold-standard set of relevant prenatal exposure studies, PEPPER achieved a recall of 94.4%.ConclusionsUsing curated exposures and food additives; PEPPER provides the first comprehensive list of 219 prenatal exposures studied in methodology papers. On average, 1.45 exposures were investigated per study. PEPPER successfully distinguished article type for all prenatal studies allowing literature gaps to be easily identified.

Context-specific interactions in literature-curated protein interaction databases

BackgroundDatabases of literature-curated protein-protein interactions (PPIs) are often used to interpret high-throughput interactome mapping studies and estimate error rates. These databases combine interactions across thousands of published studies and experimental techniques. Because the tendency for two proteins to interact depends on the local conditions, this heterogeneity of conditions means that only a subset of database PPIs are interacting during any given experiment. A typical use of these databases as gold standards in interactome mapping projects, however, assumes that PPIs included in the database are indeed interacting under the experimental conditions of the study.ResultsUsing raw data from 20 co-fractionation experiments and six published interactomes, we demonstrate that this assumption is often false, with up to 55% of purported gold standard interactions showing no evidence of interaction, on average. We identify a subset of CORUM database complexes that do show consistent evidence of interaction in co-fractionation studies, and we use this subset as gold standards to dramatically improve interactome mapping as judged by the number of predicted interactions at a given error rate.ConclusionsWe recommend using this CORUM subset as the gold standard set in future co-fractionation studies. More generally, we recommend using the subset of literature-curated PPIs that are specific to the experimental context whenever possible.

Development and validation of a search filter to identify equity-focused studies: reducing the number needed to screen

BackgroundHealth inequalities, worse health associated with social and economic disadvantage, are reported by a minority of research articles. Locating these studies when conducting an equity-focused systematic review is challenging due to a deficit in standardised terminology, indexing, and lack of validated search filters. Current reporting guidelines recommend not applying filters, meaning that increased resources are needed at the screening stage.MethodsWe aimed to design and test search filters to locate studies that reported outcomes by a social determinant of health. We developed and expanded a ‘specific terms strategy’ using keywords and subject headings compiled from recent systematic reviews that applied an equity filter. A ‘non-specific strategy’ was compiled from phrases used to describe equity analyses that were reported in titles and abstracts, and related subject headings. Gold standard evaluation and validation sets were compiled. The filters were developed in MEDLINE, adapted for Embase and tested in both. We set a target of 0.90 sensitivity (95% CI; 0.84, 0.94) in retrieving 150 gold standard validation papers. We noted the reduction in the number needed to screen in a proposed equity-focused systematic review and the proportion of equity-focused reviews we assessed in the project that applied an equity filter to their search strategy.ResultsThe specific terms strategy filtered out 93-95% of all records, and retrieved a validation set of articles with a sensitivity of 0.84 in MEDLINE (0.77, 0.89), and 0.87 (0.81, 0.92) in Embase. When combined (Boolean ‘OR’) with the non-specific strategy sensitivity was 0.92 (0.86, 0.96) in MEDLINE (Embase 0.94; 0.89, 0.97). The number needed to screen was reduced by 77% by applying the specific terms strategy, and by 59.7% (MEDLINE) and 63.5% (Embase) by applying the combined strategy. Eighty-one per cent of systematic reviews filtered studies by equity.ConclusionsA combined approach of using specific and non-specific terms is recommended if systematic reviewers wish to filter studies for reporting outcomes by social determinants. Future research should concentrate on the indexing standardisation for equity studies and further development and testing of both specific and non-specific terms for accurate study retrieval.

DynBench3D, a Web-Resource to Dynamically Generate Benchmark Sets of Large Heteromeric Protein Complexes

Multi-protein machines are responsible for most cellular tasks, and many efforts have been invested in the systematic identification and characterization of thousands of these macromolecular assemblies. However, unfortunately, the (quasi) atomic details necessary to understand their function are available only for a tiny fraction of the known complexes. The computational biology community is developing strategies to integrate structural data of different nature, from electron microscopy to X-ray crystallography, to model large molecular machines, as it has been done for individual proteins and interactions with remarkable success. However, unlike for binary interactions, there is no reliable gold-standard set of three-dimensional (3D) complexes to benchmark the performance of these methodologies and detect their limitations. Here, we present a strategy to dynamically generate non-redundant sets of 3D heteromeric complexes with three or more components. By changing the values of sequence identity and component overlap between assemblies required to define complex redundancy, we can create sets of representative complexes with known 3D structure (i.e., target complexes). Using an identity threshold of 20% and imposing a fraction of component overlap of <0.5, we identify 495 unique target complexes, which represent a real non-redundant set of heteromeric assemblies with known 3D structure. Moreover, for each target complex, we also identify a set of assemblies, of varying degrees of identity and component overlap, that can be readily used as input in a complex modeling exercise (i.e., template subcomplexes). We hope that resources like this will significantly help the development and progress assessment of novel methodologies, as docking benchmarks and blind prediction contests did. The interactive resource is accessible at https://DynBench3D.irbbarcelona.org.

Harnessing citizen science through mobile phone technology to screen for immunohistochemical biomarkers in bladder cancer

BackgroundImmunohistochemistry (IHC) is often used in personalisation of cancer treatments. Analysis of large data sets to uncover predictive biomarkers by specialists can be enormously time-consuming. Here we investigated crowdsourcing as a means of reliably analysing immunostained cancer samples to discover biomarkers predictive of cancer survival.MethodsWe crowdsourced the analysis of bladder cancer TMA core samples through the smartphone app ‘Reverse the Odds’. Scores from members of the public were pooled and compared to a gold standard set scored by appropriate specialists. We also used crowdsourced scores to assess associations with disease-specific survival.ResultsData were collected over 721 days, with 4,744,339 classifications performed. The average time per classification was approximately 15 s, with approximately 20,000 h total non-gaming time contributed. The correlation between crowdsourced and expert H-scores (staining intensity × proportion) varied from 0.65 to 0.92 across the markers tested, with six of 10 correlation coefficients at least 0.80. At least two markers (MRE11 and CK20) were significantly associated with survival in patients with bladder cancer, and a further three markers showed results warranting expert follow-up.ConclusionsCrowdsourcing through a smartphone app has the potential to accurately screen IHC data and greatly increase the speed of biomarker discovery.

Verifying Conceptual Domain Models with Human Computation: A Case Study in Software Engineering

Conceptual domain models, such as taxonomies, knowledge graphs or Extended Entity Relationship (EER) diagrams are core to all information systems. The task of verifying the correctness of these models is of high interest to the knowledge and software engineering communities and attracted the first solution approaches using human computation. Yet, since these solutions are published within the boundaries of their communities, there is a lack of concerted work on this topic. As a first step to alleviate this status quo, we formalize the problem of verifying conceptual models and propose a generic approach (VeriCoM) to solve it with human computation techniques. We show how VeriCoM was applied in a software engineering use case focusing on verifying the correctness of an EER diagram against a system specification document. An evaluation of VeriCoM in a series of four workshops within one controlled experiment performed with a crowd of semi-experts lead to the identification of a set of defects with precision of 73% and a recall from a Gold Standard defect set of 63%.

Drug Repositioning by Integrating Known Disease-Gene and Drug-Target Associations in a Semi-supervised Learning Model.

Computational drug repositioning has been proven as a promising and efficient strategy for discovering new uses from existing drugs. To achieve this goal, a number of computational methods have been proposed, which are based on different data sources of drugs and diseases. These methods approach the problem using either machine learning- or network-based models with an assumption that similar drugs can be used for similar diseases to identify new indications of drugs. Therefore, similarities between drugs and between diseases are usually used as inputs. In addition, known drug-disease associations are also needed for the methods as prior information. It should be noted that those associations are still not well established due to the fact that many of marketed drugs have been withdrawn and this could affect the outcome of the methods. In this study, we propose a novel method named RLSDR (Regularized Least Square for Drug Repositioning) to find new uses of drugs. More specifically, it relies on a semi-supervised learning model, Regularized Least Square, thus it does not require definition of non-drug-disease associations as previously proposed machine learning-based methods. In addition, the similarity between drugs measured by chemical structures of drug compounds and the similarity between diseases which share phenotypes can be represented in a form of either similarity network or similarity matrix as inputs of the method. Moreover, instead of using a gold-standard set of known drug-disease associations, we construct an artificial set of the associations based on known disease-gene and drug-target associations. Experiment results demonstrate that RLSDR achieves better prediction performance on the artificial set of drug-disease associations than that on the gold-standard ones in terms of area under the Receiver Operating Characteristic (ROC) curve (AUC). In addition, it outperforms two representative network-based methods irrespective of the prior information of drug-disease associations. Novel indications for a number of drugs are also identified and validated by evidences from a different data resource.

Sensing the cilium, digital capture of ciliary data for comparative genomics investigations

BackgroundCilia are specialized, hair-like structures that project from the cell bodies of eukaryotic cells. With increased understanding of the distribution and functions of various types of cilia, interest in these organelles is accelerating. To effectively use this great expansion in knowledge, this information must be made digitally accessible and available for large-scale analytical and computational investigation. Capture and integration of knowledge about cilia into existing knowledge bases, thus providing the ability to improve comparative genomic data analysis, is the objective of this work.MethodsWe focused on the capture of information about cilia as studied in the laboratory mouse, a primary model of human biology. The workflow developed establishes a standard for capture of comparative functional data relevant to human biology. We established the 310 closest mouse orthologs of the 302 human genes defined in the SYSCILIA Gold Standard set of ciliary genes. For the mouse genes, we identified biomedical literature for curation and used Gene Ontology (GO) curation paradigms to provide functional annotations from these publications.ResultsEmploying a methodology for comprehensive capture of experimental data about cilia genes in structured, digital form, we established a workflow for curation of experimental literature detailing molecular function and roles of cilia proteins starting with the mouse orthologs of the human SYSCILIA gene set. We worked closely with the GO Consortium ontology development editors and the SYSCILIA Consortium to improve the representation of ciliary biology within the GO. During the time frame of the ontology improvement project, we have fully curated 134 of these 310 mouse genes, resulting in an increase in the number of ciliary and other experimental annotations.ConclusionsWe have improved the GO annotations available for mouse genes orthologous to the human genes in the SYSCILIA Consortium’s Gold Standard set. In addition, ciliary terminology in the GO itself was improved in collaboration with GO ontology developers and the SYSCILIA Consortium. These improvements to the GO terms for the functions and roles of ciliary proteins, along with the increase in annotations of the corresponding genes, enhance the representation of ciliary processes and localizations and improve access to these data during large-scale bioinformatic analyses.

The best choice of equipment to obtain high quality standardised results in intra-oral photography – a comparison between the common practice in the UK and the gold standard set by the literature

Since digital technology made dental photography widely available, photographers have been trying to establish the ‘gold standard’ when obtaining intra-oral photographs. A basic knowledge of photographic principles, familiarity with particular dental techniques and a correct choice and use of equipment contribute significantly both to the standardisation and quality of photographs. A recent survey between members of the Institute of Medical Illustrators shed a light on the current practice in the UK. Medical photographers do not always have access to the full range of suitable equipment. A better selection of retractors and occlusal mirrors would undoubtedly contribute to higher quality images.

Maize GO Annotation-Methods, Evaluation, and Review (maize-GAMER).

We created a new high‐coverage, robust, and reproducible functional annotation of maize protein‐coding genes based on Gene Ontology (GO) term assignments. Whereas the existing Phytozome and Gramene maize GO annotation sets only cover 41% and 56% of maize protein‐coding genes, respectively, this study provides annotations for 100% of the genes. We also compared the quality of our newly derived annotations with the existing Gramene and Phytozome functional annotation sets by comparing all three to a manually annotated gold standard set of 1,619 genes where annotations were primarily inferred from direct assay or mutant phenotype. Evaluations based on the gold standard indicate that our new annotation set is measurably more accurate than those from Phytozome and Gramene. To derive this new high‐coverage, high‐confidence annotation set, we used sequence similarity and protein domain presence methods as well as mixed‐method pipelines that were developed for the Critical Assessment of Function Annotation (CAFA) challenge. Our project to improve maize annotations is called maize‐GAMER (GO Annotation Method, Evaluation, and Review), and the newly derived annotations are accessible via MaizeGDB (http://download.maizegdb.org/maize-GAMER) and CyVerse (B73 RefGen_v3 5b+ at doi.org/10.7946/P2S62P and B73 RefGen_v4 Zm00001d.2 at doi.org/10.7946/P2M925).

A pilot study: a teaching electronic medical record for educating and assessing residents in the care of patients

ABSTRACTObjective: We tested a novel, web-based teaching electronic medical record to teach and assess residents’ ability to enter appropriate admission orders for patients admitted to the intensive care unit. The primary objective was to determine if this tool could improve the learners’ ability to enter an evidence-based, comprehensive initial care plan for critically ill patients.Methods: The authors created three modules using de-identifed real patient data from selected patients that were admitted to the intensive care unit. All senior residents (113 total) were invited to participate in a dedicated two-hour educational session to complete the modules. Learner performance was graded against gold standard admission order sets created by study investigators based on the latest evidence-based medicine and guidelines.Results: The session was attended by 39 residents (34.5% of invitees). There was an average improvement of at least 20% in users’ scores across the three modules (Module 3-Module 1 mean difference 22.5%; p = 0.001 and Module 3-Module 2 mean difference 20.3%; p = 0.001). Diagnostic acumen improved in successive modules. Almost 90% of the residents reported the technology was an effective form of teaching and would use it autonomously if more modules were provided.Conclusions: In this pilot project, using a novel educational tool, users’ patient care performance scores improved with a high level of user satisfaction. These results identify a realistic and well-received way to supplement residents’ training and assessment on core clinical care and patient management in the face of duty hour restrictions.