Ontology Curation Research Articles

Enhancing Human-in-the-Loop Ontology Curation Results through Task Design

The success of artificial intelligence (AI) applications is heavily dependent on the quality of data they rely on. Thus, data curation, dealing with cleaning, organising, and managing data, has become a significant research area to be addressed. Increasingly, semantic data structures such as ontologies and knowledge graphs empower the new generation of AI systems. In this article, we focus on ontologies as a special type of data. Ontologies are conceptual data structures representing a domain of interest and are often used as a backbone to knowledge-based intelligent systems or as an additional input for machine learning algorithms. Low-quality ontologies, containing incorrectly represented information or controversial concepts modelled from a single viewpoint, can lead to invalid application outputs and biased systems. Thus, we focus on the curation of ontologies as a crucial factor for ensuring trust in the enabled AI systems. While some ontology quality aspects can be automatically evaluated, others require a human-in-the-loop evaluation. Yet, despite the importance of the field, several ontology quality aspects have not yet been addressed and there is a lack of guidelines for optimal design of human computation tasks to perform such evaluations. In this article, we advance the state-of-the-art by making two novel contributions. First, we propose a human computation (HC)–based approach for the verification of ontology restrictions —an ontology evaluation aspect that has not yet been addressed with HC techniques. Second, by performing two controlled experiments with a junior expert crowd, we empirically derive task design guidelines for achieving high-quality evaluation results related to (i) the formalism for representing ontology axioms and (ii) crowd qualification testing . We find that the representation format of the ontology does not significantly influence the campaign results. Nevertheless, contributors expressed a preference in working with a graphical ontology representation. Additionally, we show that an objective qualification test is better fitted at assessing contributors’ prior knowledge rather than a subjective self-assessment and that prior modelling knowledge of the contributors had a positive effect on their judgements. We make all artefacts designed and used in the experimental campaign publicly available.

Towards multi-target glioblastoma therapy: Structural, distribution, and functional insights into protein target candidates

Glioblastoma is the most commonly occurring and most lethal primary brain tumor. Treatment options are limited in number and therapeutic development remains a major challenge. However, substantial progress has been made in better understanding the underlying biology of the disease. A recent proteomic meta-analysis revealed that 270 proteins were commonly dysregulated in glioblastoma, highlighting the complexity of the disease. This motivated us to explore potential protein targets which may be collectively inhibited, based on common upregulation, as part of a multi-target therapeutic strategy. Herein, we identify and characterize structural attributes relevant to the druggability of six protein target candidates. Computational analysis of crystal structures revealed druggable cavities in each of these proteins, and various parameters of these cavities were determined. For proteins with inhibitor-bound structures available, inhibitor compounds were found to overlap with the computationally determined cavities upon structural alignment. We also performed bioinformatic analysis for normal transcriptional expression distribution of these proteins across various brain regions and various tissues, as well as gene ontology curation to gain functional insights, as this information is useful for understanding the potential for off-target adverse effects. Our findings represent initial steps towards the development of multi-target glioblastoma therapy and may aid future work exploring similar therapeutic strategies.

A Tissue-Specific and Toxicology-Focused Knowledge Graph

Molecular biology-focused knowledge graphs (KGs) are directed graphs that integrate information from heterogeneous sources of biological and biomedical data, such as ontologies and public databases. They provide a holistic view of biology, chemistry, and disease, allowing users to draw non-obvious connections between concepts through shared associations. While these massive graphs are constructed using carefully curated ontologies and annotations from public databases, much of the information relating the concepts is context specific. Two important variables that determine the applicability of a given ontology annotation are the species and (especially) the tissue type in which it takes place. Using a data-driven approach and the results from thousands of high-quality gene expression samples, we have constructed tissue-specific KGs (using liver, kidney, and heart as examples) that empirically validate the annotations provided by ontology curators. The resulting human-centered KGs are designed for toxicology applications but are generalizable to other areas of human biology, addressing the issue of tissue specificity that often limits the applicability of other large KGs. These knowledge graphs can serve as valuable tools for generating transparent explanations of experimental results in the form of mechanistic hypotheses that are highly relevant to the studied tissue. Because the data-driven relations are derived from a large collection of human in vitro data, these KGs are particularly well suited for in vitro toxicology applications.

Property Assertion Constraints for ontologies and knowledge graphs

PurposeThe curation of ontologies and knowledge graphs (KGs) is an essential task for industrial knowledge-based applications, as they rely on the contained knowledge to be correct and error-free. Often, a significant amount of a KG is curated by humans. Established validation methods, such as Shapes Constraint Language, Shape Expressions or Web Ontology Language, can detect wrong statements only after their materialization, which can be too late. Instead, an approach that avoids errors and adequately supports users is required.Design/methodology/approachFor solving that problem, Property Assertion Constraints (PACs) have been developed. PACs extend the range definition of a property with additional logic expressed with SPARQL. For the context of a given instance and property, a tailored PAC query is dynamically built and triggered on the KG. It can determine all values that will result in valid property value assertions.FindingsPACs can avoid the expansion of KGs with invalid property value assertions effectively, as their contained expertise narrows down the valid options a user can choose from. This simplifies the knowledge curation and, most notably, relieves users or machines from knowing and applying this expertise, but instead enables a computer to take care of it.Originality/valuePACs are fundamentally different from existing approaches. Instead of detecting erroneous materialized facts, they can determine all semantically correct assertions before materializing them. This avoids invalid property value assertions and provides users an informed, purposeful assistance. To the author's knowledge, PACs are the only such approach.

Knowledge Representation and Management: Notable Contributions in 2021.

To select, present, and summarize the best papers in the field of Knowledge Representation and Management (KRM) published in 2021. Following the International Medical Informatics Association (IMIA) Yearbook guidelines, a comprehensive and standardized review of the biomedical informatics literature was performed to select the best KRM papers published in 2021, based on PubMed queries. A total of 1,231 publications were retrieved from PubMed. We nominated 15 candidate best papers, and four of them were finally selected as the best papers in the KRM section. The topics covered by these papers include knowledge graph, ontology development, ontology alignment, and the International Classification of Diseases. In the KRM best paper selection for 2021, the candidate best papers covered a wider spectrum of topics compared to the last year's significant focus on ontology curation. In particular, ontology development for specific domains (e.g., Alzheimer's disease, infectious diseases, bioethics) has received the most attention.

Gene Ontology curation of the blood-brain barrier to improve the analysis of Alzheimer's and other neurological diseases.

The role of the blood–brain barrier (BBB) in Alzheimer’s and other neurodegenerative diseases is still the subject of many studies. However, those studies using high-throughput methods have been compromised by the lack of Gene Ontology (GO) annotations describing the role of proteins in the normal function of the BBB. The GO Consortium provides a gold-standard bioinformatics resource used for analysis and interpretation of large biomedical data sets. However, the GO is also used by other research communities and, therefore, must meet a variety of demands on the breadth and depth of information that is provided. To meet the needs of the Alzheimer’s research community we have focused on the GO annotation of the BBB, with over 100 transport or junctional proteins prioritized for annotation. This project has led to a substantial increase in the number of human proteins associated with BBB-relevant GO terms as well as more comprehensive annotation of these proteins in many other processes. Furthermore, data describing the microRNAs that regulate the expression of these priority proteins have also been curated. Thus, this project has increased both the breadth and depth of annotation for these prioritized BBB proteins.Database URLhttps://www.ebi.ac.uk/QuickGO/

Proteomic analysis of koala (phascolarctos cinereus) spermatozoa and prostatic bodies.

The aims of this study were to investigate the proteome of koala spermatozoa and that of the prostatic bodies with which they interact during ejaculation. For this purpose, spermatozoa and prostatic bodies were fractionated from the semen of four male koalas and analysed by HPLC MS/MS. This strategy identified 744 sperm and 1297 prostatic body proteins, which were subsequently attributed to 482 and 776 unique gene products, respectively. Gene ontology curation of the sperm proteome revealed an abundance of proteins mapping to the canonical sirtuin and 14-3-3 signalling pathways. By contrast, protein ubiquitination and unfolded protein response pathways dominated the equivalent analysis of proteins uniquely identified in prostatic bodies. Koala sperm proteins featured an enrichment of those mapping to the functional categories of cellular compromise/inflammatory response, whilst those of the prostatic body revealed an over-representation of molecular chaperone and stress-related proteins. Cross-species comparisons demonstrated that the koala sperm proteome displays greater conservation with that of eutherians (human; 93%) as opposed to reptile (crocodile; 39%) and avian (rooster; 27%) spermatozoa. Together, this work contributes to our overall understanding of the core sperm proteome and has identified biomarkers that may contribute to the exceptional longevity of koala spermatozoa during ex vivo storage.

Knowledge Representation and Management: Interest in New Solutions for Ontology Curation.

SummaryObjective: To select, present and summarize some of the best papers in the field of Knowledge Representation and Management (KRM) published in 2020. Methods: A comprehensive and standardized review of the medical informatics literature was performed to select the most interesting papers of KRM published in 2020, based on PubMed queries. This review was conducted according to the IMIA Yearbook guidelines. Results: Four best papers were selected among 1,175 publications. In contrast with the papers selected last year, the four best papers of 2020 demonstrated a significant focus on methods and tools for ontology curation and design. The usual KRM application domains (bioinformatics, machine learning, and electronic health records) were also represented. Conclusion: In 2020, ontology curation emerges as a significant topic of research interest. Bioinformatics, machine learning, and electronics health records remain significant research areas in the KRM community with various applications. Knowledge representations are key to advance machine learning by providing context and to develop novel bioinformatics metrics. As in 2019, representations serve a great variety of applications across many medical domains, with actionable results and now with growing adhesion to the open science initiative.

A Practical Approach to Forgetting in Description Logics with Nominals

This paper investigates the problem of forgetting in description logics with nominals. In particular, we develop a practical method for forgetting concept and role names from ontologies specified in the description logic ALCO, extending the basic ALC with nominals. The method always terminates, and is sound in the sense that the forgetting solution computed by the method has the same logical consequences with the original ontology. The method is so far the only approach to deductive forgetting in description logics with nominals. An evaluation of a prototype implementation shows that the method achieves a significant speed-up and notably better success rates than the Lethe tool which performs deductive forgetting for ALC-ontologies. Compared to Fame, a semantic forgetting tool for ALCOIH-ontologies, better success rates are attained. From the perspective of ontology engineering this is very useful, as it provides ontology curators with a powerful tool to produce views of ontologies.

Wikidata as a knowledge graph for the life sciences.

Wikidata is a community-maintained knowledge base that has been assembled from repositories in the fields of genomics, proteomics, genetic variants, pathways, chemical compounds, and diseases, and that adheres to the FAIR principles of findability, accessibility, interoperability and reusability. Here we describe the breadth and depth of the biomedical knowledge contained within Wikidata, and discuss the open-source tools we have built to add information to Wikidata and to synchronize it with source databases. We also demonstrate several use cases for Wikidata, including the crowdsourced curation of biomedical ontologies, phenotype-based diagnosis of disease, and drug repurposing.

Predicting instances of pathway ontology classes for pathway integration

BackgroundTo improve the outcomes of biological pathway analysis, a better way of integrating pathway data is needed. Ontologies can be used to organize data from disparate sources, and we leverage the Pathway Ontology as a unifying ontology for organizing pathway data. We aim to associate pathway instances from different databases to the appropriate class in the Pathway Ontology.ResultsUsing a supervised machine learning approach, we trained neural networks to predict mappings between Reactome pathways and Pathway Ontology (PW) classes. For 2222 Reactome classes, the neural network (NN) model generated 10,952 class recommendations. We compared against a baseline bag-of-words (BOW) model for predicting correct PW classes. A 5% subset of Reactome pathways (111 pathways) was randomly selected, and the corresponding class recommendations from both models were evaluated by two curators. The precision of the BOW model was higher (0.49 for BOW and 0.39 for NN), but the recall was lower (0.42 for BOW and 0.78 for NN). Around 78% of Reactome pathways received pertinent recommendations from the NN model.ConclusionsThe neural predictive model produced meaningful class recommendations that assisted PW curators in selecting appropriate class mappings for Reactome pathways. Our methods can be used to reduce the manual effort associated with ontology curation, and more broadly, for augmenting the curators’ ability to organize and integrate data from pathway databases using the Pathway Ontology.

Sensing the cilium, digital capture of ciliary data for comparative genomics investigations

BackgroundCilia are specialized, hair-like structures that project from the cell bodies of eukaryotic cells. With increased understanding of the distribution and functions of various types of cilia, interest in these organelles is accelerating. To effectively use this great expansion in knowledge, this information must be made digitally accessible and available for large-scale analytical and computational investigation. Capture and integration of knowledge about cilia into existing knowledge bases, thus providing the ability to improve comparative genomic data analysis, is the objective of this work.MethodsWe focused on the capture of information about cilia as studied in the laboratory mouse, a primary model of human biology. The workflow developed establishes a standard for capture of comparative functional data relevant to human biology. We established the 310 closest mouse orthologs of the 302 human genes defined in the SYSCILIA Gold Standard set of ciliary genes. For the mouse genes, we identified biomedical literature for curation and used Gene Ontology (GO) curation paradigms to provide functional annotations from these publications.ResultsEmploying a methodology for comprehensive capture of experimental data about cilia genes in structured, digital form, we established a workflow for curation of experimental literature detailing molecular function and roles of cilia proteins starting with the mouse orthologs of the human SYSCILIA gene set. We worked closely with the GO Consortium ontology development editors and the SYSCILIA Consortium to improve the representation of ciliary biology within the GO. During the time frame of the ontology improvement project, we have fully curated 134 of these 310 mouse genes, resulting in an increase in the number of ciliary and other experimental annotations.ConclusionsWe have improved the GO annotations available for mouse genes orthologous to the human genes in the SYSCILIA Consortium’s Gold Standard set. In addition, ciliary terminology in the GO itself was improved in collaboration with GO ontology developers and the SYSCILIA Consortium. These improvements to the GO terms for the functions and roles of ciliary proteins, along with the increase in annotations of the corresponding genes, enhance the representation of ciliary processes and localizations and improve access to these data during large-scale bioinformatic analyses.

Text Mining to Support Gene Ontology Curation and Vice Versa.

In this chapter, we explain how text mining can support the curation of molecular biology databases dealing with protein functions. We also show how curated data can play a disruptive role in the developments of text mining methods. We review a decade of efforts to improve the automatic assignment of Gene Ontology (GO) descriptors, the reference ontology for the characterization of genes and gene products. To illustrate the high potential of this approach, we compare the performances of an automatic text categorizer and show a large improvement of +225 % in both precision and recall on benchmarked data. We argue that automatic text categorization functions can ultimately be embedded into a Question-Answering (QA) system to answer questions related to protein functions. Because GO descriptors can be relatively long and specific, traditional QA systems cannot answer such questions. A new type of QA system, so-called Deep QA which uses machine learning methods trained with curated contents, is thus emerging. Finally, future advances of text mining instruments are directly dependent on the availability of high-quality annotated contents at every curation step. Databases workflows must start recording explicitly all the data they curate and ideally also some of the data they do not curate.

Supporting the analysis of ontology evolution processes through the combination of static and dynamic scaling functions in OQuaRE.

BackgroundThe biomedical community has now developed a significant number of ontologies. The curation of biomedical ontologies is a complex task and biomedical ontologies evolve rapidly, so new versions are regularly and frequently published in ontology repositories. This has the implication of there being a high number of ontology versions over a short time span. Given this level of activity, ontology designers need to be supported in the effective management of the evolution of biomedical ontologies as the different changes may affect the engineering and quality of the ontology. This is why there is a need for methods that contribute to the analysis of the effects of changes and evolution of ontologies.ResultsIn this paper we approach this issue from the ontology quality perspective. In previous work we have developed an ontology evaluation framework based on quantitative metrics, called OQuaRE. Here, OQuaRE is used as a core component in a method that enables the analysis of the different versions of biomedical ontologies using the quality dimensions included in OQuaRE. Moreover, we describe and use two scales for evaluating the changes between the versions of a given ontology. The first one is the static scale used in OQuaRE and the second one is a new, dynamic scale, based on the observed values of the quality metrics of a corpus defined by all the versions of a given ontology (life-cycle). In this work we explain how OQuaRE can be adapted for understanding the evolution of ontologies. Its use has been illustrated with the ontology of bioinformatics operations, types of data, formats, and topics (EDAM).ConclusionsThe two scales included in OQuaRE provide complementary information about the evolution of the ontologies. The application of the static scale, which is the original OQuaRE scale, to the versions of the EDAM ontology reveals a design based on good ontological engineering principles. The application of the dynamic scale has enabled a more detailed analysis of the evolution of the ontology, measured through differences between versions. The statistics of change based on the OQuaRE quality scores make possible to identify key versions where some changes in the engineering of the ontology triggered a change from the OQuaRE quality perspective. In the case of the EDAM, this study let us to identify that the fifth version of the ontology has the largest impact in the quality metrics of the ontology, when comparative analyses between the pairs of consecutive versions are performed.

OntoBrowser: a collaborative tool for curation of ontologies by subject matter experts.

SummaryThe lack of controlled terminology and ontology usage leads to incomplete search results and poor interoperability between databases. One of the major underlying challenges of data integration is curating data to adhere to controlled terminologies and/or ontologies. Finding subject matter experts with the time and skills required to perform data curation is often problematic. In addition, existing tools are not designed for continuous data integration and collaborative curation. This results in time-consuming curation workflows that often become unsustainable. The primary objective of OntoBrowser is to provide an easy-to-use online collaborative solution for subject matter experts to map reported terms to preferred ontology (or code list) terms and facilitate ontology evolution. Additional features include web service access to data, visualization of ontologies in hierarchical/graph format and a peer review/approval workflow with alerting.Availability and implementationThe source code is freely available under the Apache v2.0 license. Source code and installation instructions are available at http://opensource.nibr.com. This software is designed to run on a Java EE application server and store data in a relational database.

Guidelines for the functional annotation of microRNAs using the Gene Ontology.

MicroRNA regulation of developmental and cellular processes is a relatively new field of study, and the available research data have not been organized to enable its inclusion in pathway and network analysis tools. The association of gene products with terms from the Gene Ontology is an effective method to analyze functional data, but until recently there has been no substantial effort dedicated to applying Gene Ontology terms to microRNAs. Consequently, when performing functional analysis of microRNA data sets, researchers have had to rely instead on the functional annotations associated with the genes encoding microRNA targets. In consultation with experts in the field of microRNA research, we have created comprehensive recommendations for the Gene Ontology curation of microRNAs. This curation manual will enable provision of a high-quality, reliable set of functional annotations for the advancement of microRNA research. Here we describe the key aspects of the work, including development of the Gene Ontology to represent this data, standards for describing the data, and guidelines to support curators making these annotations. The full microRNA curation guidelines are available on the GO Consortium wiki (http://wiki.geneontology.org/index.php/MicroRNA_GO_annotation_manual).

Capturing domain knowledge from multiple sources: the rare bone disorders use case.

BackgroundLately, ontologies have become a fundamental building block in the process of formalising and storing complex biomedical information. The community-driven ontology curation process, however, ignores the possibility of multiple communities building, in parallel, conceptualisations of the same domain, and thus providing slightly different perspectives on the same knowledge. The individual nature of this effort leads to the need of a mechanism to enable us to create an overarching and comprehensive overview of the different perspectives on the domain knowledge.ResultsWe introduce an approach that enables the loose integration of knowledge emerging from diverse sources under a single coherent interoperable resource. To accurately track the original knowledge statements, we record the provenance at very granular levels. We exemplify the approach in the rare bone disorders domain by proposing the Rare Bone Disorders Ontology (RBDO). Using RBDO, researchers are able to answer queries, such as: “What phenotypes describe a particular disorder and are common to all sources?” or to understand similarities between disorders based on divergent groupings (classifications) provided by the underlying sources.AvailabilityRBDO is available at http://purl.org/skeletome/rbdo. In order to support lightweight query and integration, the knowledge captured by RBDO has also been made available as a SPARQL Endpoint at http://bio-lark.org/se_skeldys.html.

Integrating information retrieval with distant supervision for gene ontology annotation.

This article describes our participation of the Gene Ontology Curation task (GO task) in BioCreative IV where we participated in both subtasks: A) identification of GO evidence sentences (GOESs) for relevant genes in full-text articles and B) prediction of GO terms for relevant genes in full-text articles. For subtask A, we trained a logistic regression model to detect GOES based on annotations in the training data supplemented with more noisy negatives from an external resource. Then, a greedy approach was applied to associate genes with sentences. For subtask B, we designed two types of systems: (i) search-based systems, which predict GO terms based on existing annotations for GOESs that are of different textual granularities (i.e., full-text articles, abstracts, and sentences) using state-of-the-art information retrieval techniques (i.e., a novel application of the idea of distant supervision) and (ii) a similarity-based system, which assigns GO terms based on the distance between words in sentences and GO terms/synonyms. Our best performing system for subtask A achieves an F1 score of 0.27 based on exact match and 0.387 allowing relaxed overlap match. Our best performing system for subtask B, a search-based system, achieves an F1 score of 0.075 based on exact match and 0.301 considering hierarchical matches. Our search-based systems for subtask B significantly outperformed the similarity-based system.Database URL: https://github.com/noname2020/Bioc

Annotation of phenotypic diversity: decoupling data curation and ontology curation using Phenex

BackgroundPhenex (http://phenex.phenoscape.org/) is a desktop application for semantically annotating the phenotypic character matrix datasets common in evolutionary biology. Since its initial publication, we have added new features that address several major bottlenecks in the efficiency of the phenotype curation process: allowing curators during the data curation phase to provisionally request terms that are not yet available from a relevant ontology; supporting quality control against annotation guidelines to reduce later manual review and revision; and enabling the sharing of files for collaboration among curators.ResultsWe decoupled data annotation from ontology development by creating an Ontology Request Broker (ORB) within Phenex. Curators can use the ORB to request a provisional term for use in data annotation; the provisional term can be automatically replaced with a permanent identifier once the term is added to an ontology. We added a set of annotation consistency checks to prevent common curation errors, reducing the need for later correction. We facilitated collaborative editing by improving the reliability of Phenex when used with online folder sharing services, via file change monitoring and continual autosave.ConclusionsWith the addition of these new features, and in particular the Ontology Request Broker, Phenex users have been able to focus more effectively on data annotation. Phenoscape curators using Phenex have reported a smoother annotation workflow, with much reduced interruptions from ontology maintenance and file management issues.

Annotation of phenotypic diversity: decoupling data curation and ontology curation using Phenex

Annotation of phenotypic diversity: decoupling data curation and ontology curation using Phenex