Concept Annotation Research Articles

Exploring Patient-Generated Annotations to Digital Clinical Symptom Measures for Patient-Centered Communication

Patients' self-reports are crucial for effective care management of clinical conditions involving subjective symptoms. While patients often value the ability to bring in different forms of self-report data to convey their lived experiences, they often struggle to make their data practically usable in clinical settings. To better center patient needs in communicating illness experiences in clinical contexts, we explore the idea of patient annotations to digital clinical self-report measures, specifically in the context of discontinuing antidepressants. Through interviews with 20 patients with AT Annotator, a digital aid to introduce the concept of annotations, we found that participants perceived annotations to digital clinical measures as a means to enrich self-report measures and reduce the cognitive and emotional burden of logging. However, concerns were raised regarding potential disruptions in patient-provider relationships and the sensitive and complex nature of mental health contexts. We discuss opportunities for annotations to promote patient-centered communication by balancing with clinical practicality and incorporating customization support for patients' communication needs.

CACER: Clinical concept Annotations for Cancer Events and Relations.

Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. We present Clinical concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48000 medical problems and drug events and 10000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction models such as Bidirectional Encoder Representations from Transformers (BERT), Fine-tuned Language Net Text-To-Text Transfer Transformer (Flan-T5), Large Language Model Meta AI (Llama3), and Generative Pre-trained Transformers-4 (GPT-4) using fine-tuning and in-context learning (ICL). In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, large language models offer no performance advantage over the smaller BERT models. We introduce CACER, a novel corpus with fine-grained annotations for medical problems, drugs, and their relationships in clinical narratives of oncology notes. State-of-the-art transformer models achieved performance comparable to IAA for several extraction tasks.

DiSSCover the Potential of FAIR Digital Object Annotations and How You Can Use Them!

The infrastructure for the Distributed System of Scientific Collections (DiSSCo) is in full development. Work within the DiSSCo Transition Project has been focused on building infrastructure, creating data models, and setting up Application Programming Interfaces (APIs) (Koureas et al. 2024). In the past years, DiSSCo has presented this work at different Biodiversity Information Standards (TDWG) conferences (Leeflang and Addink 2023, Leeflang et al. 2022, Addink et al. 2021). In this year’s session, we would like to focus on the human-facing application: DiSSCover. DiSSCover is the graphical user interface through which users can interact with Findable, Accessible, Interoperable and Reusable (FAIR) Digital Objects (FDOs), facilitating the curation and enhancement of specimen data (Islam 2024). Development started in 2022 and is ongoing. The interface acts as a gateway into the DiSSCo infrastructure, providing access to digital specimens and media. Extracted from the core DiSSCo API, the data is converted into an easily readable format and made discoverable through a diverse set of filters. DiSSCover’s main focus is to allow users to make annotations upon the data. Through the concept of annotations, we connect expert and machine-generated information, to create extended digital specimens (Hardisty et al. 2022), e.g, by creating linkages to other infrastructures, correcting or adding new information, or by triggering machine annotation services. Machine annotation services are automated scalable tools that run in the background and automatically curate and extend the specimen (Addink et al. 2023). Human users will remain important, as all annotations made by machine annotation services can be reviewed by a trusted person. Annotations are Fair Digital Objects and target a specific part of a specimen, be it a data fragment or an associated media file. At the heart of DiSSCover lies the Open Digital Specimen data specification (Leeflang and Addink 2023). It tries to harmonise multiple data standards into one generic specification based on the new Global Biodiversity Information Facility (GBIF) Unified Model (Robertson et al. 2022). The data is stored as JavaScript Object Notation (JSON) based on JSON Schemas (Anonymous 2024). Annotations are linked to specific data attributes using a JSON-path as the identifier. Data attributes can be individual terms, collections of terms called classes, or the whole object. This creates a flexible but complex data structure, the basis for which we used the World Wide Web Consortium (W3C) web annotation data model (Sanderson et al. 2017). The W3C Web annotation data model contains two main components: the target and body. The target specifies which data attribute the annotation is made on, for example, the term: ‘ods:specimenName’. This is a local term within the open Digital Specimen namespace (ods), which holds the accepted name of the digital specimen. The annotation body holds the value(s) that are appended to the digital specimen, and differ based upon the annotation motivation. DiSSCo recognises five different annotation motivations: addition, modification, comment, assessment and deletion, each of which has its own unique function. This creates a flexible structure that should be able to handle any information the user wants to add to the object. The challenge of DiSSCover is to preserve the complex structure of annotations, whilst making it convenient for users to work with. The session will provide a look at the different kinds of annotations and their use from a practical perspective. A demonstration of DiSSCover will show how users can create annotations, providing knowledge about the process that will give shape to DiSSCo’s main goal of enriching natural history data.

Foresight—a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study

An electronic health record (EHR) holds detailed longitudinal information about a patient's health status and general clinical history, a large portion of which is stored as unstructured, free text. Existing approaches to model a patient's trajectory focus mostly on structured data and a subset of single-domain outcomes. This study aims to evaluate the effectiveness of Foresight, a generative transformer in temporal modelling of patient data, integrating both free text and structured formats, to predict a diverse array of future medical outcomes, such as disorders, substances (eg, to do with medicines, allergies, or poisonings), procedures, and findings (eg, relating to observations, judgements, or assessments). Foresight is a novel transformer-based pipeline that uses named entity recognition and linking tools to convert EHR document text into structured, coded concepts, followed by providing probabilistic forecasts for future medical events, such as disorders, substances, procedures, and findings. The Foresight pipeline has four main components: (1) CogStack (data retrieval and preprocessing); (2) the Medical Concept Annotation Toolkit (structuring of the free-text information from EHRs); (3) Foresight Core (deep-learning model for biomedical concept modelling); and (4) the Foresight web application. We processed the entire free-text portion from three different hospital datasets (King's College Hospital [KCH], South London and Maudsley [SLaM], and the US Medical Information Mart for Intensive Care III [MIMIC-III]), resulting in information from 811 336 patients and covering both physical and mental health institutions. We measured the performance of models using custom metrics derived from precision and recall. Foresight achieved a precision@10 (ie, of 10 forecasted candidates, at least one is correct) of 0·68 (SD 0·0027) for the KCH dataset, 0·76 (0·0032) for the SLaM dataset, and 0·88 (0·0018) for the MIMIC-III dataset, for forecasting the next new disorder in a patient timeline. Foresight also achieved a precision@10 value of 0·80 (0·0013) for the KCH dataset, 0·81 (0·0026) for the SLaM dataset, and 0·91 (0·0011) for the MIMIC-III dataset, for forecasting the next new biomedical concept. In addition, Foresight was validated on 34 synthetic patient timelines by five clinicians and achieved a relevancy of 33 (97% [95% CI 91-100]) of 34 for the top forecasted candidate disorder. As a generative model, Foresight can forecast follow-on biomedical concepts for as many steps as required. Foresight is a general-purpose model for biomedical concept modelling that can be used for real-world risk forecasting, virtual trials, and clinical research to study the progression of disorders, to simulate interventions and counterfactuals, and for educational purposes. National Health Service Artificial Intelligence Laboratory, National Institute for Health and Care Research Biomedical Research Centre, and Health Data Research UK.

MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment

Black-box deep learning approaches have showcased significant potential in the realm of medical image analysis. However, the stringent trustworthiness requirements intrinsic to the medical field have catalyzed research into the utilization of Explainable Artificial Intelligence (XAI), with a particular focus on concept-based methods. Existing concept-based methods predominantly apply concept annotations from a single perspective (e.g., global level), neglecting the nuanced semantic relationships between sub-regions and concepts embedded within medical images. This leads to underutilization of the valuable medical information and may cause models to fall short in harmoniously balancing interpretability and performance when employing inherently interpretable architectures such as Concept Bottlenecks. To mitigate these shortcomings, we propose a multi-modal explainable disease diagnosis framework that meticulously aligns medical images and clinical-related concepts semantically at multiple strata, encompassing the image level, token level, and concept level. Moreover, our method allows for model intervention and offers both textual and visual explanations in terms of human-interpretable concepts. Experimental results on three skin image datasets demonstrate that our method, while preserving model interpretability, attains high performance and label efficiency for concept detection and disease diagnosis. The code is available at https://github.com/Tommy-Bie/MICA.

Unsupervised SapBERT-based bi-encoders for medical concept annotation of clinical narratives with SNOMED CT

Objective Clinical narratives provide comprehensive patient information. Achieving interoperability involves mapping relevant details to standardized medical vocabularies. Typically, natural language processing divides this task into named entity recognition (NER) and medical concept normalization (MCN). State-of-the-art results require supervised setups with abundant training data. However, the limited availability of annotated data due to sensitivity and time constraints poses challenges. This study addressed the need for unsupervised medical concept annotation (MCA) to overcome these limitations and support the creation of annotated datasets. Method We use an unsupervised SapBERT-based bi-encoder model to analyze n-grams from narrative text and measure their similarity to SNOMED CT concepts. At the end, we apply a syntactical re-ranker. For evaluation, we use the semantic tags of SNOMED CT candidates to assess the NER phase and their concept IDs to assess the MCN phase. The approach is evaluated with both English and German narratives. Result Without training data, our unsupervised approach achieves an F1 score of 0.765 in English and 0.557 in German for MCN. Evaluation at the semantic tag level reveals that “disorder” has the highest F1 scores, 0.871 and 0.648 on English and German datasets. Furthermore, the MCA approach on the semantic tag “disorder” shows F1 scores of 0.839 and 0.696 in English and 0.685 and 0.437 in German for NER and MCN, respectively. Conclusion This unsupervised approach demonstrates potential for initial annotation (pre-labeling) in manual annotation tasks. While promising for certain semantic tags, challenges remain, including false positives, contextual errors, and variability of clinical language, requiring further fine-tuning.

Enhanced Cardiovascular Risk Assessment Incorporating Natural Language Processing and Qrisk-3 Score Determination in Patients with Essential Thrombocythaemia

Introduction: Fundamental to management of patients with essential thrombocythaemia (ET) is assessment, and reduction of thrombotic risk. We present a machine learning approach to summarise patient electronic health records (EHR) to determine prevalence of cardiovascular comorbidities and risk factors. We then review use of the QRISK-3 score to assess cardiovascular risk. Methods: We used a natural language processing (NLP) pipeline to identify mentions of hypertension (HTN), hypercholesterolaemia (HC), diabetes mellitus (DM), smoking (SM), unspecified thrombosis (VTE), deep vein thrombosis (DVT), pulmonary embolism (PE), portal vein thrombosis (PVT), myocardial infarction (MI) and stroke (CVA) in EHR. CogStack is an information retrieval and extraction architecture incorporating structured and unstructured EHR components. Data extracted from CogStack was processed by a medical concept annotation toolkit (MedCAT). MedCAT was used to disambiguate and capture synonyms and acronyms for Systematized Nomenclature of Medicine-Clinical Terms (SNOMED-CT) concepts. Using deep learning MedCAT determined linguistic and grammatical context such as negation, subject, and temporality. The base MedCAT model was trained in an unsupervised manner on >18 million EHR documents and this was further fine-tuned through 500 clinician annotated haematology documents. MedCAT mapped mentions of relevant concepts to respective SNOMED-CT codes and total counts were aggregated and grouped by individual patient. Manual validations were performed and an optimizer was applied to convert counts to a binary state by applying a threshold, above which a patient's condition was inferred to be present (Fig. i). QRISK-3 is an advanced validated score incorporating age, ethnicity, body mass index and other cardiovascular risk factors to determine 10-year cardiovascular risk in people aged 25-84. Results: 12905 documents from 560 ET patients were reviewed (median 20 per patient, IQR 8-34). In the manual validation dataset (n=120), MedCAT achieved excellent real-world F1 scores (model accuracy) for most concepts (HTN 0.91, HC 0.81, DM 1.0, VTE 0.73, CVA 0.87 and MI 0.67). Using a threshold of >2 mentions to define a positive population; HTN was identified in 21.3% (119) of patients, DM in 4.6% (26), MI in 3.6% (20), CVA in 7.7% (43), VTE in 8% (45), DVT in 1.4% (8), PE in 1.8% (10), PVT in 1.3% (7) and positive smoking status in 6.6% (37). HC was identified in 9.6% (54) using a threshold >1. 52% (28) of patients with HC and 69.2% (18) of those with DM also had HTN. Obesity was not identified in any patients using this approach. Patients with a diagnosis of HTN were more likely to have CVA than those without (15:104 vs 28:413, p=0.03). Patients with HTN were also more likely to have VTE (13:106 vs 19:422, p=0.01). Of patients with CVA/MI; 58.1% (25) /55% (11) had this event pre or at diagnosis and 30.2% (13)/ 10% (2) while receiving cytoreductive therapy. QRISK-3 analysis was performed in 32 patients with prior thrombosis and baseline criteria to evaluate predictive value; then 137 patients classified as low or intermediate (LIM) risk and not receiving cytoreductive therapy. Mean QRISK-3 was 8 in the thrombosis group, validating its relevance, and 2.5 (p<0.0001, Fig. ii) in the LIM cohort. Using the recognised QRISK-3 score threshold of >7.5 to define a high-risk population, 5.1% (7) patients from the LIM group were reclassified as high-risk due to additional comorbidities relevant to QRISK-3 including HTN in 8% (11), migraine 7.3% (10), DM 2.2% (3), severe mental illness 2.9% (4) and antipsychotic medication 0.7% (1). Discussion: We describe a novel approach to cardiovascular risk assessment in patients with ET, incorporating machine learning, allowing large volume data analysis, and detailed risk assessment using QRISK-3 scoring. We provide a rare ‘real-world’ report on the prevalence of comorbidities in this group, confirming increased CVA and VTE in patients with HTN. A previous report of 891 patients with ET showed prevalence of 5% for CVA, 2% for MI and 4% for VTE, suggesting that detection rate using our approach is within acceptable limits (Carobbio et al., Blood, 2011). Finally, as a novel finding, we show that QRISK-3 scoring is predictive of increased thrombotic risk and identifies a small group of patients who should be considered high-risk and may benefit from cytoreductive therapy, that are not detected using standard approaches.

A Visualization Method of Knowledge Graphs for the Computation and Comprehension of Ultrasound Reports.

Knowledge graph visualization in ultrasound reports is essential for enhancing medical decision making and the efficiency and accuracy of computer-aided analysis tools. This study aims to propose an intelligent method for analyzing ultrasound reports through knowledge graph visualization. Firstly, we provide a novel method for extracting key term networks from the narrative text in ultrasound reports with high accuracy, enabling the identification and annotation of clinical concepts within the report. Secondly, a knowledge representation framework based on ultrasound reports is proposed, which enables the structured and intuitive visualization of ultrasound report knowledge. Finally, we propose a knowledge graph completion model to address the lack of entities in physicians' writing habits and improve the accuracy of visualizing ultrasound knowledge. In comparison to traditional methods, our proposed approach outperforms the extraction of knowledge from complex ultrasound reports, achieving a significantly higher extraction index (η) of 2.69, surpassing the general pattern-matching method (2.12). In comparison to other state-of-the-art methods, our approach achieves the highest P (0.85), R (0.89), and F1 (0.87) across three testing datasets. The proposed method can effectively utilize the knowledge embedded in ultrasound reports to obtain relevant clinical information and improve the accuracy of using ultrasound knowledge.

Detecting evidence of invasive fungal infections in cytology and histopathology reports enriched with concept-level annotations

Invasive fungal infections (IFIs) are particularly dangerous to high-risk patients with haematological malignancies and are responsible for excessive mortality and delays in cancer therapy. Surveillance of IFI in clinical settings offers an opportunity to identify potential risk factors and evaluate new therapeutic strategies. However, manual surveillance is both time- and resource-intensive. As part of a broader project aimed to develop a system for automated IFI surveillance by leveraging electronic medical records, we present our approach to detecting evidence of IFI in the key diagnostic domain of histopathology. Using natural language processing (NLP), we analysed cytology and histopathology reports to identify IFI-positive reports. We compared a conventional bag-of-words classification model to a method that relies on concept-level annotations. Although the investment to prepare data supporting concept annotations is substantial, extracting targeted information specific to IFI as a pre-processing step increased the performance of the classifier from the PR AUC of 0.84 to 0.92 and enabled model interpretability. We have made publicly available the annotated dataset of 283 reports, the Cytology and Histopathology IFI Reports corpus (CHIFIR), to allow the clinical NLP research community to further build on our results.

Learning from undercoded clinical records for automated International Classification of Diseases (ICD) coding.

To develop an unbiased objective for learning automatic coding algorithms from clinical records annotated with only partial relevant International Classification of Diseases codes, as annotation noise in undercoded clinical records used as training data can mislead the learning process of deep neural networks. We use Medical Information Mart for Intensive Care III as our dataset. We employ positive-unlabeled learning to achieve unbiased loss estimation, which is free of misleading training signal. We then utilize reweighting mechanism to compensate for the imbalance between positive and negative samples. To further close the performance gap caused by poor quality annotation, we integrate the supervision provided by the automatic annotation tool Medical Concept Annotation Toolkit which can ease the heavy burden of manual validation. Our benchmarking results show that positive-unlabeled learning with reweighting outperforms competitive baseline methods over a range of missing label ratios. Integrating supervision provided by annotation tool further boosted the performance. Considering the annotation noise and severe imbalance, unbiased loss estimation and reweighting mechanism are both important for learning from undercoded clinical records. Unbiased loss requires the estimation of false negative ratios and estimation through trained models is practical and competitive. The combination of positive-unlabeled learning with reweighting and supervision provided by the annotation tool is a promising solution to learn from undercoded clinical records.

Interpretable Generative Adversarial Networks

Learning a disentangled representation is still a challenge in the field of the interpretability of generative adversarial networks (GANs). This paper proposes a generic method to modify a traditional GAN into an interpretable GAN, which ensures that filters in an intermediate layer of the generator encode disentangled localized visual concepts. Each filter in the layer is supposed to consistently generate image regions corresponding to the same visual concept when generating different images. The interpretable GAN learns to automatically discover meaningful visual concepts without any annotations of visual concepts. The interpretable GAN enables people to modify a specific visual concept on generated images by manipulating feature maps of the corresponding filters in the layer. Our method can be broadly applied to different types of GANs. Experiments have demonstrated the effectiveness of our method.

Visual Consensus Modeling for Video-Text Retrieval

In this paper, we propose a novel method to mine the commonsense knowledge shared between the video and text modalities for video-text retrieval, namely visual consensus modeling. Different from the existing works, which learn the video and text representations and their complicated relationships solely based on the pairwise video-text data, we make the first attempt to model the visual consensus by mining the visual concepts from videos and exploiting their co-occurrence patterns within the video and text modalities with no reliance on any additional concept annotations. Specifically, we build a shareable and learnable graph as the visual consensus, where the nodes denoting the mined visual concepts and the edges connecting the nodes representing the co-occurrence relationships between the visual concepts. Extensive experimental results on the public benchmark datasets demonstrate that our proposed method, with the ability to effectively model the visual consensus, achieves state-of-the-art performances on the bidirectional video-text retrieval task. Our code is available at https://github.com/sqiangcao99/VCM.

Updating and using the EO4GEO Body of Knowledge for (AI) concept annotation

Abstract. The EO4GEO Body of Knowledge (BoK) serves as a vocabulary for the domain of geoinformation and earth observation, supporting the annotation of online resources. This paper presents how the BoK is designed, maintained and improved. We discuss how the BoK content can be extended, using the example of integrating artificial intelligence (AI) concepts and show how annotation is done by adding persistent concept identifiers in the metadata of training materials. This platform allows us to share online information with clarified semantics. A prolonged use necessitates the incentivisation of an active expert community and a further adoption of infrastructure standards.

Developing a Dietary Lifestyle Ontology to Improve the Interoperability of Dietary Data: Proof-of-Concept Study

BackgroundDietary habits offer crucial information on one's health and form a considerable part of the patient-generated health data. Dietary data are collected through various channels and formats; thus, interoperability is a significant challenge to reusing this type of data. The vast scope of dietary concepts and the colloquial expression style add difficulty to standardizing the data. The interoperability issues of dietary data can be addressed through Common Data Elements with metadata annotation to some extent. However, making culture-specific dietary habits and questionnaire-based dietary assessment data interoperable still requires substantial efforts.ObjectiveThe main goal of this study was to address the interoperability challenge of questionnaire-based dietary data from different cultural backgrounds by combining ontological curation and metadata annotation of dietary concepts. Specifically, this study aimed to develop a Dietary Lifestyle Ontology (DILON) and demonstrate the improved interoperability of questionnaire-based dietary data by annotating its main semantics with DILON.MethodsBy analyzing 1158 dietary assessment data elements (367 in Korean and 791 in English), 515 dietary concepts were extracted and used to construct DILON. To demonstrate the utility of DILON in addressing the interoperability challenges of questionnaire-based multicultural dietary data, we developed 10 competency questions that asked to identify data elements sharing the same dietary topics and assessment properties. We instantiated 68 data elements on dietary habits selected from Korean and English questionnaires and annotated them with DILON to answer the competency questions. We translated the competency questions into Semantic Query-Enhanced Web Rule Language and reviewed the query results for accuracy.ResultsDILON was built with 262 concept classes and validated with ontology validation tools. A small overlap (72 concepts) in the concepts extracted from the questionnaires in 2 languages indicates that we need to pay closer attention to representing culture-specific dietary concepts. The Semantic Query-Enhanced Web Rule Language queries reflecting the 10 competency questions yielded correct results.ConclusionsEnsuring the interoperability of dietary lifestyle data is a demanding task due to its vast scope and variations in expression. This study demonstrated that we could improve the interoperability of dietary data generated in different cultural contexts and expressed in various styles by annotating their core semantics with DILON.

Multi-domain clinical natural language processing with MedCAT: The Medical Concept Annotation Toolkit

Electronic health records (EHR) contain large volumes of unstructured text, requiring the application of information extraction (IE) technologies to enable clinical analysis. We present the open source Medical Concept Annotation Toolkit (MedCAT) that provides: (a) a novel self-supervised machine learning algorithm for extracting concepts using any concept vocabulary including UMLS/SNOMED-CT; (b) a feature-rich annotation interface for customizing and training IE models; and (c) integrations to the broader CogStack ecosystem for vendor-agnostic health system deployment. We show improved performance in extracting UMLS concepts from open datasets (F1:0.448–0.738 vs 0.429–0.650). Further real-world validation demonstrates SNOMED-CT extraction at 3 large London hospitals with self-supervised training over ∼8.8B words from ∼17M clinical records and further fine-tuning with ∼6K clinician annotated examples. We show strong transferability (F1 > 0.94) between hospitals, datasets and concept types indicating cross-domain EHR-agnostic utility for accelerated clinical and research use cases.

Data modeling and evaluation of deep semantic annotation for cultural heritage images

PurposeTo meet the emerging demand for fine-grained annotation and semantic enrichment of cultural heritage images, this paper proposes a new approach that can transcend the boundary of information organization theory and Panofsky's iconography theory.Design/methodology/approachAfter a systematic review of semantic data models for organizing cultural heritage images and a comparative analysis of the concept and characteristics of deep semantic annotation (DSA) and indexing, an integrated DSA framework for cultural heritage images as well as its principles and process was designed. Two experiments were conducted on two mural images from the Mogao Caves to evaluate the DSA framework's validity based on four criteria: depth, breadth, granularity and relation.FindingsResults showed the proposed DSA framework included not only image metadata but also represented the storyline contained in the images by integrating domain terminology, ontology, thesaurus, taxonomy and natural language description into a multilevel structure.Originality/valueDSA can reveal the aboutness, ofness and isness information contained within images, which can thus meet the demand for semantic enrichment and retrieval of cultural heritage images at a fine-grained level. This method can also help contribute to building a novel infrastructure for the increasing scholarship of digital humanities.

A method for archaeological and dendrochronological concept annotation using domain knowledge in information extraction

A method for archaeological and dendrochronological concept annotation using domain knowledge in information extraction

A method for archaeological and dendrochronological concept annotation using domain knowledge in information extraction

Advances in Natural Language Processing allow the process of deriving information from large volumes of text to be automated. Attention is turned to one of the most important, but traditionally difficult to access resources in archaeology, commonly known as 'grey literature'. This paper presents the development of two separate Named-Entity Recognition (NER) pipelines aimed at the extraction of Archaeological and of Dendrochronological concepts in Dutch, respectively. The role of domain vocabulary is discussed for the development of a Knowledge Organisation System (KOS)-driven, Rule-Based method of NER which makes complementary use of ontology, thesauri and domain vocabulary for information extraction and attribute assignment of semantic annotations. The NER task is challenged by a series of domain and language-oriented aspects and evaluated against a human-annotated Gold Standard. The results suggest the suitability of Rule-based KOS driven approaches for attaining the low-hanging fruits of NER, using a combination of quality vocabulary and rules.

Affect in Multimedia: Benchmarking Violent Scenes Detection

In this article, we report on the creation of a publicly available, common evaluation framework for Violent Scenes Detection (VSD) in Hollywood and YouTube videos. We propose a robust data set, the VSD96, with more than 96 hours of video of various genres, annotations at different levels of detail (e.g., shot-level, segment-level), annotations of mid-level concepts (e.g., blood, fire), various pre-computed multi-modal descriptors, and over 230 system output results as baselines. This is the most comprehensive data set available to this date tailored to the VSD task and was extensively validated during the MediaEval benchmarking campaigns. Furthermore, we provide an in-depth analysis of the crucial components of VSD algorithms, by reviewing the capabilities and the evolution of existing systems (e.g., overall trends and outliers, the influence of the employed features and fusion techniques, the influence of deep learning approaches). Finally, we discuss the possibility of going beyond state-of-the-art performance via an ad-hoc late fusion approach. Experimentation is carried out on the VSD96 data. We provide the most important lessons learned and gained insights. The increasing number of publications using the VSD96 data underline the importance of the topic. The presented and published resources are a practitioner's guide and also a strong baseline to overcome, which will help researchers for the coming years in analyzing aspects of audio-visual affect and violence detection in movies and videos.

Characterizing Chronic Pain Episodes in Clinical Text at Two Health Care Systems: Comprehensive Annotation and Corpus Analysis.

BackgroundChronic pain affects more than 20% of adults in the United States and is associated with substantial physical, mental, and social burden. Clinical text contains rich information about chronic pain, but no systematic appraisal has been performed to assess the electronic health record (EHR) narratives for these patients. A formal content analysis of the unstructured EHR data can inform clinical practice and research in chronic pain.ObjectiveWe characterized individual episodes of chronic pain by annotating and analyzing EHR notes for a stratified cohort of adults with known chronic pain.MethodsWe used the Rochester Epidemiology Project infrastructure to screen all residents of Olmsted County, Minnesota, for evidence of chronic pain, between January 1, 2005, and September 30, 2015. Diagnosis codes were used to assemble a cohort of 6586 chronic pain patients; people with cancer were excluded. The records of an age- and sex-stratified random sample of 62 patients from the cohort were annotated using an iteratively developed guideline. The annotated concepts included date, location, severity, causes, effects on quality of life, diagnostic procedures, medications, and other treatment modalities.ResultsA total of 94 chronic pain episodes from 62 distinct patients were identified by reviewing 3272 clinical notes. Documentation was written by clinicians across a wide spectrum of specialties. Most patients (40/62, 65%) had 1 pain episode during the study period. Interannotator agreement ranged from 0.78 to 1.00 across the annotated concepts. Some pain-related concepts (eg, body location) had 100% (94/94) coverage among all the episodes, while others had moderate coverage (eg, effects on quality of life) (55/94, 59%). Back pain and leg pain were the most common types of chronic pain in the annotated cohort. Musculoskeletal issues like arthritis were annotated as the most common causes. Opioids were the most commonly captured medication, while physical and occupational therapies were the most common nonpharmacological treatments.ConclusionsWe systematically annotated chronic pain episodes in clinical text. The rich content analysis results revealed complexity of the chronic pain episodes and of their management, as well as the challenges in extracting pertinent information, even for humans. Despite the pilot study nature of the work, the annotation guideline and corpus should be able to serve as informative references for other institutions with shared interest in chronic pain research using EHRs.