Entity Coreference Resolution Research Articles

The evaluation of a semi-automatic authoring tool for knowledge extraction in the AC&NL Tutor

This paper describes and evaluates the performance of a semi-automatic authoring tool (SAAT) for knowledge extraction in the AC&NL Tutor, highlighting its strengths and weaknesses. We assessed the accuracy of automatic annotation tasks (Part-of-Speech tagging, Name Entity Recognition, Dependency parsing, and Coreference Resolution) performed on a dataset of 160 sentences from unstructured Wikipedia text on a computer. We compared the automatic annotations to the gold standard, created after human post-editing and validation. Human-error analysis included 3769 words, 582 subsentences, 1129 questions, 917 propositions, 1020 concepts, and 667 relations. It resulted in the error type classification and the set of custom rules further used for automatic error identification and correction. The results showed that an average of 68.7% of the error corrections referred to CoreNLP performance and 31.3% to the SAAT extraction algorithms. Our main contributions include an integrated approach to the comprehensive pre-processing of the text, knowledge extraction and visualization; the consolidated evaluation of natural language processing tasks and knowledge extraction output (sentences, subsentences, questions, concept maps) and the newly developed reference dataset. 

Multilingual Coreference Resolution in Multiparty Dialogue

AbstractExisting multiparty dialogue datasets for entity coreference resolution are nascent, and many challenges are still unaddressed. We create a large-scale dataset, Multilingual Multiparty Coref (MMC), for this task based on TV transcripts. Due to the availability of gold-quality subtitles in multiple languages, we propose reusing the annotations to create silver coreference resolution data in other languages (Chinese and Farsi) via annotation projection. On the gold (English) data, off-the-shelf models perform relatively poorly on MMC, suggesting that MMC has broader coverage of multiparty coreference than prior datasets. On the silver data, we find success both using it for data augmentation and training from scratch, which effectively simulates the zero-shot cross-lingual setting.

A Novel Threat Intelligence Information Extraction System Combining Multiple Models

The increasing number of cyberattacks has made the cybersecurity situation more serious. Thus, it is urgent to use cyber threat intelligence to deal with the complex and changing cyber environment. However, cyber threat intelligence usually exists in an unstructured form, and a huge amount of data poses a great challenge to security analysts. To this end, this paper proposes a novel threat intelligence information extraction system combining multiple models, which contains four key steps: entity extraction, coreference resolution, relation extraction, and knowledge graph construction. In the entity extraction task, a multihead self-attention mechanism is adopted to extract the dependency relationships between words. In the coreference resolution task, contextual information and mention embedding are fused to improve the mention representation. Meanwhile, features of different dimensions are extracted using a convolutional neural network. In the relation extraction task, additional features such as part of speech, mention width, entity type, and distance of entity pairs are incorporated to improve the embedding representation. Finally, a knowledge graph is constructed to explicitly present entities and their relationships. Experimental results indicate that compared with the baseline model, the F1 score of our model is improved by at least 8.87, 9.82, and 10.56 on entity extraction, coreference resolution, and relation extraction, respectively. The knowledge graph in Neo4j demonstrates the effectiveness of our system.

Constructing public health evidence knowledge graph for decision-making support from COVID-19 literature of modelling study

The needs of mitigating COVID-19 epidemic prompt policymakers to make public health-related decision under the guidelines of science. Tremendous unstructured COVID-19 publications make it challenging for policymakers to obtain relevant evidence. Knowledge graphs (KGs) can formalize unstructured knowledge into structured form and have been used in supporting decision-making recently. Here, we introduce a novel framework that can extract the COVID-19 public health evidence knowledge graph (CPHE-KG) from papers relating to a modelling study. We screen out a corpus of 3096 COVID-19 modelling study papers by performing a literature assessment process. We define a novel annotation schema to construct the COVID-19 modelling study-related IE dataset (CPHIE). We also propose a novel multi-tasks document-level information extraction model SS-DYGIE++ based on the dataset. Leveraging the model on the new corpus, we construct CPHE-KG containing 60,967 entities and 51,140 relations. Finally, we seek to apply our KG to support evidence querying and evidence mapping visualization. Our SS-DYGIE++(SpanBERT) model has achieved a F1 score of 0.77 and 0.55 respectively in document-level entity recognition and coreference resolution tasks. It has also shown high performance in the relation identification task. With evidence querying, our KG can present the dynamic transmissions of COVID-19 pandemic in different countries and regions. The evidence mapping of our KG can show the impacts of variable non-pharmacological interventions to COVID-19 pandemic. Analysis demonstrates the quality of our KG and shows that it has the potential to support COVID-19 policy making in public health.

DWIE: An entity-centric dataset for multi-task document-level information extraction

This paper presents DWIE, the ‘Deutsche Welle corpus for Information Extraction’, a newly created multi-task dataset that combines four main Information Extraction (IE) annotation subtasks: (i) Named Entity Recognition (NER), (ii) Coreference Resolution, (iii) Relation Extraction (RE), and (iv) Entity Linking. DWIE is conceived as an entity-centric dataset that describes interactions and properties of conceptual entities on the level of the complete document. This contrasts with currently dominant mention-driven approaches that start from the detection and classification of named entity mentions in individual sentences. Further, DWIE presented two main challenges when building and evaluating IE models for it. First, the use of traditional mention-level evaluation metrics for NER and RE tasks on entity-centric DWIE dataset can result in measurements dominated by predictions on more frequently mentioned entities. We tackle this issue by proposing a new entity-driven metric that takes into account the number of mentions that compose each of the predicted and ground truth entities. Second, the document-level multi-task annotations require the models to transfer information between entity mentions located in different parts of the document, as well as between different tasks, in a joint learning setting. To realize this, we propose to use graph-based neural message passing techniques between document-level mention spans. Our experiments show an improvement of up to 5.5 F1 percentage points when incorporating neural graph propagation into our joint model. This demonstrates DWIE’s potential to stimulate further research in graph neural networks for representation learning in multi-task IE. We make DWIE publicly available at https://github.com/klimzaporojets/DWIE.

Research on Entity Coreference Resolution Technology Oriented to Military Knowledge Graph

Data processing in military field is in the stage of fusion disambiguation. The way of addressing the same thing is complex and difficult to integrate. Entity coreference resolution can effectively solve this problem. At the same time, entity coreference resolution is also an important part of knowledge fusion stage in the process of building military knowledge graph. This paper studies and implements Chinese coreference resolution from two aspects: Named Entity Recognition and coreference resolution. Firstly, use the BiLSTM+CRF model of neural network to realize NER in military field. By mining Wikipedia corpus, construct a pattern base, and iteratively find the coreference relationship in text based on pattern, and finally establish a model. To complete the rapid and effective construction of a total of 220,000 thesaurus, covering the military field of aircraft and ships two types of objectives, to achieve the military field entity coreference resolution, to provide strong support for the construction of military knowledge graph.

Story Analysis Using Natural Language Processing and Interactive Dashboards

ABSTRACT This paper discusses Story Analyzer, which uses a natural language processing (NLP) library and sophisticated data visualization libraries to produce dashboards of interrelated and user-responsive visualizations depicting actors and their interactions in a textual narrative, along with locations, times, and other contexts. Story Analyzer performs information extraction using Stanford’s CoreNLP’s NLP services including sentence recognition, tokenizing, parts-of-speech identification, dependency parsing, named entity recognition, coreference resolution, and temporal tagging. Visualization is done through D3 and scalable vector graphics (SVG) which provide powerful control over gelements and shapes in browser-based user interfaces. Google Charts and Maps are also used for visualizations. Development using NLP for unstructured textual data involves challenges, limitations, and ambiguities that distinguishes it from applications using structured data. Therefore, the paper also discusses issues and limitations inherent with using NLP libraries, and presents workarounds when applied to story analysis. Story Analyzer is applied to a contemporary news article regarding data privacy issues.

Coreference resolution Ontology-based in sentiment analysis

Entity co-reference resolution and sentiment analysis are independent problems and popular research topics in the community of natural language processing. However, the combination of those two problems has not been getting much attention. Thus, this paper susgests to apply knowledge base to solve co- reference between object and aspect with sentiment. In addition, the paper also proposes the model of Ontology-based co-reference resolution in sentiment analysis for English text. Finally, we also discuss evaluation methods applied for our model and the results obtained.

Combining Information Extraction and Text Segmentation methods in Greek Texts

In this paper we examine the benefit of performing named entity recognition (NER) and co-reference resolution to a Greek corpus used for text segmentation. The aim here is to examine whether the combination of text segmentation and information extraction is beneficial for identifying various topics that appear in a document. NER was performed using an already existing tool for the Greek corpus. Produced annotations were manually corrected and enriched to cover four types of named entities. Co-reference resolution was subsequently performed manually. The evaluation, using four text segmentation algorithms leads to the conclusion that, information extraction techniques appear to be a promising solution in capturing semantic information for segmentation purposes.

EREL: an Entity Recognition and Linking algorithm

ABSTRACTThis paper introduces the EREL algorithm that integrates Entity Recognition, Co-reference Resolution (CR) and Disambiguation. The algorithm recognizes entity mentions as the longest name based on the name dictionary constructed from the Wikipedia data. The CR is integrated into the algorithm to improve the performance in processing short-form or abbreviated names. The algorithm employs a new approach in disambiguation entities using new features as entity-level context information and case-sensitive data about the mention in disambiguation. Tested on four benchmark data sets in the GERBIL framework, EREL outperforms the current Entity Linking methods. EREL achieves the micro f-score as 0.83 in both tasks Disambiguate to Wikipedia and Annotate to Wikipedia.

Applying named entity recognition and co-reference resolution for segmenting English texts

In this paper we examine the benefit of performing named entity recognition (NER) and co-reference resolution to an English corpus used for text segmentation. The aim here is to examine whether the combination of text segmentation and information extraction can be beneficial for the identification of the various topics that appear in a document. NER was performed manually in the English corpus and was compared with the output produced by publicly available annotation tools. Produced annotations were manually corrected and enriched to cover four types of named entities (person, location, date and group of names). Co-reference resolution, i.e., substitution of every reference of the same instance with the same named entity identifier was subsequently performed. The evaluation, using five text segmentation algorithms leads to the conclusion that, manual annotation appears to be a promising solution due to the obscure contribution of publicly available tools.

Machine Learning for Entity Coreference Resolution: A Retrospective Look at Two Decades of Research

Though extensively investigated since the 1960s, entity coreference resolution, a core task in natural language understanding, is far from being solved. Nevertheless, significant progress has been made on learning-based coreference research since its inception two decades ago. This paper provides an overview of the major milestones made in learning-based coreference research and discusses a hard entity coreference task, the Winograd Schema Challenge, which has recently received a lot of attention in the AI community.

Generalising semantic category disambiguation with large lexical resources for fun and profit.

BackgroundSemantic Category Disambiguation (SCD) is the task of assigning the appropriate semantic category to given spans of text from a fixed set of candidate categories, for example Protein to “Fibrin”. SCD is relevant to Natural Language Processing tasks such as Named Entity Recognition, coreference resolution and coordination resolution. In this work, we study machine learning-based SCD methods using large lexical resources and approximate string matching, aiming to generalise these methods with regard to domains, lexical resources and the composition of data sets. We specifically consider the applicability of SCD for the purposes of supporting human annotators and acting as a pipeline component for other Natural Language Processing systems.ResultsWhile previous research has mostly cast SCD purely as a classification task, we consider a task setting that allows for multiple semantic categories to be suggested, aiming to minimise the number of suggestions while maintaining high recall. We argue that this setting reflects aspects which are essential for both a pipeline component and when supporting human annotators. We introduce an SCD method based on a recently introduced machine learning-based system and evaluate it on 15 corpora covering biomedical, clinical and newswire texts and ranging in the number of semantic categories from 2 to 91.With appropriate settings, our system maintains an average recall of 99% while reducing the number of candidate semantic categories on average by 65% over all data sets.ConclusionsMachine learning-based SCD using large lexical resources and approximate string matching is sensitive to the selection and granularity of lexical resources, but generalises well to a wide range of text domains and data sets given appropriate resources and parameter settings. By substantially reducing the number of candidate categories while only very rarely excluding the correct one, our method is shown to be applicable to manual annotation support tasks and use as a high-recall component in text processing pipelines. The introduced system and all related resources are freely available for research purposes at: https://github.com/ninjin/simsem.

Biological network extraction from scientific literature: state of the art and challenges

Networks of molecular interactions explain complex biological processes, and all known information on molecular events is contained in a number of public repositories including the scientific literature. Metabolic and signalling pathways are often viewed separately, even though both types are composed of interactions involving proteins and other chemical entities. It is necessary to be able to combine data from all available resources to judge the functionality, complexity and completeness of any given network overall, but especially the full integration of relevant information from the scientific literature is still an ongoing and complex task. Currently, the text-mining research community is steadily moving towards processing the full body of the scientific literature by making use of rich linguistic features such as full text parsing, to extract biological interactions. The next step will be to combine these with information from scientific databases to support hypothesis generation for the discovery of new knowledge and the extension of biological networks. The generation of comprehensive networks requires technologies such as entity grounding, coordination resolution and co-reference resolution, which are not fully solved and are required to further improve the quality of results. Here, we analyse the state of the art for the extraction of network information from the scientific literature and the evaluation of extraction methods against reference corpora, discuss challenges involved and identify directions for future research.

BioNLP Shared Task--The Bacteria Track.

BackgroundWe present the BioNLP 2011 Shared Task Bacteria Track, the first Information Extraction challenge entirely dedicated to bacteria. It includes three tasks that cover different levels of biological knowledge. The Bacteria Gene Renaming supporting task is aimed at extracting gene renaming and gene name synonymy in PubMed abstracts. The Bacteria Gene Interaction is a gene/protein interaction extraction task from individual sentences. The interactions have been categorized into ten different sub-types, thus giving a detailed account of genetic regulations at the molecular level. Finally, the Bacteria Biotopes task focuses on the localization and environment of bacteria mentioned in textbook articles.We describe the process of creation for the three corpora, including document acquisition and manual annotation, as well as the metrics used to evaluate the participants' submissions.ResultsThree teams submitted to the Bacteria Gene Renaming task; the best team achieved an F-score of 87%. For the Bacteria Gene Interaction task, the only participant's score had reached a global F-score of 77%, although the system efficiency varies significantly from one sub-type to another. Three teams submitted to the Bacteria Biotopes task with very different approaches; the best team achieved an F-score of 45%. However, the detailed study of the participating systems efficiency reveals the strengths and weaknesses of each participating system.ConclusionsThe three tasks of the Bacteria Track offer participants a chance to address a wide range of issues in Information Extraction, including entity recognition, semantic typing and coreference resolution. We found commond trends in the most efficient systems: the systematic use of syntactic dependencies and machine learning. Nevertheless, the originality of the Bacteria Biotopes task encouraged the use of interesting novel methods and techniques, such as term compositionality, scopes wider than the sentence.

Exploiting information extraction techniques for automatic semantic video indexing with an application to Turkish news videos

This paper targets at the problem of automatic semantic indexing of news videos by presenting a video annotation and retrieval system which is able to perform automatic semantic annotation of news video archives and provide access to the archives via these annotations. The presented system relies on the video texts as the information source and exploits several information extraction techniques on these texts to arrive at representative semantic information regarding the underlying videos. These techniques include named entity recognition, person entity extraction, coreference resolution, and semantic event extraction. Apart from the information extraction components, the proposed system also encompasses modules for news story segmentation, text extraction, and video retrieval along with a news video database to make it a full-fledged system to be employed in practical settings. The proposed system is a generic one employing a wide range of techniques to automate the semantic video indexing process and to bridge the semantic gap between what can be automatically extracted from videos and what people perceive as the video semantics. Based on the proposed system, a novel automatic semantic annotation and retrieval system is built for Turkish and evaluated on a broadcast news video collection, providing evidence for its feasibility and convenience for news videos with a satisfactory overall performance.

Introduction to Linguistic Annotation and Text Analytics

Linguistic annotation and text analytics are active areas of research and development, with academic conferences and industry events such as the Linguistic Annotation Workshops and the annual Text Analytics Summits. This book provides a basic introduction to both fields, and aims to show that good linguistic annotations are the essential foundation for good text analytics. After briefly reviewing the basics of XML, with practical exercises illustrating in-line and stand-off annotations, a chapter is devoted to explaining the different levels of linguistic annotations. The reader is encouraged to create example annotations using the WordFreak linguistic annotation tool. The next chapter shows how annotations can be created automatically using statistical NLP tools, and compares two sets of tools, the OpenNLP and Stanford NLP tools. The second half of the book describes different annotation formats and gives practical examples of how to interchange annotations between different formats using XSLT transformations. The two main text analytics architectures, GATE and UIMA, are then described and compared, with practical exercises showing how to configure and customize them. The final chapter is an introduction to text analytics, describing the main applications and functions including named entity recognition, coreference resolution and information extraction, with practical examples using both open source and commercial tools. Copies of the example files, scripts, and stylesheets used in the book are available from the companion website, located at http://sites.morganclaypool.com/wilcock. Table of Contents: Working with XML / Linguistic Annotation / Using Statistical NLP Tools / Annotation Interchange / Annotation Architectures / Text Analytics

Application of association rules mining to Named Entity Recognition and co-reference resolution for the Indonesian language

In this paper, we propose a new method, association rules mining for Named Entity Recognition (NER) and co-reference resolution. The method uses several morphological and lexical features such as Pronoun Class (PC) and Name Class (NC), String Similarity (SP) and Position (P) in the text, into a vector of attributes. Applied to a corpus of newspaper in the Indonesian language, the method outperforms state-of-the-art maximum entropy method in name entity recognition and is comparable with state-of-the-art machine learning methods, decision tree, for co-reference resolution.