Information Extraction Tasks Research Articles

Contrastive Information Extraction With Generative Transformer

Information extraction tasks such as entity relation extraction and event extraction are of great importance for natural language processing and knowledge graph construction. In this paper, we revisit the end-to-end information extraction task for sequence generation. Since generative information extraction may struggle to capture long-term dependencies and generate unfaithful triples, we introduce a novel model, contrastive information extraction with a generative transformer. Specifically, we introduce a single shared transformer module for an encoder-decoder-based generation. To generate faithful results, we propose a novel triplet contrastive training object. Moreover, we introduce two mechanisms to further improve model performance (i.e., batch-wise dynamic attention-masking and triple-wise calibration). Experimental results on five datasets (i.e., NYT, WebNLG, MIE, ACE-2005, and MUC-4) show that our approach achieves better performance than baselines.1

Extending semantic context analysis using machine learning services to process unstructured data

The primary focus of technical communication (TC) in the past decade has been the system-assisted generation and utilization of standardized, structured, and classified content for dynamic output solutions. Nowadays, machine learning (ML) approaches offer a new opportunity to integrate unstructured data into existing knowledge bases without the need to manually organize information into topic-based content enriched with semantic metadata. To make the field of artificial intelligence (AI) more accessible for technical writers and content managers, cloud-based machine learning as a service (MLaaS) solutions provide a starting point for domain-specific ML modelling while unloading the modelling process from extensive coding, data processing and storage demands. Therefore, information architects can focus on information extraction tasks and on prospects to include pre-existing knowledge from other systems into the ML modelling process. In this paper, the capability and performance of a cloud-based ML service, IBM Watson, are analysed to assess their value for semantic context analysis. The ML model is based on a supervised learning method and features deep learning (DL) and natural language processing (NLP) techniques. The subject of the analysis is a corpus of scientific publications on the 2019 Coronavirus disease. The analysis focuses on information extractions regarding preventive measures and effects of the pandemic on healthcare workers.

Metric-Type Identification for Multilevel Header Numerical Tables in Scientific Papers

Numerical tables are widely used to present experimental results in scientific papers. For table understanding, a metric-type is essential to discriminate numbers in the tables. We introduce a new information extraction task, metric-type identification from multi-level header numerical tables, and provide a dataset extracted from scientific papers consisting of header tables, captions, and metric-types. We then propose two joint-learning neural classification and generation schemes featuring pointer-generator-based and BERT-based models. Our results show that the joint models can handle both in-header and out-of-header metric-type identification problems.

Exploiting Rich Event Representation to Improve Event Causality Recognition

Event causality identification is an essential task for information extraction that has attracted growing attention. Early researchers were accustomed to combining the convolutional neural network or recurrent neural network models with external causal knowledge, but these methods ignore the importance of rich semantic representation of the event. The event is more structured, so it has more abundant semantic representation. We argue that the elements of the event, the interaction of the two events, and the context between the two events can enrich the event’s semantic representation and help identify event causality. Therefore, the effective semantic representation of events in event causality recognition deserves further study. To verify the effectiveness of rich event semantic representation for event causality identification, we proposed a model exploiting rich event representation to improve event causality recognition. Our model is based on multi-column convolutional neural networks, which integrate rich event representation, including event tensor representation, event interaction representation, and context-aware event representation. We designed various experimental models and conducted experiments on the Chinese emergency corpus, the most comprehensive annotation of events and event elements, enabling us to study the semantic representation of events from all aspects. The extensive experiments showed that the rich semantic representation of events achieved significant performance improvement over the baseline model on event causality recognition, indicating that the semantic representation of events plays an important role in event causality recognition.

Geoparcel-Based Spatial Prediction Method for Grassland Fractional Vegetation Cover Mapping

Grassland resources guarantee the balance of ecosystems and the sustainable development of animal husbandry. Spatial information is essential for grass resource management in pastoral areas, which can be extracted quickly on a large-scale by using remote sensing data. However, most conventional methods are based on the grid pixels of remote sensing images. The spatial information based on these regular units inevitably has the phenomenon of mixed pixels, which leads to the unreliability of grassland resource information with irregular spatial heterogeneity. To resolve this problem, this article takes the spatial mapping of fractional vegetation cover (FVC) as a typical target task of information extraction of grassland resources and proposes a geoparcel-based spatial prediction method in which irregular geographic objects from high spatial resolution remote sensing images, i.e., grassland geoparcels, are used as basic mapping units instead of traditional regular units. This change can make the spatial expression of mapping closer to the reality of grassland due to the fine spatial structure. Moreover, multisource spatial data can be integrated together in the regression-based procedure of prediction through the geoparcel as a unified spatial benchmark. A case experiment of Abag Banner, Inner Mongolia, China, shows that the proposed method can achieve good FVC mapping results. The spatial prediction of FVC based on grassland geoparcels is verified to be effective when a random forest regression is used in the modeling. In comparison with traditional regular grid-based methods, the proposed method achieves higher accuracy with 11.89% in relative root mean squared error (%RMSE), and 0.86 in determination coefficient of regression ( R 2). It has advantages in sensing the tiny spatial heterogeneity of FVC at the boundary of grassland change. The formalized procedure potentially promotes the development of spatial mapping technology for grassland resources.

TwiCS: Lightweight Entity Mention Detection in Targeted Twitter Streams

Microblogging sites, like Twitter, continuously generate a large volume of streaming data. This streaming environment creates new challenges for two concomitant Information Extraction tasks: Entity Mention Detection (EMD) and Entity Detection (ED). The new challenges include (1) continuously evolving topics, which may deprecate model-based approaches quickly; (2) non-literary nature of posts, which makes traditional NLP techniques less effective; and (3) huge volume of streaming data, which makes computationally expensive approaches less suitable. In this paper, we propose an approach for EMD/ED whose creation is guided by the constraints specific to streaming environments from the ground up. Our system TwiCS implements this approach. TwiCS employs a computationally light two-phase process. In the first phase, it exploits simple (low computation) syntactic cues to suggest Entity Mention (EM) candidates. In the second phase, it uses occurrence mining to classify candidates according to their likelihood of being true EMs. Our experiments show that TwiCS achieves an average effectiveness improvement of 14.6%, while maintaining at least 2.64 times higher throughput, when compared to several state-of-the-art systems.

C-Norm: a neural approach to few-shot entity normalization

BackgroundEntity normalization is an important information extraction task which has gained renewed attention in the last decade, particularly in the biomedical and life science domains. In these domains, and more generally in all specialized domains, this task is still challenging for the latest machine learning-based approaches, which have difficulty handling highly multi-class and few-shot learning problems. To address this issue, we propose C-Norm, a new neural approach which synergistically combines standard and weak supervision, ontological knowledge integration and distributional semantics.ResultsOur approach greatly outperforms all methods evaluated on the Bacteria Biotope datasets of BioNLP Open Shared Tasks 2019, without integrating any manually-designed domain-specific rules.ConclusionsOur results show that relatively shallow neural network methods can perform well in domains that present highly multi-class and few-shot learning problems.

Trainable Framework for Information Extraction, Structuring and Summarization of Unstructured Data, Using Modified NER

The World Wide Web is an ever expanding source of data in today’s world. Millions of tera-bytes of data and information is getting added every second. In this information age as the data is getting generated at an exponential rate, the fact to be noted is that most of the information is already available is in the form of natural language text. The task of information extraction from mammoth data leads us to think on the quality and the form of available data. Secondly, the ever increasing data poses a challenging task of extracting useful information from the available data. The third task is to extract information as efficiently as possible. For retrieving the information there is a need to develop ingenious way to answer any kind of query put up by a user from the available unstructured data. This paper proposes a novel trainable and integrated Natural Language Information Interpretation and Representation System (NLIIRS) that accepts any available un-annotated corpus of data in the form of natural language, and performs the following tasks: finds out the useful data, extracts relevant information in usable form (structured form/tables), summarizes the data and structures the data in relational form. At the end the Question and Answering (Q&A) module shows the cognitive abilities of NLIIR system by answering the questions in natural language relevant to the text. This multispecialty system beyond just Q&A. This is a trainable system capable of handling any unstructured data to be transformed into structured and well organized information. It allows the user to ask questions in natural language. It adopts the advantages of a modified named entity recognition so as to bypass the time consuming process of parts of speech tagging while pre-processing the available corpus (data) for information extraction.

A novel joint biomedical event extraction framework via two-level modeling of documents

With the rapid development of information technology, the amount of textual data generated in biomedical field becomes larger and larger. Biomedical event extraction, which is a fundamental information extraction task, has gained a growing interest in biomedical community. Although researchers have proposed various approaches to this task, the performance is still undesirable since previous approaches fail to model biomedical documents effectively. In this paper, we propose an end-to-end framework for document-level joint biomedical event extraction. To better capture the complex relationships among contexts in biomedical documents, a two-level modeling approach is introduced for biomedical documents. More specifically, the dependency-based GCN and hypergraph are used to model local context and global context in each biomedical document, respectively. In addition, a fine-grained interaction mechanism is proposed to model effectively the interaction between local and global contexts to learn better contextualized representations for biomedical event extraction. Comprehensive experiments on two widely used datasets are conducted and the results demonstrate the effectiveness of the proposed framework over state-of-the-art methods.

Empower Chinese event detection with improved atrous convolution neural networks

Event Detection (ED) is an essential and challenging task in Information Extraction (IE). Recent advances in neural networks make it possible to build reliable models without complicated feature engineering. Neural network-based models commonly regard event detection as a char-wise or word-wise labeling task, which suffers from the problems of long-distance information miss-capturing, discontinuous labeling errors, etc., between characters/words and event triggers, especially in languages without natural word delimiters such as Chinese. In this paper, we propose a novel multi-layer Residual and Gated-Based Atrous Convolution Neural Network (RG-ACNN) framework, which attempts to alleviate above-mentioned problems. Specifically, the ACNN is introduced in our model to expand the receptive field to obtain multi-scale context information to capture dependencies between long-distance information. While gated and residual mechanisms are both imported to ACNN to improve our networks’ capability of the information filtering and aggregation. Besides, RG-ACNN performs event detection in a char-wise paradigm, where a novel “head-tail dual-pointer” labeled strategy is used to overcome the incomplete continuous labeling problem. Experiments on the ACE2005-CN and several standard benchmark datasets show that RG-ACNN significantly outperforms state-of-the-art (SOTA) methods.

Mining information from sentences through Semantic Web data and Information Extraction tasks

The Semantic Web provides guidelines for the representation of information about real-world objects (entities) and their relations (properties). This is helpful for the dissemination and consumption of information by people and applications. However, the information is mainly contained within natural language sentences, which do not have a structure or linguistic descriptions ready to be directly processed by computers. Thus, the challenge is to identify and extract the elements of information that can be represented. Hence, this article presents a strategy to extract information from sentences and its representation with Semantic Web standards. Our strategy involves Information Extraction tasks and a hybrid semantic similarity measure to get entities and relations that are later associated with individuals and properties from a Knowledge Base to create RDF triples (Subject–Predicate–Object structures). The experiments demonstrate the feasibility of our method and that it outperforms the accuracy provided by a pattern-based method from the literature.

Mining product innovation ideas from online reviews

The importance of online customer reviews to product innovation has been well-recognized in prior literature. Mining online reviews has received extensive attention and efforts. Most existing research on mining online reviews focus on issues such as the impact of reviews on sales, helpfulness of reviews, and customers’ participation in reviews. Few research studies, however, seek to identify and extract innovation ideas for products from online reviews. This type of information is particularly important for product functionality improvement and new feature development from a manufacturer's perspective. Mining product innovation ideas allows a manufacturer to proactively review customer opinion and unlock insights about new functionality and features that the market expects, in order to gain a competitive advantage. In this paper, we propose a deep learning-based approach to identify sentences that contain innovation ideas from online reviews. Specifically, we develop a novel ensemble embedding method to generate semantic and contextual representations of the words in review sentences. The resultant representations in each sentence are then used in a long short-term memory (LSTM) model for innovation-sentence identification. Moreover, we adopt a focal loss function in our model to address the class imbalance problem. We validate our approach with a dataset of 10,000 customer reviews from Amazon. Our model achieves an AUC score of 0.91 and an F1 score of 0.89, outperforming a set of state-of-the-art baseline models in the comparison. Our approach can be extended and applied to many other information extraction tasks.

Natural language processing systems for pathology parsing in limited data environments with uncertainty estimation.

ObjectiveCancer is a leading cause of death, but much of the diagnostic information is stored as unstructured data in pathology reports. We aim to improve uncertainty estimates of machine learning-based pathology parsers and evaluate performance in low data settings.Materials and methodsOur data comes from the Urologic Outcomes Database at UCSF which includes 3232 annotated prostate cancer pathology reports from 2001 to 2018. We approach 17 separate information extraction tasks, involving a wide range of pathologic features. To handle the diverse range of fields, we required 2 statistical models, a document classification method for pathologic features with a small set of possible values and a token extraction method for pathologic features with a large set of values. For each model, we used isotonic calibration to improve the model’s estimates of its likelihood of being correct.ResultsOur best document classifier method, a convolutional neural network, achieves a weighted F1 score of 0.97 averaged over 12 fields and our best extraction method achieves an accuracy of 0.93 averaged over 5 fields. The performance saturates as a function of dataset size with as few as 128 data points. Furthermore, while our document classifier methods have reliable uncertainty estimates, our extraction-based methods do not, but after isotonic calibration, expected calibration error drops to below 0.03 for all extraction fields.ConclusionsWe find that when applying machine learning to pathology parsing, large datasets may not always be needed, and that calibration methods can improve the reliability of uncertainty estimates.

Joint event extraction along shortest dependency paths using graph convolutional networks

Event extraction (EE) is one of the core information extraction tasks, whose purpose is to automatically identify and extract information about incidents and their actors from texts. This may be beneficial to several domains such as knowledge base construction, question answering and summarization tasks, to name a few. The problem of extracting event information from texts is longstanding and usually relies on elaborately designed lexical and syntactic features, which, however, take a large amount of human effort and lack generalization. More recently, deep neural network approaches have been adopted as a means to learn underlying features automatically. However, existing networks do not make full use of syntactic features, which play a fundamental role in capturing very long-range dependencies. Also, most approaches extract each argument of an event separately without considering associations between arguments which ultimately leads to low efficiency, especially in sentences with multiple events. To address the above-referred problems, we propose a novel joint event extraction framework that aims to extract multiple event triggers and arguments simultaneously by introducing shortest dependency path in the dependency graph. We do this by eliminating irrelevant words in the sentence, thus capturing long-range dependencies. Also, an attention-based graph convolutional network is proposed, to carry syntactically related information along the shortest paths between argument candidates that captures and aggregates the latent associations between arguments; a problem that has been overlooked by most of the literature. Our results show a substantial improvement over state-of-the-art methods on two datasets, namely ACE 2005 and TAC KBP 2015.

Highly Language-Independent Word Lemmatization Using a Machine-Learning Classifier

Lemmatization is a process of finding the base morphological form (lemma) of a word. It is an important step in many natural language processing, information retrieval, and information extraction tasks, among others. We present an open-source language-independent lemmatizer based on the Random Forest classification model. This model is a supervised machine-learning algorithm with decision trees that are constructed corresponding to the grammatical features of the language. This lemmatizer does not require any manual work for hard-coding of the rules, and at the same time it is simple and interpretable. We compare the performance of our lemmatizer with that of the UDPipe lemmatizer on twenty-two out of twenty-five languages we work on for which UDPipe has models. Our lemmatization method shows good performance on different languages from various language groups, and it is easily extensible to other languages.

An Extensible Framework of Leveraging Syntactic Skeleton for Semantic Relation Classification

Relation classification is one of the most fundamental upstream tasks in natural language processing and information extraction. State-of-the-art approaches make use of various deep neural networks (DNNs) to extract higher-level features directly. They can easily access to accurate classification results by taking advantage of both local entity features and global sentential features. Recent works on relation classification devote efforts to modify these neural networks, but less attention has been paid to the feature design concerning syntax. However, from a linguistic perspective, syntactic features are essential for relation classification. In this article, we present a novel linguistically motivated approach that enhances relation classification by imposing additional syntactic constraints. We investigate to leverage syntactic skeletons along with the sentential contexts to identify hidden relation types. The syntactic skeletons are extracted under the guidance of prior syntax knowledge. During extraction, the input sentences are recursively decomposed into syntactically shorter and simpler chunks. Experimental results on the SemEval-2010 Task 8 benchmark show that incorporating syntactic skeletons into current DNN models enhances the task of relation classification. Our systems significantly surpass two strong baseline systems. One of the substantial advantages of our proposal is that this framework is extensible for most current DNN models.

User-centered design of a web-based crowdsourcing-integrated semantic text annotation tool for building a mental health knowledge base.

One in five U.S. adults lives with some kind of mental health condition and 4.6% of all U.S. adults have a serious mental illness. The Internet has become the first place for these people to seek online mental health information for help. However, online mental health information is not well-organized and often of low quality. There have been efforts in building evidence-based mental health knowledgebases curated with information manually extracted from the high-quality scientific literature. Manual extraction is inefficient. Crowdsourcing can potentially be a low-cost mechanism to collect labeled data from non-expert laypeople. However, there is not an existing annotation tool integrated with popular crowdsourcing platforms to perform the information extraction tasks. In our previous work, we prototyped a Semantic Text Annotation Tool (STAT) to address this gap. We aimed to refine the STAT prototype (1) to improve its usability and (2) to enhance the crowdsourcing workflow efficiency to facilitate the construction of evidence-based mental health knowledgebase, following a user-centered design (UCD) approach. Following UCD principles, we conducted four design iterations to improve the initial STAT prototype. In the first two iterations, usability testing focus groups were conducted internally with 8 participants recruited from a convenient sample, and the usability was evaluated with a modified System Usability Scale (SUS). In the following two iterations, usability testing was conducted externally using the Amazon Mechanical Turk (MTurk) platform. In each iteration, we summarized the usability testing results through thematic analysis, identified usability issues, and conducted a heuristic evaluation to map identified usability issues to Jakob Nielsen's usability heuristics. We collected suggested improvements in the usability testing sessions and enhanced STAT accordingly in the next UCD iteration. After four UCD iterations, we conducted a case study of the system on MTurk using mental health related scientific literature. We compared the performance of crowdsourcing workers with two expert annotators from two aspects: efficiency and quality. The SUS score increased from 70.3±12.5 to 81.1±9.8 after the two internal UCD iterations as we improved STAT's functionality based on the suggested improvements. We then evaluated STAT externally through MTurk in the following two iterations. The SUS score decreased to 55.7±20.1 in the third iteration, probably because of the complexity of the tasks. After further simplification of STAT and the annotation tasks with an improved annotation guideline, the SUS score increased to 73.8±13.8 in the fourth iteration of UCD. In the evaluation case study, on average, the workers spent 125.5±69.2s on the onboarding tutorial and the crowdsourcing workers spent significantly less time on the annotation tasks compared to the two experts. In terms of annotation quality, the workers' annotation results achieved average F1-scores ranged from 0.62 to 0.84 for the different sentences. We successfully developed a web-based semantic text annotation tool, STAT, to facilitate the curation of semantic web knowledgebases through four UCD iterations. The lessons learned from the UCD process could serve as a guide to further enhance STAT and the development and design of other crowdsourcing-based semantic text annotation tasks. Our study also showed that a well-organized, informative annotation guideline is as important as the annotation tool itself. Further, we learned that a crowdsourcing task should consist of multiple simple microtasks rather than a complicated task.

Constant-Delay Enumeration for Nondeterministic Document Spanners

One of the classical tasks in information extraction is to extract subparts of texts through regular expressions. In the database theory literature, this approach has been generalized and formalized as document spanners. In this model, extraction is performed by evaluating a particular kind of automata, called a sequential variable-set automaton (VA). The efficiency of this task is then measured in the context of enumeration algorithms: we first run a preprocessing phase computing a compact representation of the answers, and second we produce the results one after the other with a short time between consecutive answers, called the delay of the enumeration. Our goal is to have an algorithm that is tractable in combined complexity, i.e., in the sizes of the input document and the VA, while ensuring the best possible data complexity bounds in the input document size, i.e., a constant delay that does not depend on the document. We present such an algorithm for a variant of VAs called extended sequential VAs and give an experimental evaluation of this algorithm.

Can We Survive without Labelled Data in NLP? Transfer Learning for Open Information Extraction

Various tasks in natural language processing (NLP) suffer from lack of labelled training data, which deep neural networks are hungry for. In this paper, we relied upon features learned to generate relation triples from the open information extraction (OIE) task. First, we studied how transferable these features are from one OIE domain to another, such as from a news domain to a bio-medical domain. Second, we analyzed their transferability to a semantically related NLP task, namely, relation extraction (RE). We thereby contribute to answering the question: can OIE help us achieve adequate NLP performance without labelled data? Our results showed comparable performance when using inductive transfer learning in both experiments by relying on a very small amount of the target data, wherein promising results were achieved. When transferring to the OIE bio-medical domain, we achieved an F-measure of 78.0%, only 1% lower when compared to traditional learning. Additionally, transferring to RE using an inductive approach scored an F-measure of 67.2%, which was 3.8% lower than training and testing on the same task. Hereby, our analysis shows that OIE can act as a reliable source task.

Surface pattern-enhanced relation extraction with global constraints

Relation extraction is one of the most important tasks in information extraction. The traditional works either use sentences or surface patterns (i.e., the shortest dependency paths of sentences) to build extraction models. Intuitively, the integration of these two kinds of methods will further obtain more robust and effective extraction models, which is, however, ignored in most of the existing works. In this paper, we aim to learn the embeddings of surface patterns to further augment the sentence-based models. To achieve this purpose, we propose a novel pattern embedding learning framework with the weighted multi-dimensional attention mechanism. To suppress noise in the training dataset, we mine the global statistics between patterns and relations and introduce two kinds of prior knowledge to guide the pattern embedding learning. Based on the learned embeddings, we present two augmentation strategies to improve the existing relation extraction models. We conduct extensive experiments on two popular datasets (i.e., NYT and KnowledgeNet) and observe promising performance improvements.