Automatic Relation Extraction Research Articles

Extracting Drug-Protein Relation from Literature Using Ensembles of Biomedical Transformers.

Automatic extraction of relations between drugs/chemicals and proteins from ever-growing biomedical literature is required to build up-to-date knowledge bases in biomedicine. To promote the development of automated methods, BioCreative-VII organized a shared task - the DrugProt track, to recognize drug-protein entity relations from PubMed abstracts. We participated in the shared task and leveraged deep learning-based transformer models pre-trained on biomedical data to build ensemble approaches to automatically extract drug-protein relation from biomedical literature. On the main corpora of 10,750 abstracts, our best system obtained an F1-score of 77.60% (ranked 4th among 30 participating teams), and on the large-scale corpus of 2.4M documents, our system achieved micro-averaged F1-score of 77.32% (ranked 2nd among 9 system submissions). This demonstrates the effectiveness of domain-specific transformer models and ensemble approaches for automatic relation extraction from biomedical literature.

Комплексный подход к анализу аргументативных отношений в текстах научной коммуникации

The problem of automatic analysis of argumentation in scientific communication texts is considered. Argumentation is understood as an ordered set of arguments used to support a certain thesis. An argument includes at least one premise and one conclusion, connected by an argumentative relation. The purpose of the work is an experimental study of neu-ral network approaches to solving the problem of searching and extracting argumentative relations between statements located closely in the text. The study was conducted on a corpus of texts with argumentative markup created using the previously developed web platform. The corpus included texts of scientific news, analytical articles from the Habr web-site, scientific articles and reviews. Datasets for machine learning were built based on these texts. To improve the quali-ty of neural network models training, these sets were supplemented with new data by using automatic paraphrasing and double translation methods. Two approaches to training models were considered: the first one with labeling of indi-cators in texts and the second one with preliminary training of a language model on the task of predicting indicators. To evaluate the models performance, an approach was proposed based on estimates of agreement between experts, usu-ally used to compare markups of manually created texts. A comparison of agreement coefficients between experts and trained models showed that the quality threshold for extracting argumentative relations was almost reached on the model with labeled indicators. A manual analysis of model errors was carried out by visualizing the obtained results. Thus, the novelty of the work lies in the application of an integrated approach to creating data sets, training models and evaluating the results obtained from the automatic extraction of argumentative relations.

A Novel Automated Approach to Mutation-Cancer Relation Extraction by Incorporating Heterogeneous Knowledge.

Automatic extraction of relations between gene mutations and cancer entities occurring in the cancer literature using text mining can rapidly provide vital information to support precision cancer medicine. However, mutation-cancer relation extraction is more challenging than general relation extraction from free text, since it is often not possible without cancer-specific background knowledge and thus the model replies on a deeper understanding of complex surrounding tokens. We propose a deep learning model that jointly extracts mutations and their associated cancers. Background knowledge comes from two different knowledge bases which store different types of information about mutations. Given the different ways in which knowledge is stored in these two resources, we propose two separate methods for embedding knowledge, namely sentence-based knowledge integration and attribute-aware knowledge integration. The evaluation demonstrated that our model outperforms a number of baseline models and gains 96.00%, 92.57% and 94.57% F1 scores on three public datasets, EMU BCa, EMU PCa, and BRONCO, thus illustrating the effectiveness of our knowledge integration approach. The auxiliary experiments show that our models can utilize more informative text from the KBs and link the mutations to their corresponding cancer disease although the input text provides insufficient context.

SCREENER: Streamlined CollaboRative Learning of NER and RE for Automatic Extraction of Gene-Disease relations

SCREENER: Streamlined CollaboRative Learning of NER and RE for Automatic Extraction of Gene-Disease relations

SCREENER: Streamlined CollaboRative Learning of NER and RE for Automatic Extraction of Gene-Disease relations

SCREENER: Streamlined CollaboRative Learning of NER and RE for Automatic Extraction of Gene-Disease relations

AUTOMATIC RELATION EXTRACTION BETWEEN ENTITIES FOR AMHARIC TEXT

This research work primarily focused on the automatic relation extraction between entities for Amharic text using supervised machine learning approach.The Walta Information Centre online archive resources were used to create the studys own corpus, which consisted of 2000 sentences and a reasonable quantity of 30,466 words or tokens. The proposed solution has four processes namely preprocessing, Text labeling, feature extraction and feature selection and Recognition. The tokenization and POS are used as preprocessing. After the text is tokenized and giving POS for each of tokens the next step is text labeling system. For text labeling mechanism BIO scheme is used. The tag features are selected for building the model. The Tag feature consists of name entity type and relation type. The name entity type features are represented by Location (LOC), Organization (ORG) and Person (PER) and the relation type features are identified every word which existed between two entities for instance between location-location relation type or location-organization relation type and all the corresponding entities that are appeared it. Vectorizations are done using DictVectorizer and word2features. Support vector machines and conditional random field machine learning are used to for recognizing the entity relation between Amharic texts. SVM with SGD achieved the weighted precision of 49%, recall 10% and f1-score 13% are scored. SVM with Multinomial Naive Bayes Classifier Algorithm achieve precision of 61%, recall 41% and f1-score 48%.SVM with Passive Aggressive classifier achieved weighted average precision of 55%, recall 19% and f1-score 27%. CRF algorithm achieved precision of 87%, recall 87% and f1-score 86%. The CRF model outperform compared with other SVM algorithms.

Relation extraction for manufacturing knowledge graphs based on feature fusion of attention mechanism and graph convolution network

Relation extraction is a crucial step in the constructions of knowledge graphs (KGs). However, relation extraction is performed manually in the manufacturing field due to the sentence characteristics, which include weak correlation and high entity density. This approach has the disadvantages of low efficiency and high dependence on experts. At present, very few studies have been performed on relation extraction in the manufacturing field, so establishing a relation extraction model with high efficiency is an urgent need. Therefore, in this paper, a relation extraction model is proposed for manufacturing knowledge (MKREM), in which word embedding is obtained by the Bi-LSTM layer to improve robustness, and a Simplified Graph Convolution Network (SGC) layer is applied to quickly mine the entity information. Then, dependency and semantic features are extracted by the multi-head stacked GCN and relation attention mechanism, respectively. Finally, the dependency and semantic features are fused to generate the comprehensive features for relation extraction so that better performance on texts with weak correlation and high entity density can be obtained. The performance of MKREM is tested by experiments on the equipment maintenance dataset and the quality dataset from an automobile enterprise, and its effectiveness is verified in the automobile manufacture filed. The results show that the F1 scores obtained using MKREM are 2% higher than those of the commonly used models on both datasets, and the F1 scores when using Contextualized-MKREM are improved by 3%, so MKREM is very suitable for the automatic relation extraction during establishing manufacturing KGs.

Literature Mining of Disease Associated Noncoding RNA in the Omics Era.

Noncoding RNAs (ncRNA) are transcripts without protein-coding potential that play fundamental regulatory roles in diverse cellular processes and diseases. The application of deep sequencing experiments in ncRNA research have generated massive omics datasets, which require rapid examination, interpretation and validation based on exiting knowledge resources. Thus, text-mining methods have been increasingly adapted for automatic extraction of relations between an ncRNA and its target or a disease condition from biomedical literature. These bioinformatics tools can also assist in more complex research, such as database curation of candidate ncRNAs and hypothesis generation with respect to pathophysiological mechanisms. In this concise review, we first introduced basic concepts and workflow of literature mining systems. Then, we compared available bioinformatics tools tailored for ncRNA studies, including the tasks, applicability, and limitations. Their powerful utilities and flexibility are demonstrated by examples in a variety of diseases, such as Alzheimer’s disease, atherosclerosis and cancers. Finally, we outlined several challenges from the viewpoints of both system developers and end users. We concluded that the application of text-mining techniques will booster disease-associated ncRNA discoveries in the biomedical literature and enable integrative biology in the current omics era.

Causality in requirements artifacts: prevalence, detection, and impact

Causal relations in natural language (NL) requirements convey strong, semantic information. Automatically extracting such causal information enables multiple use cases, such as test case generation, but it also requires to reliably detect causal relations in the first place. Currently, this is still a cumbersome task as causality in NL requirements is still barely understood and, thus, barely detectable. In our empirically informed research, we aim at better understanding the notion of causality and supporting the automatic extraction of causal relations in NL requirements. In a first case study, we investigate 14.983 sentences from 53 requirements documents to understand the extent and form in which causality occurs. Second, we present and evaluate a tool-supported approach, called CiRA, for causality detection. We conclude with a second case study where we demonstrate the applicability of our tool and investigate the impact of causality on NL requirements. The first case study shows that causality constitutes around 28 % of all NL requirements sentences. We then demonstrate that our detection tool achieves a macro-hbox {F}_{1} score of 82 % on real-world data and that it outperforms related approaches with an average gain of 11.06 % in macro-Recall and 11.43 % in macro-Precision. Finally, our second case study corroborates the positive correlations of causality with features of NL requirements. The results strengthen our confidence in the eligibility of causal relations for downstream reuse, while our tool and publicly available data constitute a first step in the ongoing endeavors of utilizing causality in RE and beyond.

APPLYING CLUSTER ANALYSIS OF EDUCATIONAL CONTENT FOR IDENTIFYING SIMILAR DOCUMENTS

Research was conducted on the intellectual support of the process for improving the quality of the educational content forming professional competencies of higher education programs. At the same time, the identification of relevant working curricula of disciplines, taking into account the context of entities in documents with automatic extraction of entities and relations between them, has allowed to implement it without laborious processing and adaptation of knowledge bases. The subject of the study is the content of working curricula (syllabus), which are defined as a set of data that characterize the results of training and the content of the discipline. As a result of the work [1], the author created a corpus of texts from the documents of working curricula in the disciplines of the specialty "Information Systems". Existing approaches to the intellectual support of the formation of educational programs based on ontological models of systems [2-4] based on knowledge and rules, heuristic algorithms for the automated compilation of curricula, methods of expert assessments and cognitive maps do not allow to effectively take into account and quickly track changes in the market labor, and also in the space of educational content. In turn, the formation and actualization of ontological models, systems of rules and precedents by experts for all existing subject areas the preparation of educational programs is an extremely labor-intensive process that requires the involvement of a representative composition of experts in each of the subject areas to ensure the required accuracy.

The contributions of the ventral and the dorsal visual streams to the automatic processing of action relations of familiar and unfamiliar object pairs

Recent studies suggest that action relations between objects affect behavioral and neural responses to action-related object pairs. Existing evidence indicates the involvement of both visual streams in this process. However, uncertainty remains regarding the functional roles of the ventral and the dorsal visual streams, and their interaction in the perception of the action relations between objects. In particular, it is not clear whether the involvement of either stream is dependent on object recognition. The present study aims to dissociate the effect of object familiarity and automatic extraction of action relations by presenting familiar and novel object pairs, which either indicate action relations or not, in a context where the objects and their identification were task-irrelevant. The present study examines the possibility that the activation of the ventral visual stream is dependent on facilitated object recognition exclusively associated with familiar action relations, and tests whether the dorsal visual stream is recruited in the automatic processing of the action relations in paired-object scenarios. With a set of registered analyses, we revealed that both the dorsal and the ventral streams respond to action relations in paired-object scenarios, and the responses were not exclusive to familiar action relations. Registered dynamic causal modeling analysis revealed that the inherent inter-stream connectivity was inhibited by action relations, and further unregistered analysis revealed that there lacks significant inherent effective connectivity between the two streams. These results suggest that both visual streams respond to the experimental manipulation of action relations in paired-object scenarios, but contribute corresponding information to different computations, leading to dissociations between the neural activities of the two streams. These results for the first time suggested a division of labor between the two visual streams in the automatic extraction of action relations in paired-object scenarios. Future study is needed to further explore the context-dependency of the collaboration of the two steams in processing action-related features in multiple-object scenarios.

Exploiting Latent Semantic Subspaces to Derive Associations for Specific Pharmaceutical Semantics

State-of-the-art approaches in the field of neural embedding models (NEMs) enable progress in the automatic extraction and prediction of semantic relations between important entities like active substances, diseases, and genes. In particular, the prediction property is making them valuable for important research-related tasks such as hypothesis generation and drug repositioning. A core challenge in the biomedical domain is to have interpretable semantics from NEMs that can distinguish, for instance, between the following two situations: (a) drug x induces disease y and (b) drug x treats disease y. However, NEMs alone cannot distinguish between associations such as treats or induces. Is it possible to develop a model to learn a latent representation from the NEMs capable of such disambiguation? To what extent do we need domain knowledge to succeed in the task? In this paper, we answer both questions and show that our proposed approach not only succeeds in the disambiguation task but also advances current growing research efforts to find real predictions using a sophisticated retrospective analysis.Furthermore, we investigate which type of associations is generally better contextualized and therefore probably has a stronger influence in our disambiguation task. In this context, we present an approach to extract an interpretable latent semantic subspace from the original embedding space in which therapeutic drug–disease associations are more likely.

Исследование ассоциативных связей слов в корпусе социальных сетей с помощью дистрибутивно-семантических моделей

Данная статья отражает результаты эксперимента по автоматическому извлечению ассоциативных связей из корпуса русскоязычных текстов социальной сети Facebook с помощью алгоритмов и инструментов дистрибутивной семантики. Для лексем, выражающих ключевые понятия русскоязычной картины мира, автоматически извлечены ассоциаты из корпуса Facebook с использованием нейросетевых архитектур Word 2 Vec ( CBOW и Skip - gram ). Был проведен сопоставительный анализ полученных данных и данных Русского ассоциативного словаря и Русского дистрибутивного тезауруса. Полученные результаты позволяют провести лингвистический анализ языкового сознания современных пользователей социальных сетей.

Grammar checking and relation extraction in text: approaches, techniques and open challenges

Purpose The purpose of this paper is to provide an overall review of grammar checking and relation extraction (RE) literature, their techniques and the open challenges associated with them; and, finally, suggest future directions. Design/methodology/approach The review on grammar checking and RE was carried out using the following protocol: we prepared research questions, planed for searching strategy, addressed paper selection criteria to distinguish relevant works, extracted data from these works, and finally, analyzed and synthesized the data. Findings The output of error detection models could be used for creating a profile of a certain writer. Such profiles can be used for author identification, native language identification or even the level of education, to name a few. The automatic extraction of relations could be used to build or complete electronic lexical thesauri and knowledge bases. Originality/value Grammar checking is the process of detecting and sometimes correcting erroneous words in the text, while RE is the process of detecting and categorizing predefined relationships between entities or words that were identified in the text. The authors found that the most obvious challenge is the lack of data sets, especially for low-resource languages. Also, the lack of unified evaluation methods hinders the ability to compare results.

Research on Construction Method of Chinese Domain Ontology Based on Relation Extraction

Domain ontology plays an increasingly important role in the information extraction process and has become a useful tool for knowledge acquisition, knowledge sharing, and modeling. However, the existing domain ontology construction methods are mainly based on manual mode, lacking systematic and engineering methods. This paper proposes a construction method of Chinese domain ontology based on relation extraction, which fully considers the automatic extraction of semantic hierarchical relations, and guarantees the semantic richness and accuracy of relational triples. The construction method of Chinese domain ontology based on relation extraction is applied to the field of food safety events. The constructed ontology has the advantages of reusability and scalability, which proves that the method is logical and operability.

Extraction of chemical-protein interactions from the literature using neural networks and narrow instance representation.

The scientific literature contains large amounts of information on genes, proteins, chemicals and their interactions. Extraction and integration of this information in curated knowledge bases help researchers support their experimental results, leading to new hypotheses and discoveries. This is especially relevant for precision medicine, which aims to understand the individual variability across patient groups in order to select the most appropriate treatments. Methods for improved retrieval and automatic relation extraction from biomedical literature are therefore required for collecting structured information from the growing number of published works. In this paper, we follow a deep learning approach for extracting mentions of chemical–protein interactions from biomedical articles, based on various enhancements over our participation in the BioCreative VI CHEMPROT task. A significant aspect of our best method is the use of a simple deep learning model together with a very narrow representation of the relation instances, using only up to 10 words from the shortest dependency path and the respective dependency edges. Bidirectional long short-term memory recurrent networks or convolutional neural networks are used to build the deep learning models. We report the results of several experiments and show that our best model is competitive with more complex sentence representations or network structures, achieving an F1-score of 0.6306 on the test set. The source code of our work, along with detailed statistics, is publicly available.

A general approach for improving deep learning-based medical relation extraction using a pre-trained model and fine-tuning.

The automatic extraction of meaningful relations from biomedical literature or clinical records is crucial in various biomedical applications. Most of the current deep learning approaches for medical relation extraction require large-scale training data to prevent overfitting of the training model. We propose using a pre-trained model and a fine-tuning technique to improve these approaches without additional time-consuming human labeling. Firstly, we show the architecture of Bidirectional Encoder Representations from Transformers (BERT), an approach for pre-training a model on large-scale unstructured text. We then combine BERT with a one-dimensional convolutional neural network (1d-CNN) to fine-tune the pre-trained model for relation extraction. Extensive experiments on three datasets, namely the BioCreative V chemical disease relation corpus, traditional Chinese medicine literature corpus and i2b2 2012 temporal relation challenge corpus, show that the proposed approach achieves state-of-the-art results (giving a relative improvement of 22.2, 7.77, and 38.5% in F1 score, respectively, compared with a traditional 1d-CNN classifier). The source code is available at https://github.com/chentao1999/MedicalRelationExtraction.

Extracting chemical-protein relations with ensembles of SVM and deep learning models.

Mining relations between chemicals and proteins from the biomedical literature is an increasingly important task. The CHEMPROT track at BioCreative VI aims to promote the development and evaluation of systems that can automatically detect the chemical–protein relations in running text (PubMed abstracts). This work describes our CHEMPROT track entry, which is an ensemble of three systems, including a support vector machine, a convolutional neural network, and a recurrent neural network. Their output is combined using majority voting or stacking for final predictions. Our CHEMPROT system obtained 0.7266 in precision and 0.5735 in recall for an F-score of 0.6410 during the challenge, demonstrating the effectiveness of machine learning-based approaches for automatic relation extraction from biomedical literature and achieving the highest performance in the task during the 2017 challenge.Database URL: http://www.biocreative.org/tasks/biocreative-vi/track-5/

Proposal for Automatic Extraction of Taxonomic Relations in Domain Corpus

Proposal for Automatic Extraction of Taxonomic Relations in Domain Corpus

Improving chemical disease relation extraction with rich features and weakly labeled data

BackgroundDue to the importance of identifying relations between chemicals and diseases for new drug discovery and improving chemical safety, there has been a growing interest in developing automatic relation extraction systems for capturing these relations from the rich and rapid-growing biomedical literature. In this work we aim to build on current advances in named entity recognition and a recent BioCreative effort to further improve the state of the art in biomedical relation extraction, in particular for the chemical-induced disease (CID) relations.ResultsWe propose a rich-feature approach with Support Vector Machine to aid in the extraction of CIDs from PubMed articles. Our feature vector includes novel statistical features, linguistic knowledge, and domain resources. We also incorporate the output of a rule-based system as features, thus combining the advantages of rule- and machine learning-based systems. Furthermore, we augment our approach with automatically generated labeled text from an existing knowledge base to improve performance without additional cost for corpus construction. To evaluate our system, we perform experiments on the human-annotated BioCreative V benchmarking dataset and compare with previous results. When trained using only BioCreative V training and development sets, our system achieves an F-score of 57.51 %, which already compares favorably to previous methods. Our system performance was further improved to 61.01 % in F-score when augmented with additional automatically generated weakly labeled data.ConclusionsOur text-mining approach demonstrates state-of-the-art performance in disease-chemical relation extraction. More importantly, this work exemplifies the use of (freely available) curated document-level annotations in existing biomedical databases, which are largely overlooked in text-mining system development.