Distant Supervision For Relation Extraction Research Articles

Sentence Bag Graph Formulation for Biomedical Distant Supervision Relation Extraction

Sentence Bag Graph Formulation for Biomedical Distant Supervision Relation Extraction

Distant supervision for relation extraction with hierarchical attention-based networks

Distant supervision for relation extraction with hierarchical attention-based networks

Reinforcement learning-based distant supervision relation extraction for fault diagnosis knowledge graph construction under industry 4.0

Fault diagnosis is the key concern in the operation and maintenance of industrial assets. A fault diagnosis knowledge graph (KG) can provide decision support to the engineers to efficiently conduct maintenance tasks. However, as a type of domain KG, it would be time-consuming to manually label the corpus collected from the multi-source including the maintenance log, handbook and article. Meanwhile, the existence of the noisy sentence in the multi-source corpus jeopardises the performance of relation extraction modelling. In order to address this issue, this paper proposes a distant supervision relation extraction (DSRE)-based approach to construct a fault diagnosis KG. In this approach, the ontology of the fault diagnosis KG is firstly designed. Subsequently, a DSRE algorithm named relation-aware-based sentence-level attention enhanced piecewise convolutional neural network with reinforcement learning strategy (PCNN-ATTRA-RL) is proposed. The algorithm can effectively lower the impact of noisy sentences and accurately label the relation of different entities when the labelled data is insufficient. In this algorithm, PCNN-ATTRA is designed as the DSRE classifier to effectively extract the relation between entity pairs. RL is conducted to remove the noisy sentence so as to further improve the performance. An experimental study based on the multi-source corpus collected from the real world reveals that the proposed approach shows merits in comparison with the state-of-the-art algorithms. Meanwhile, a fault diagnosis KG, which can greatly support the decision-making of the engineers in the fault diagnosis, is established via the proposed approach.

Improving distant supervision relation extraction with entity-guided enhancement feature

Improving distant supervision relation extraction with entity-guided enhancement feature

SwitchNet: A modular neural network for adaptive relation extraction

This paper presents a portable toolkit, SwitchNet, for extracting relations from textual input. We summarize four data protocols for relation extraction tasks, including relation classification, relation extraction, triple extraction, and distant supervision relation extraction. This neural architecture is modular, so it can take as input data at different stages of the information extraction process (simple text, text and entities or entity pairs as relation candidates) and compute the rest of the process (named entity recognition and relation classification). We systematically design four information flows to integrate the above protocols by sharing network building blocks and switching different information flows. This framework can extract multiple triples (subject, predicate, object) in one pass. This framework enhances the use of relation classification models in end-to-end triple extraction by inferring pairs of entities of interest and using the shared representation mechanism.

EANT: Distant Supervision for Relation Extraction with Entity Attributes via Negative Training

Distant supervision for relation extraction (DSRE) automatically acquires large-scale annotated data by aligning the corpus with the knowledge base, which dramatically reduces the cost of manual annotation. However, this technique is plagued by noisy data, which seriously affects the model’s performance. In this paper, we introduce negative training to filter them out. Specifically, we train the model with the complementary label based on the idea that “the sentence does not express the target relation”. The trained model can discriminate the noisy data from the training set. In addition, we believe that additional entity attributes (such as description, alias, and types) can provide more information for sentence representation. On this basis, we propose a DSRE model with entity attributes via negative training called EANT. While filtering noisy sentences, EANT also relabels some false negative sentences and converts them into useful training data. Our experimental results on the widely used New York Times dataset show that EANT can significantly improve the relation extraction performance over the state-of-the-art baselines.

Adversarial training for supervised relation extraction

Most supervised methods for relation extraction (RE) involve time-consuming human annotation. Distant supervision for RE is an efficient method to obtain large corpora that contains thousands of instances and various relations. However, the existing approaches rely heavily on knowledge bases (e.g., Freebase), thereby introducing data noise. Various relations and noisy labeling instances make the issue difficult to solve. In this study, we propose a model based on a piecewise convolution neural network with adversarial training. Inspired by generative adversarial networks, we adopt a heuristic algorithm to identify noisy datasets and apply adversarial training to RE. Experiments on the extended dataset of SemEval-2010 Task 8 show that our model can obtain more accurate training data for RE and significantly outperforms several competitive baseline models. Our model has an Fi score of 89.61%.

Distant supervision relation extraction based on mutual information and multi-level attention

Distant supervision for relation extraction, an effective method to reduce labor costs, has been widely used to search for novel relational facts from text. However, distant supervision always suffers from incorrect labelling problems. Meanwhile, existing methods for noise reduction oftentimes ignore the commonalities in the instances. To alleviate this issue, we propose a distant supervision relation extraction model based on mutual information and multi-level attention. In our proposed method, we calculate mutual information based on the attention mechanism. Mutual information are used to build attention at both word and sentence levels, which is expected to dynamically reduce the influence of noisy instances. Extensive experiments using a benchmark dataset have validated the effectiveness of our proposed method.

Hybrid attention-based transformer block model for distant supervision relation extraction

With an exponential explosive growth of various digital text information, it is challenging to efficiently obtain specific knowledge from massive unstructured text information. As one basic task for natural language processing (NLP), relation extraction (RE) aims to extract semantic relations between entity pairs based on the given text. To avoid manual labeling of datasets, distant supervision relation extraction (DSRE) has been widely used, aiming to utilize knowledge base to automatically annotate datasets. Unfortunately, this method heavily suffers from wrong labelling due to its underlying strong assumptions. To address this issue, we propose a new framework using hybrid attention-based Transformer block with multi-instance learning for DSRE. More specifically, the Transformer block is, for the first time, used as a sentence encoder, which mainly utilizes multi-head self-attention to capture syntactic information at the word level. Then, a novel sentence-level attention mechanism is proposed to calculate the bag representation, aiming to exploit all useful information in each sentence. Experimental results on the public dataset New York Times (NYT) demonstrate that the proposed approach can outperform the state-of-the-art algorithms on the adopted dataset, which verifies the effectiveness of our model on the DSRE task.

Piecewise convolutional neural networks with position attention and similar bag attention for distant supervision relation extraction

Piecewise convolutional neural networks with position attention and similar bag attention for distant supervision relation extraction

Improved distant supervision relation extraction based on edge-reasoning hybrid graph model

Distant supervision relation extraction (DSRE) trains a classifier by automatically labeling data through aligning triples in the knowledge base (KB) with large-scale corpora. Training data generated by distant supervision may contain many mislabeled instances, which is harmful to the training of the classifier. Some recent methods show that relevant background information in KBs, such as entity type (e.g., Organization and Book), can improve the performance of DSRE. However, there are three main problems with these methods. Firstly, these methods are tailored for a specific type of information. A specific type of information only has a positive effect on a part of instances and will not be beneficial to all cases. Secondly, different background information is embedded independently, and no reasonable interaction is achieved. Thirdly, previous methods do not consider the side effect of the introduced noise of background information. To address these issues, we leverage five types of background information instead of a specific type of information in previous works and propose a novel edge-reasoning hybrid graph (ER-HG) model to realize reasonable interaction between different kinds of information. In addition, we further employ an attention mechanism for the ER-HG model to alleviate the side effect of noise. The ER-HG model integrates all types of information efficiently and is very robust to the noise of information. We conduct experiments on two widely used datasets. The experimental results demonstrate that our model outperforms the state-of-the-art methods significantly in held-out metric and robustness tests.

Self-selective attention using correlation between instances for distant supervision relation extraction

Distant supervision relation extraction methods are widely used to extract relational facts in text. The traditional selective attention model regards instances in the bag as independent of each other, which makes insufficient use of correlation information between instances and supervision information of all correctly labeled instances, affecting the performance of relation extractor. Aiming at this problem, a distant supervision relation extraction method with self-selective attention is proposed. The method uses a layer of convolution and self-attention mechanism to encode instances to learn the better semantic vector representation of instances. The correlation between instances in the bag is used to assign a higher weight to all correctly labeled instances, and the weighted summation of instances in the bag is used to obtain a bag vector representation. Experiments on the NYT dataset show that the method can make full use of the information of all correctly labeled instances in the bag. The method can achieve better results as compared with baselines.

Distant Supervision for Relation Extraction with Sentence Selection and Interaction Representation

Distant supervision (DS) has been widely used for relation extraction (RE), which automatically generates large‐scale labeled data. However, there is a wrong labeling problem, which affects the performance of RE. Besides, the existing method suffers from the lack of useful semantic features for some positive training instances. To address the above problems, we propose a novel RE model with sentence selection and interaction representation for distantly supervised RE. First, we propose a pattern method based on the relation trigger words as a sentence selector to filter out noisy sentences to alleviate the wrong labeling problem. After clean instances are obtained, we propose the interaction representation using the word‐level attention mechanism‐based entity pairs to dynamically increase the weights of the words related to entity pairs, which can provide more useful semantic information for relation prediction. The proposed model outperforms the strongest baseline by 2.61 in F1‐score on a widely used dataset, which proves that our model performs significantly better than the state‐of‐the‐art RE systems.

Improved Distant Supervision Relation Extraction Based on Edge- Reasoning Hybrid Graph Model

Abstract Distant supervision relation extraction (DSRE) trains a classifier by automatically labeling data through aligning triples in the knowledge base (KB) with large-scale corpora. Training data generated by distant supervision may contain many mislabeled instances, which is harmful to the training of the classifier. Some recent methods show that relevant background information in KBs, such as entity type (e.g., Organization and Book), can improve the performance of DSRE. However, there are three main problems with these methods. Firstly, these methods are tailored for a specific type of information. A specific type of information only has a positive effect on a part of instances and will not be beneficial to all cases. Secondly, different background information is embedded independently, and no reasonable interaction is achieved. Thirdly, previous methods do not consider the side effect of the introduced noise of background information. To address these issues, we leverage five types of background information instead of a specific type of information in previous works and propose a novel edge-reasoning hybrid graph (ER-HG) model to realize reasonable interaction between different kinds of information. In addition, we further employ an attention mechanism for the ER-HG model to alleviate the side effect of noise. The ER-HG model integrates all types of information efficiently and is very robust to the noise of information. We conduct experiments on two widely used datasets. The experimental results demonstrate that our model outperforms the state-of-the-art methods significantly in held-out metric and robustness tests.

Distant Supervision Relation Extraction via adaptive dependency-path and additional knowledge graph supervision

Relation Extraction systems train an extractor by aligning relation instances in Knowledge Base with a large amount of labeled corpora. Since the labeled datasets are very expensive, Distant Supervision Relation Extraction (DSRE) utilizes rough corpus annotated with Knowledge Graph to reduce the cost of acquisition. Nevertheless, the data noise problem limits the performance of the DSRE. Dependency trees can be used to filter the wrong-labeled instances in the distant supervision bag. However, existing dependency tree relation extraction strategies are all based on manually-set paths between the subject and object entities, and suffer from the problem of pruning the trees too aggressively or too insufficiently. To circumvent the shortcomings, in this paper, we propose a novel DSRE framework A2DSRE, based on the Adaptive dependency-path and Additional KG supervision. To obtain the dependency paths related to entity relations adaptively, we introduce an advanced graph neural network—GeniePath into DSRE, which assigns higher weights to those direct neighbor words that contribute more to relation prediction through breadth exploration, and conducts depth exploration to determine the correlation between relations and high-order neighbors. In this way, the irrelevant nodes are pruned while the relevant nodes are kept, our method can obtain more appropriate paths associated with relations. At the same time, to further reduce the noises in the data, we incorporate additional supervision information from the knowledge graph by retracting the margin between the representation of the bag and the pre-training knowledge graph embedding. Extensive numerical experiments validate the effectiveness of our new method.

A gated piecewise CNN with entity-aware enhancement for distantly supervised relation extraction

The piecewise convolutional neural network (PCNN) is an important method for distant supervision relation extraction. However, the existing methods based on the PCNN still have the following shortcomings: these methods lack the consideration of the impacts of entity pairs and the sentence context on word encoding and do not distinguish the different contributions of the three segments in PCNN to relation classification. To solve these problems, we propose a novel gated piecewise CNN with entity-aware enhancement for distantly supervised relation extraction. First, we use a multi-head self-attention mechanism to combine the word embedding with the head/tail entity embedding and relative position embedding to generate an entity-aware enhanced word representation, which is capable of capturing the semantic dependency between each word and entity pair. Then we introduce a global gate to combine each entity-aware enhanced word representation with their average in the input sentence to form the final word representation of the PCNN input. Moreover, to determine the key segments where the most important information for relation classification appears, we design another gate mechanism to assign a different weight to each sentence segment to highlight the effects of key segments on the PCNN. Experiments on New York Times dataset demonstrate that our model significantly outperforms most of the state-of-the-art models.

Improving Neural Relation Extraction with Positive and Unlabeled Learning

We present a novel approach to improve the performance of distant supervision relation extraction with Positive and Unlabeled (PU) Learning. This approach first applies reinforcement learning to decide whether a sentence is positive to a given relation, and then positive and unlabeled bags are constructed. In contrast to most previous studies, which mainly use selected positive instances only, we make full use of unlabeled instances and propose two new representations for positive and unlabeled bags. These two representations are then combined in an appropriate way to make bag-level prediction. Experimental results on a widely used real-world dataset demonstrate that this new approach indeed achieves significant and consistent improvements as compared to several competitive baselines.

Curriculum learning for distant supervision relation extraction

Relation extraction under distant supervision leverages the existing knowledge base to label data automatically, thus greatly reduced the consumption of human labors. Although distant supervision is an efficient method to obtain a large amount of labeled data, the training dataset labeled by distant supervision suffers from noise problem resulting in poor generalization ability of the relation extractor. To alleviate the noise problem, we propose a novel relation extraction method based on curriculum learning. Curriculum learning is utilized to guide the training process of relation extractor, specifically through the predefined curriculum-driven mentor network. Mentor network can dynamically adjust the weights of sentences during training, giving lower weights to noisy sentences and higher weights to truly labeled sentences. Relation extractor and mentor network are trained collaboratively to optimize joint objective. The experimental results show that the proposed method can improve the generalization ability of relation extractor in a noisy environment and obtains better performance for relation extraction.

Curriculum Learning for Distant Supervision Relation Extraction

Curriculum Learning for Distant Supervision Relation Extraction

Deep ranking based cost-sensitive multi-label learning for distant supervision relation extraction

Knowledge base provides a potential way to improve the intelligence of information retrieval (IR) systems, for that knowledge base has numerous relations between entities which can help the IR systems to conduct inference from one entity to another entity. Relation extraction is one of the fundamental techniques to construct a knowledge base. Distant supervision is a semi-supervised learning method for relation extraction which learns with labeled and unlabeled data. However, this approach suffers the problem of relation overlapping in which one entity tuple may have multiple relation facts. We believe that relation types can have latent connections, which we call class ties, and can be exploited to enhance relation extraction. However, this property between relation classes has not been fully explored before. In this paper, to exploit class ties between relations to improve relation extraction, we propose a general ranking based multi-label learning framework combined with convolutional neural networks, in which ranking based loss functions with regularization technique are introduced to learn the latent connections between relations. Furthermore, to deal with the problem of class imbalance in distant supervision relation extraction, we further adopt cost-sensitive learning to rescale the costs from the positive and negative labels. Extensive experiments on a widely used dataset show the effectiveness of our model to exploit class ties and to relieve class imbalance problem.