Abstract

Transfer learning (TL) proposes to enhance machine learning performance on a problem, by reusing labeled data originally designed for a related problem. In particular, domain adaptation consists, for a specific task, in reusing training data developed for the same task but a distinct domain. This is particularly relevant to the applications of deep learning in Natural Language Processing, because those usually require large annotated corpora that may not exist for the targeted domain, but exist for side domains. In this paper, we experiment with TL for the task of Relation Extraction (RE) from biomedical texts, using the TreeLSTM model. We empirically show the impact of TreeLSTM alone and with domain adaptation by obtaining better performances than the state of the art on two biomedical RE tasks and equal performances for two others, for which few annotated data are available. Furthermore, we propose an analysis of the role that syntactic features may play in TL for RE.

Highlights

  • A bottleneck problem for training deep learningbased architecture on text is the availability of large enough annotated training corpora

  • We compare two deep learning strategies for Relation Extraction (RE): (1) the MultiChannel Convolutional Neural Network (CNN) (MCCNN) model (Quan et al, 2016), which has been successfully applied to the task of proteinprotein interaction extraction without using any syntactic feature as input and (2) the TreeLSTM model (Tai et al, 2015), which is designed for considering dependency trees

  • We empirically showed that a Transfer learning (TL) strategy can benefit biomedical RE tasks when using the TreeLSTM model, whereas it is mainly harmful with a model that does not consider syntax

Read more

Summary

Introduction

A bottleneck problem for training deep learningbased architecture on text is the availability of large enough annotated training corpora. Deep learning methods have demonstrated good ability for RE (Zeng et al, 2014), but one of their drawbacks is that, in order to obtain reasonable performances, they generally require a large amount of training data, i.e., text corpora where entities and relationships between them are annotated The assembly of this kind of domain- and task-specific corpora, such as those of interest in biomedicine, is time consuming and expensive because it involves complex entities (e.g., genomic variations, complex phenotypes), complex relationships (which may be hypothetical, contextualized, negated, n-ary) and requires trained annotators. This explains why only few and relatively small (i.e., few hundreds of sentences) corpora are available for some biomedical RE tasks, making these resources valuable. We propose a syntax-based analysis, using both quantitative criteria and qualitative observations, to better understand the role of syntactic features in the TL behavior

Deep Learning Models for Relation Extraction
Transfer learning
Models
Input layer
Composition layers
TreeLSTM
Scoring layer
Datasets
Target corpora
Source corpora
Training and Experimental Settings
Transfer learning experiment
Comparison with the state of the art
On the role of syntactic features in transfer learning
Findings
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call