Abstract

This paper investigates two different neural architectures for the task of relation classification: convolutional neural networks and recurrent neural networks. For both models, we demonstrate the effect of different architectural choices. We present a new context representation for convolutional neural networks for relation classification (extended middle context). Furthermore, we propose connectionist bi-directional recurrent neural networks and introduce ranking loss for their optimization. Finally, we show that combining convolutional and recurrent neural networks using a simple voting scheme is accurate enough to improve results. Our neural models achieve state-of-the-art results on the SemEval 2010 relation classification task.

Highlights

  • Relation classification is the task of assigning sentences with two marked entities to a predefined set of relations

  • recurrent neural networks (RNNs) and convolutional neural networks (CNNs) models were implemented with theano (Bergstra et al, 2010; Bastien et al, 2012)

  • Afterwards, we investigate the impact of different position features on the performance of unidirectional RNNs (position embeddings, position embeddings concatenated with a flag indicating whether the current word is an entity or not, and RNN uni-directional (Baseline, emb dim: 50) uni-directional + position embs uni-directional + position embs + entity flag uni-directional + position indicators bi-directional + position indicators connectionist-bi-directional+position indicators + ranking layer + increase emb dim to 400 ensemble

Read more

Summary

Introduction

Relation classification is the task of assigning sentences with two marked entities to a predefined set of relations. This study investigates two different types of NNs: recurrent neural networks (RNNs) and convolutional neural networks (CNNs) as well as their combination. (1) We propose extended middle context, a new context representation for CNNs for relation classification. (2) We present connectionist bi-directional RNN models which are especially suited for sentence classification tasks since they combine all intermediate hidden layers for their final decision. The ranking loss function is introduced for the RNN model optimization which has not been investigated in the literature for relation classification before. (3) we combine CNNs and RNNs using a simple voting scheme and achieve new state-of-theart results on the SemEval 2010 benchmark dataset

Related Work
Input: Extended Middle Context
Convolutional Layer
Connectionist Bi-directional RNNs
Word Representations
Objective Function
Experiments and Results
Performance of CNNs
Performance of RNNs
Combination of CNNs and RNNs
Comparison with State of the Art
Conclusion
Full Text
Paper version not known

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.