Abstract

Learning commonsense causal and temporal relation between events is one of the major steps towards deeper language understanding. This is even more crucial for understanding stories and script learning. A prerequisite for learning scripts is a semantic framework which enables capturing rich event structures. In this paper we introduce a novel semantic annotation framework, called Causal and Temporal Relation Scheme (CaTeRS), which is unique in simultaneously capturing a comprehensive set of temporal and causal relations between events. By annotating a total of 1,600 sentences in the context of 320 five-sentence short stories sampled from ROCStories corpus, we demonstrate that these stories are indeed full of causal and temporal relations. Furthermore, we show that the CaTeRS annotation scheme enables high inter-annotator agreement for broad-coverage event entity annotation and moderate agreement on semantic link annotation.

Highlights

  • Understanding events and their relations in natural language has become increasingly important for various NLP tasks

  • The real-world order of events is more complex than just a sequence of before relations, we can simplify our set of semantic links to make an approximation: we count the number of links which connect an event entity appearing in position X to an event entity appearing in position X − i

  • In this paper we introduced a novel framework for semantic annotation of event-event relations in commonsense stories, called Causal and Temporal Relation Scheme (CaTeRS)

Read more

Summary

Introduction

Understanding events and their relations in natural language has become increasingly important for various NLP tasks. Earlier work (Chambers and Jurafsky, 2008; Chambers and Jurafsky, 2009; Pichotta and Mooney, 2014; Rudinger et al, 2015) defines verbs as events and uses TimeML-based (Pustejovsky et al, 2003) learning for temporal ordering of events This clearly has many shortcomings, including, but not limited to (1) not capturing a wide range of non-verbal events such as ‘earthquake’, (2) not capturing a more comprehensive set of semantic relations between events such as causality, which is a core relation in stories. Using this semantic framework we annotated 320 stories sampled from ROCStories to extract inter-event semantic structures. This work focuses on stories, the CaTeRS annotation framework for capturing inter-event relations can be applied to other genres

Definition
How to annotate events?
The Case for Embedded Events
The Case for Copulas
The Semantic Relations Between Event Entities
Temporal Relation
Equals Y
Causal Relation
Temporal Implications of Causality
How to annotate semantic relations between events?
Annotating at Story level
Annotated Dataset Analysis
Statistics
Inter-annotator Agreement
Agreement on Event Entities
Agreement on Semantic Links
Related Work
Findings
Conclusion
Full Text
Paper version not known

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.