Abstract

Pradeep Dasigi, Nelson F. Liu, Ana Marasović, Noah A. Smith, Matt Gardner. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Highlights

  • Byzantines were avid players of tavli (Byzantine Greek: τάβλη), a game known in English as backgammon, which is still popular in former Byzantine realms, and still known by the name tavli in Greece

  • Requirement of coreference resolution We found that 78% of the manually analyzed questions cannot be answered without coreference resolution

  • We present QUOREF, a focused reading comprehension benchmark that evaluates the ability of models to resolve coreference

Read more

Summary

Introduction

Byzantines were avid players of tavli (Byzantine Greek: τάβλη), a game known in English as backgammon, which is still popular in former Byzantine realms, and still known by the name tavli in Greece. Aside from Constantinople and Trebizond, other Byzantine cities featured tzykanisteria, most notably Sparta, Ephesus, and Athens, an indication of a thriving urban aristocracy. What is the Byzantine name of the game that Emperor Basil I excelled at? Significant progress has recently been made in reading comprehension research, due to large crowdsourced datasets (Rajpurkar et al, 2016; Bajaj et al, 2016; Joshi et al, 2017; Kwiatkowski et al, 2019, inter alia) These datasets focus largely on understanding local predicateargument structure, with very few questions requiring long-distance entity tracking. These findings indicate that this dataset is an appropriate benchmark for coreference-aware reading comprehension

Dataset Construction
Semantic Phenomena in QUOREF
Baseline Model Performance on QUOREF
Reading Comprehension Models
Heuristic Baselines
Results
Error Analysis
Related Work
Conclusion
Instructions
Examples of Good Questions
Examples of Bad Questions
B Experimental Setup Details
Worker Pool Management

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.