Abstract
In 2021, we organized the second iteration of a shared task dedicated to the underlying units used in discourse parsing across formalisms: the DISRPT Shared Task (Discourse Relation Parsing and Treebanking). Adding to the 2019 tasks on Elementary Discourse Unit Segmentation and Connective Detection, this iteration of the Shared Task included for the first time a track on discourse relation classification across three formalisms: RST, SDRT, and PDTB. In this paper we review the data included in the Shared Task, which covers nearly 3 million manually annotated tokens from 16 datasets in 11 languages, survey and compare submitted systems and report on system performance on each task for both annotated and plain-tokenized versions of the data.
Highlights
With the progress achieved on discourse segmentation and connective detection since 2019, this year we decided to extend the competition to a new task: discourse relation classification across frameworks
Results on EDU datasets are difficult to compare to previous work due to the focus on full discourse parsing for frameworks like RST, whereas the relation classification task has focused solely on labeling gold graph structures
Since the relation classification task is new, it is difficult to evaluate the quality of the scores obtained in the results
Summary
Building on rapid progress in NLP for discourse parsing in the past decade (e.g. Iruskieta et al 2013, Zhou et al 2014, Afantenos et al 2012, Braud et al 2017, Wang and Lan 2015, Li et al 2016, Perret et al 2016), the past two years since the DISRPT 2019 Shared Task (Zeldes et al, 2019) have seen unprecedented performance on benchmark datasets for discourse parsing (e.g. Guz and Carenini 2020; Liu et al 2020; Kurfali 2020; Zhang et al 2021b). The DISRPT 2021 Shared Task data comprises 16 datasets in 11 languages, 13 of which target elementary discourse unit segmentation, and 3 dedicated to explicit connective annotation, and all 16 being included in the relation classification task.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
