Hierarchical multi-task learning with CRF for implicit discourse relation recognition

Abstract

Implicit discourse relation recognition (IDRR) remains an ongoing challenge. Recently, various neural network models have been proposed for this task, and have achieved promising results. However, almost all of them predict multi-level discourse senses separately, which not only ignores the semantic hierarchy of and mapping relationships between senses, but also may result in inconsistent predictions at different levels. In this paper, we propose a hierarchical multi-task neural network with a conditional random field layer (HierMTN-CRF) for multi-level IDRR. Specifically, a HierMTN component is designed to jointly model multi-level sense classifications, with these senses as supervision signals at different feature layers. Consequently, the hierarchical semantics of senses are explicitly encoded into features at different layers. To further exploit the mapping relationships between adjacent-level senses, a CRF layer is introduced to perform collective sense predictions. In this way, our model infers a sequence of multi-level senses rather than separate sense predictions in previous models. In addition, our model can be easily constructed based on existing IDRR models. Experimental results and in-depth analyses on the benchmark PDTB data set show that our model achieves significantly better and more consistent results over several competitive baselines on multi-level IDRR, without additional time overhead.