Lexical knowledge enhanced text matching via distilled word sense disambiguation

Abstract

This study proposes a method to improve text matching through integration of lexical knowledge from external resources to model the senses of potentially ambiguous words. Specifically, a sense-aware mechanism is designed wherein a word sense disambiguation (WSD) model is introduced into text matching and both tasks (WSD and matching) are simultaneously optimized via multi-task learning. The proposed WSD is a lightweight model that distills a pre-trained BERT-based model by leveraging the lexical knowledge obtained from WordNet. The sense information obtained from the WSD is integrated into matching explicitly and adaptively through the fusion of the learned sense representations with the word context representations generated from the baseline matching model. The effectiveness of the proposed approach is verified through extensive experiments with three distinct matching-based tasks: natural language inference, paraphrase identification, and answer selection. The results obtained for the respective datasets indicate that the proposed sense knowledge-enhanced matching mechanism outperforms several BERT-based baselines and other recent matching approaches.