CBKI: A confidence-based knowledge integration framework for multi-choice machine reading comprehension

Abstract

External knowledge has been introduced into pretrained models to enhance answer reasoning in machine reading comprehension tasks. However, most methods directly take the retrieved knowledge as input without considering the reliability and applicability, which may induce information redundancy and generate noise. In this paper, we propose a confidence-based knowledge integration framework (CBKI) to enhance the pretrained model with the ability of active knowledge acquisition and identification. In particular, we introduce a confidence mechanism, which is composed of three modules, namely, confidence estimation, confidence perception and knowledge integration. Specifically, for the candidate knowledge retrieved from the external knowledge base, the confidence estimation module first calculates the probability score of the candidate knowledge effectiveness in answer prediction. Then, the confidence perception module determines whether the candidate knowledge should be adopted for semantic inference at the current step. Finally, the knowledge integration module evaluates how much information should be absorbed from the candidate knowledge and merges the knowledge into the model. Experimental results on standard datasets show that the proposed CBKI is superior to BERT with an increase of 4.08% in accuracy. Furthermore, the CBKI also shows great advantages in robustness and interpretability.