Knowing What I Don’t Know: A Generation Assisted Rejection Framework in Knowledge Base Question Answering

Abstract

AbstractExisting Knowledge Base Question Answering (KBQA) systems suffer from the sparsity issue of Knowledge Graphs (KG). To alleviate the issue of KG sparsity, some recent research works introduce external text for KBQA. However, such external information is not always readily available. We argue that it is critical for a KBQA system to know whether it lacks the knowledge to answer a given question. In this paper, we present a novel Generation Assisted Rejection (Gear) framework that identifies unanswerable questions well. Gear can be applied to almost all KBQA systems as an add-on component. Specifically, the backbone of Gear is a sequence-to-sequence model that generates candidate predicates to rerank the original results of a KBQA system. Furthermore, we devise a Probability Distribution Reranking algorithm to ensemble Gear and KBQA since the architectural distinctions between Gear and KBQA are vast. Empirical results and case study demonstrates the effectiveness of our framework in improving the performance of KBQA, particularly in identifying unanswerable questions.KeywordsKnowledge graphKnowledge base question answeringNatural language generation