BERT and hierarchical cross attention-based question answering over bridge inspection knowledge graph

Abstract

Aiming at the problem of insufficient knowledge service in the field of bridge inspection, this paper proposes a knowledge graph question answering (KGQA) model by using BERT and a novel hierarchical cross-attention mechanism. First, the BERT and static domain dictionaries are used as embedding layer for bridge inspection QA pairs to extract multi-granularity features. Second, in order to extract the topic entities from the domain-specific questions, the bidirectional long short-term memory model is employed to further extract features of the contextual dependency. Finally, the proposed hierarchical cross-attention mechanism realizes information interaction among the questions and knowledge triples, and calculates the similarity from shallow vocabulary and deep semantics. The proposed model is evaluated by using a general KGQA benchmark and a Chinese bridge inspection KGQA dataset. The experimental results show that the proposed approach achieves outstanding performance on both datasets. In addition, a KGQA prototype system is employed as use case to illustrate the application effect in the practical scenario.