Hierarchical Graph Attention Network with Heterogeneous Tripartite Graph for Numerical Reasoning over Text

Abstract

Numerical reasoning machine reading comprehension (MRC) is a challenging natural language understanding task, which requires machines to have the numerical reasoning ability to add, subtract, sort and count over text to predict answers. Previous state-of-the-art models typically use graph neural networks (GNNs) to perform reasoning over heterogeneous graphs containing numbers and entities, to enhance their numerical reasoning ability. However, their constructed graphs contain only number nodes or only consider related entities in the same sentence, ignoring the relationship between entities. To alleviate the issue, in this paper, we propose a hierarchical graph attention network with heterogeneous tripartite graph (HGAT-HTG) for the numerical reasoning MRC task. It introduces additional sentence nodes as the intermediary between number nodes and entity nodes to construct the heterogeneous tripartite graph, then its hierarchical graph attention network (GAT) reasoning module first performs coarse-grained reasoning on the whole graph, then performs iterative fine-grained reasoning on the number-sentence subgraph and entity-sentence subgraph respectively, to enhance its numerical reasoning ability for the task. Experimental results indicate that HGAT-HTG model offers significant and consistent improvements, outperforming previous state-of-the-art QDGAT on DROP dataset.