Heterogeneous-branch integration framework: Introducing first-order predicate logic in Logical Reasoning Question Answering

Abstract

The logical reasoning question-answering is a critical task in natural language processing, as it equips models with human-like logical reasoning intelligence. Existing approaches focus on extracting and leveraging the hidden logical structures within text. However, previous works explore partial logical relationships and neglect the holistic extraction within the text. Moreover, they struggle to fully model logical connections, including long-distance dependencies and local topology information. To address these issues, we propose a novel heterogeneous-branch integration framework. Our framework is based on first-order predicate logic theory and consists of three primary components. First, we construct two heterogeneous logical graphs to model logical relationships within and between propositions. Second, we propose a novel Graph-Masked Transformer with a novel graph-masked multi-head attention mechanism to enable distant node interactions and local sparse relationship modeling. Third, we propose a novel multi-branch fusion module to integrate information from multiple sources and generate answer predictions. The proposed heterogeneous-branch integration framework outperforms the VDGN method by 2.73% in accuracy on the ReClor dataset and 2.15% on the LogiQA dataset. Our code and models will be made available at https://github.com/starry-y/HBI.