Graph matching based reasoner: A symbolic approach to question answering

Abstract

Text understanding and reasoning are among the core areas of artificial intelligence. Even a total solution to automatic text understanding and reasoning is still beyond the current techniques, thus it is time to build the stepping stones to solve a more straightforward problem set like question-answering (QA) without ambiguous utterances in the contexts or questions. The reported state-of-the-art approaches to this kind of problem are nearly all connectionist models based on neural networks. Significant progress has even been made in this direction. It is still hard for the pure connectionist models to handle logical reasoning. They generally suffer from the longstanding drawback of poor explainability and sensitivity to data noise and distribution. In this paper, we propose a complementary symbolic approach, GMR (Graph Matching based Reasoner) to QA — it automatically generates reasoning rules in the form of graphs from the training set and uses the generated rules to infer answers to the questions in the test set via graph matching. By employing this symbolic approach, 20 tasks in bAbI are solved with an average accuracy of 99.38%, and it outperforms the state-of-the-art for a real-life QA dataset WikiTableQuestions. After analyzing the accuracy of the basic evaluation indicators, we studied the generalization ability of the model in the paper, including the anti-noise ability, the convergence of the model, the stability of the model, the complexity of the algorithm, and the uncertainty of the parameters. Through comprehensive analysis and comparison, our model is stronger than the neural network model regarding anti-noise interference. Compared with the neural network model, our model performs very well in multi-tasking, and the stability of the model is quite high. The diversity of tasks did not reduce the stability of the model. We have conducted a comprehensive analysis and comparison of the parameter uncertainties. Our model can optimally select parameter configurations and will not cause a sharp drop in performance due to the parameter uncertainties. Finally, we describe the complexity of the GMR method and the optimal configuration of its parameters.