Help Transformer Improve Performance in Automatic Mathematics Word Problem-Solving

Abstract

Solving Mathematics Word Problem (MWP) is a basic ability of humanity, which can be mastered by most students at a young age. The existing artificial intelligence system is not good enough in numerical questions, like MWPs. The hard part of this problem is translating natural language sentences in MWP into mathematical expressions or equations. In recent researches, the Transformer network, which proved a great success in machine translation, is applied to automatic mathematic word problem-solving. While previous works have only shown the ability of Transformer model in MWP, how multiple factors such as encoding, decoding, and pre-training affect the performance of Transformer model has not received enough attention. The study is the first to examine the role of these factors experimentally. This paper proposes several methods to improve Transformer network performance in MWPs under the basis of previous studies, achieves higher accuracy compared to the previous state of the art. Pre-training on target tasks dataset improves the translation quality of the Transformer model greatly. Different token encoding and search algorithms also benefit prediction accuracy at the expense of more training and testing time.