Effectiveness of an Intelligent Question Answering System for Teaching Financial Literacy: A Pilot Study

Abstract

AbstractQuestion answering systems have shown promise as a learning tool in several fields but are yet to be used in the financial literacy domain. The objective of this research is to build an intelligent question answering (IQA) system using state-of-the-art natural language processing technology and test its effectiveness in helping students learn personal finance. The purpose of the system is twofold—one as an intelligent system for practicing and two as a scaffolding tool for finding answers to questions. The IQA system consists of three modules—the question answering module, the evaluation module, and the feedback module—and is built using Google’s state-of-the-art language model for Natural Language Processing: Bidirectional Encoder Representations from Transformers (BERT). The primary research question we attempt to answer is whether the use of the IQA system results in increased learning in the domain of personal finance and if so how. We find significant learning gains (effect size = 1.89) with the affordances of the IQA system contributing to the gains. Students felt that the IQA system helped them learn better mainly by allowing them to practice at their own pace and by providing them with indications of where they had gone wrong and allowing them to read, think, and re-attempt the answer. These promising results suggest that an IQA system could be effective as a support tool for improving learning of personal finance.KeywordsIntelligent question answeringNatural language processingFinancial literacy