On Using Conversational Frameworks to Support Natural Language Interaction in Intelligent Tutoring Systems

Abstract

In this paper, we analyse the potential of conversational frameworks to support the adaptation of existing tutoring systems to a natural language form of interaction. We have based our research on a pilot study, in which the open-source machine learning framework Rasa has been used to build a conversational agent that interacts with an existing Intelligent Tutoring System (ITS) called Hypergraph Based Problem Solver (HBPS). This agent has been seamlessly integrated into the ITS to replace the previously available button-based user interface and allow the user to interact with HBPS in natural language. Once appropriately trained, the conversational agent was capable of identifying the intention of a given user utterance and extracting the relevant entities related to the message content, with average weighted F1-scores of 0.965 and 0.989 for intents and entities, respectively. Methodological guidelines are provided to generate a realistic training set that enables the creation of the required Natural Language Understanding (NLU) model and also evaluate the resulting system. These guidelines can be easily exported to other ITS and contexts, to provide an enhanced interaction based on natural language processing methods.