Automatic estimation of exercises' difficulty levels in a tutoring system for teaching the conversion of natural language into first‐order logic

Abstract

AbstractThe estimation of the difficulty level of exercises is a fundamental aspect of intelligent tutoring systems, and it is necessary in order to achieve better adaptation to the students' needs and maximize learning efficiency. In this article, we present an approach to automatically estimates the difficulty level of exercises in natural language (NL) to first‐order of logic (FOL). The estimation of an exercise's difficulty level is based on the complexity of the corresponding answer, that is the FOL formula, as well as the structure and the semantics of the exercise, that is a natural language sentence and it is carried out in two main steps. Initially, a preliminary estimation is performed based on the complexity of the FOL formula. The system takes as input parameters the number, the type and the order of quantifiers, the number of implications, and the number of different connectives. Afterwards, the final estimation is made based on both semantic aspects of the NL sentence and the structure of the FOL formula. An evaluation study was conducted to assess the system's performance, and the results are very encouraging.