Generative AI in the service of situational judgment tests (SJT) for self-regulation learning during problem solving (SRL-PS)

Abstract

Those involved in education see great importance in promoting self-regulated learning (SRL) and problem-solving (PS) among learners. Teachers are a major factor in promoting SRL-PS in the classroom, but very little is known about whether and how they promote it in their classroom, therefore a tool is needed to assess teachers in this domain. Furthermore, the situational judgment test (SJT) is a potential tool to diagnose teachers’ knowledge. However, such a tool is currently lacking in the SRL-PS domain. Human interaction with generative artificial intelligence (GenAI) enables overcoming difficulties and mutually complementing each other. The purpose of this paper is to present an initial attempt to develop an SJT tool for assessing teachers’ SRL-PS knowledge using the assistance of ChatGPT. This humane-machine interaction led to the formulation of 15 difficulties categories and 20 scenarios that form the basis of the SJT tool for SRL-PS. It was found that scenarios created by the researchers can be complemented by those generated by ChatGPT. In some instances, the scenarios from both sources are quite similar, while in others, those formulated by ChatGPT either expand upon or present an alternative perspective on the difficulty. Moreover, ChatGPT has proposed new scenarios in certain cases. A Significant product of the study is a map that enables the analysis of the pool of scenarios and the identification of over- or underrepresented difficulties categories.