Abstract
Developing entrepreneurship assessment questions that measure higher-order thinking skills (HOTS) is a challenge that requires time and in-depth analysis. Artificial intelligence such as Chat GPT is here to simplify the process. However, its use in creating HOTS-based questions in entrepreneurship still faces obstacles. This research is proposed based on the background of various obstacles in the use of Chat GPT to develop an effective questionnaire that meets the learning objectives. To bridge the gap, this study explores the potential of Chat GPT in crafting assessment questions that focus on higher-order thinking by utilizing existing strategies to enhance critical, analytical, creative and problem-solving thinking in entrepreneurship education. A total of 20 entrepreneurship experts at Surabaya State University were involved in this research. We developed LLM (Large Language Model) using the approach to create HOTS-based multiple-choice questions (MCQs) from an advanced course context. Subsequently, we evaluated 470 MCQs generated by Chat GPT-4 and 337 MCQs generated by Chat GPT-3.5 as well as 595 MCQs corresponding to higher-order cognitive levels. An assessment instrument was used to evaluate the quality of questions generated through 23 items adopted from previous studies. To test the difference in performance, the Mann-Whitney Test was analysed with a p value < 0.05. The results showed that ChatGPT-4 was able to generate questions with clear language usage, providing a single correct answer, as well as a high-quality selection of distractors. In addition, the questions developed by ChatGPT-4 are in accordance with HOTS aspects. This finding can be utilized by teachers as the latest generative model to increase the bank of diverse multiple-choice questions without replacing human expertise.
Published Version
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call