The Quality of AI-Generated Dental Caries Multiple Choice Questions: A Comparative Analysis of ChatGPT and Google Bard Language Models

Abstract

Statement of problemAI technology presents a variety of benefits and challenges for educators. PurposeTo investigate whether ChatGPT and Google Bard (now is named Gemini) are valuable resources for generating multiple-choice questions for educators of dental caries. Material and methodsA book on dental caries was used. Sixteen paragraphs were extracted by an expert consultant based on applicability and potential for developing multiple-choice questions. ChatGPT and Bard language models were used to produce multiple-choice questions based on this input, and 64 questions were generated. Three dental specialists assessed the relevance, accuracy, and complexity of the generated questions. The questions were qualitatively evaluated using cognitive learning objectives and item writing flaws. Paired sample t-tests and two-way analysis of variance (ANOVA) were used to compare the generated multiple-choice questions and answers between ChatGPT and Bard. ResultsThere were no significant differences between the questions generated by ChatGPT and Bard. Moreover, the analysis of variance found no significant differences in question quality. Bard-generated questions tended to have higher cognitive levels than those of ChatGPT. Format error was predominant in ChatGPT-generated questions. Finally, Bard exhibited more absolute terms than ChatGPT. ConclusionsChatGPT and Bard could generate questions related to dental caries, mainly at the cognitive level of knowledge and comprehension. Clinical significanceLanguage models are crucial for generating subject-specific questions used in quizzes, tests, and education. By using these models, educators can save time and focus on lesson preparation and student engagement instead of solely focusing on assessment creation. Additionally, language models are adept at generating numerous questions, making them particularly valuable for large-scale exams. However, educators must carefully review and adapt the questions to ensure they align with their learning goals.