ChatGPT Conversations on Oral Cancer: Unveiling ChatGPT's Potential and Pitfalls

Abstract

BackgroundThe present research investigates the application of Chat Generative Pre-trained Transformer (ChatGPT), an artificial intelligence (AI)-based language processing model that utilize deep learning programs, in responding to various questions related to oral cancer. Materials and methodsA dataset of 25 questions on oral cancer was compiled from diverse sources and categorized into five groups based on key aspects of the disease. ChatGPT version 3.5 was used through a structured approach of submitting questions individually for evaluation. Two experienced oral cancer surgeons independently assessed response accuracy and reproducibility using a grading scale. ResultsChatGPT demonstrated an overall accuracy rate of 80%, providing “comprehensive/correct" responses for 20 questions. No instances of mixed or misleading responses were observed. The highest accuracy was recorded in questions related to basic knowledge on oral cancer and oral cancer prevention, while questions on oral cancer treatment and post-operative aspects showed a mix of “comprehensive/correct" and “incomplete/partially correct" responses. There were no significant differences in response grades between question categories. ConclusionDespite intrinsic constraints and the potential impact of the freely accessible version used, ChatGPT shows promise as a valuable information source for oral cancer-related queries. Caution is advised in relying solely on ChatGPT for the most current and reliable medical information, emphasizing the need for continuing refinements for its widespread use.