Can We Trust the Information on Allergic Rhinitis From Chat Generated Pre-Trained Transformer?

Abstract

Background and Objectives Chat Generated Pre-Trained Transformer (ChatGPT) is a large language model, which allows consumers to get information with one simple question for free. There are few studies that reported the reliability and usefullness of ChatGPT in the field of otorhinolarynoglogy, so we would like to investigate the reliability of information about allergic rhinitis generated by ChatGPT.Materials and Method We asked ChatGPT 35 questions related to allergic rhinitis based on the textbook from Korean Society of Otorhinolaryngology-Head and Neck Surgery, the clinical guidelines of American Academy of Otolaryngology-Head and Neck Surgery, and the guidelines of Allergic Rhinitis and Its impact on Asthma, and recorded the generated answers. Five specialists from our department were shown the answers in a blind test to assess their reliability. Each question was rated either ‘accurate’ or ‘inaccurate,’ and if the contents were not found in the textbook or the guideline, it was rated as ‘unreliable.’Results Of the 35 questions, 26 (74%) were identified as ‘accurate,’ 9 (26%) were ‘inaccurate,’ and 0 (0%) were ‘unreliable.’ Questions about epidemiology, causes, diagnosis, and prognosis were found to be more accurate than the average, whereas definitions and treatments were less accurate than the average.Conclusion The information about allergic rhinitis generated by ChatGPT was quite reliable, showing that ChatGPT can be helpful in understanding and treating the disease. It is necessary to use the developing medical artificial intelligence wisely.