Assessing the Knowledge of ChatGPT in Answering Questions Regarding Female Urology.

Abstract

With the recent increase in the use of artificial intelligence in the medical field, this study aimed to evaluate the accuracy and adequacy of ChatGPT's responses to questions related to female urology. Intensive internet research was performed to prepare a frequently asked question (FAQs) list. Scientific questions were created in accordance with the European Urology Association (EAU) Non-neurogenic Female Lower Urinary Tract Symptoms Guidelines, EAU Chronic Pelvis Pain Guidelines, and EAU Neuro-Urology Guidelines. All answers by ChatGPT were analysed by two experienced urologists and each answer was scored between 1 and 4 by the physicians. A score of 1 was the highest and showed that the answer was completely true and sufficient. The reproducibility of ChatGPT answers was evaluated by asking each question twice using two different computers. A total of 96 (97.0%) ChatGPT answers about female urology were accurate and sufficient, and categorized as grade 1. Additionally, two (2.0%) answers were scored as grade 2, and one answer (1.0%) was scored as grade 3. None of ChatGPT's responses about female urology were classified as grade 4. In total, 83 questions were prepared according to EAU guidelines recommendations, and ChatGPT gave complete accurate and satisfactory answers for 68 (82.9%) questions. The reproducibility rate was highest for ChatGPT answers for questions related to urinary incontinence, pelvic organ prolapses, and pelvic pain syndromes, and reproducibility rate was 100% for each subgroup. The reproducibility rate for ChatGPT answers was lowest for CPG questions (84.1%). For the first time our study revealed that ChatGPT had an excellent accuracy rate in answering questions related to female urology with 97% success rate. In addition, the outcomes of this study showed that ChatGPT accurately and satisfactorily answered 82.9% of questions about female urology based on EAU guidelines.