Reliability and Usefulness of ChatGPT for Inflammatory Bowel Diseases: An Analysis for Patients and Healthcare Professionals.

Abstract

We aimed to evaluate the performance of Chat Generative Pre-trained Transformer (ChatGPT) within the context of inflammatory bowel disease(IBD), which is expected to become an increasingly significant health issue in the future. In addition, the objective of the study was to assess whether ChatGPT serves as a reliable and useful resource for both patients and healthcare professionals. For this study, 20 specific questions were identified for the two main components of IBD, which are Crohn's disease (CD) and ulcerative colitis (UC). The questions were divided into two sets: one set contained questions directed at healthcare professionals while the second set contained questions directed towardpatients. The responses were evaluated with seven-point Likert-type reliability and usefulness scales. The distribution of the reliability and utility scores was calculated into four groups (two diseases and two question sources) by averaging the mean scores from both raters.The highest scores in both reliability and usefulness were obtained from professional sources (5.00± 1.21 and 5.15±1.08, respectively). The ranking in terms of reliability and usefulness, respectively, was as follows: CD questions(4.70±1.26 and 4.75±1.06) and UC questions (4.40±1.21 and 4.55±1.31). The reliability scores of the answers for the professionals were significantly higher than those for the patients (both raters, p=0.032). Conclusion: Despite its capacity for reliability and usefulness in the context of IBD, ChatGPT still has some limitations and deficiencies.The correction of ChatGPT's deficiencies and its enhancement by developers with more detailed and up-to-date information could make it a significant source of information for both patients and medical professionals.