Is ChatGPT an Accurate and Reliable Source of Information for Patients with Vaccine and Statin Hesitancy?

Abstract

Chat Generative Pre-trained Transformer (ChatGPT) is an artificial intelligence (AI) language model that is trained to respond to questions across a wide range of topics. Our aim is to elucidate whether it would be beneficial for patients who are hesitant about vaccines and statins to use ChatGPT. This cross-sectional and observational study was conducted from March 2 to March 30, 2023, using OpenAI ChatGPT-3.5. ChatGPT provided responses to 7 questions related to vaccine and statin hesitancy. The same questions were also directed at physicians. Both the answers from ChatGPT and the physicians were assessed for accuracy, clarity, and conciseness by experts in cardiology, internal medicine, and microbiology, who possessed a minimum of 30 years of professional experience. Responses were rated on a scale of 0-4, and the ChatGPT's average score was compared with that of physicians using the Mann-Whitney U test. The mean scores of ChatGPT (3.78±0.36) and physicians (3.65±0.57) were similar (Mann-Whitney U test p=0.33). The mean scores of ChatGPT were 3.85±0.34 for vaccination and 3.68±0.35 for statin use. The mean scores of physicians were 3.73±0.51 for vaccination and 3.58±0.61 for statin use. There was no statistically significant difference between the mean scores of ChatGPT and physicians for both vaccine and statin use (p=0.403 for vaccination, p=0.678 for statin). ChatGPT did not consider sources of conspiratorial information on vaccines and statins. This study suggests that ChatGPT can be a valuable source of information for guiding patients with vaccine and statin hesitancy.