786 An Analysis of Artificial Intelligence Responses to Common Patient and Family Questions in Cranial Neurosurgery

Abstract

INTRODUCTION: In recent decades, the internet has revolutionized how patients receive and navigate medical information. The development of artificial intelligence (AI) may provide patients and their families with the next landmark development in information transfer. METHODS: ChatGPT was asked 3 questions (pertaining to risks & benefits, length of operation, and length of recovery) about each of these 3 surgeries: transphenoidal resection of pituitary adenoma, deep brain stimulation for Parkinson’s tremor, and hemicraniectomy for a subdural hematoma. Neurosurgeons were asked to appraise the quality of AI responses from 1 (low) to 5 (high), 3 being a threshold that they would be satisfied with a patient reading. The reviewing neurosurgeons were unaware that responses were AI generated, to limit potential bias. To gauge patient accessibility, responses were also graded using readability tools. RESULTS: Quality ratings were collected from 5 neurosurgeons on 3 AI responses for each of the 3 questions, totaling 45 quality ratings. 44 of 45 ratings (97.8%) indicated that the AI generated response was of a high enough quality to show to patients (quality rating > or = to 3). 14 of 45 ratings (31.1%) indicated that AI generated responses were 5 out of 5 quality. AI response readability averaged a Flesch-Kincaid Grade Level of 15.12, and a reach of 46.9% of the English-reading population. CONCLUSIONS: While readability was not optimal, our trials produced physician satisfaction scores which were high enough to warrant further investigation. Furthermore, the readability scores were comparable to many other patient resources. As artificial intelligence grows in technological sophistication, its utilization will no-doubt continue to increase among patients. It will be the duty of medical professionals to optimize this resource for patient use, and increase access to quality patient and family healthcare information.