Flesch-Kincaid Research Articles

Comparing Provider and ChatGPT Responses to Breast Reconstruction Patient Questions in the Electronic Health Record.

Patient-directed Electronic Health Record (EHR) messaging is used as an adjunct to enhance patient-physician interactions but further burdens the physician. There is a need for clear electronic patient communication in all aspects of medicine, including plastic surgery. We can potentially utilize innovative communication tools like ChatGPT. This study assesses ChatGPT's effectiveness in answering breast reconstruction queries, comparing its accuracy, empathy, and readability with healthcare providers' responses. Ten deidentified questions regarding breast reconstruction were extracted from electronic messages. They were presented to ChatGPT3, ChatGPT4, plastic surgeons, and advanced practice providers for response. ChatGPT3 and ChatGPT4 were also prompted to give brief responses. Using 1-5 Likert scoring, accuracy and empathy were graded by 2 plastic surgeons and medical students, respectively. Readability was measured using Flesch Reading Ease. Grades were compared using 2-tailed t tests. Combined provider responses had better Flesch Reading Ease scores compared to all combined chatbot responses (53.3 ± 13.3 vs 36.0 ± 11.6, P < 0.001) and combined brief chatbot responses (53.3 ± 13.3 vs 34.7 ± 12.8, P < 0.001). Empathy scores were higher in all combined chatbot than in those from combined providers (2.9 ± 0.8 vs 2.0 ± 0.9, P < 0.001). There were no statistically significant differences in accuracy between combined providers and all combined chatbot responses (4.3 ± 0.9 vs 4.5 ± 0.6, P = 0.170) or combined brief chatbot responses (4.3 ± 0.9 vs 4.6 ± 0.6, P = 0.128). Amid the time constraints and complexities of plastic surgery decision making, our study underscores ChatGPT's potential to enhance patient communication. ChatGPT excels in empathy and accuracy, yet its readability presents limitations that should be addressed.

Text readability: its impact on reading comprehension and reading time

Recently, the readability of texts has become the focus of reading research because it is believed to have implications for reading comprehension, which is of utmost importance in the field of English as a foreign language (EFL), particularly in the teaching, learning and assessment of reading comprehension. Unfortunately, the influence of text readability on reading comprehension (and reading time) has not been well studied in the EFL context. Most text readability studies are conducted in medical contexts, but these studies are often limited in predicting readability scores for sample texts. To address this gap, the current study aimed to evaluate the influence of text readability levels (based on the Flesch-Kincaid grade level (FKGL)) on students’ reading comprehension and reading time. Data were collected through reading test and analyzed using SPSS version 22. The Friedman test revealed that the distribution of students' reading comprehension score (X2=197.532, p=0.000) and reading time (X2=215.323, p=0.000) are different in each text, suggesting that the readability of texts has a significant influence on both. This study contributed to the practices of reading instruction and assessment. Limitations and suggestions for further research are briefly discussed.

Enhancing patient understanding in obstetrics: The role of generative AI in simplifying informed consent for labor induction with oxytocin.

Informed consent is a cornerstone of ethical medical practice, particularly in obstetrics where procedures like labor induction carry significant risks and require clear patient understanding. Despite legal mandates for patient materials to be accessible, many consent forms remain too complex, resulting in patient confusion and dissatisfaction. This study explores the use of Generative Artificial Intelligence (GAI) to simplify informed consent for labor induction with oxytocin, ensuring content is both medically accurate and comprehensible at an 8th-grade readability level. GAI-generated consent forms streamline the process, automatically tailoring content to meet readability standards while retaining essential details such as the procedure's nature, risks, benefits, and alternatives. Through iterative prompts and expert refinement, the AI produces clear, patient-friendly language that bridges the gap between medical jargon and patient comprehension. Flesch Reading Ease scores show improved readability, meeting recommended levels for health literacy. GAI has the potential to revolutionize healthcare communication by enhancing patient understanding, promoting shared decision-making, and improving satisfaction with the consent process. However, human oversight remains critical to ensure that AI-generated content adheres to legal and ethical standards. This case study demonstrates that GAI can be an effective tool in creating accessible, standardized, yet personalized consent documents, contributing to better-informed patients and potentially reducing malpractice claims.

Exploring the role of AI-driven chatbots in patient care: a critical evaluation amidst healthcare staff shortages

Abstract Background Amidst the backdrop of staff shortages in healthcare systems, patients and their families are increasingly turning to chatbots, powered by Large Language Models (LLMs), for information about their medical conditions. These AI-driven chatbots, capable of generating human-like responses across a broad range of topics, have become a prevalent tool in the healthcare landscape. Given the proliferation of these chatbots, it is crucial to evaluate the quality and accuracy of the responses they provide. Methods We selected five freely accessible chatbots (Bard, Microsoft Copilot, PiAI, ChatGPT, and ChatSpot) for our study. These chatbots were posed questions spanning three medical fields: cardiology, cardio-oncology, and cardio-rheumatology. The responses generated by the chatbots were then compared against established guidelines from the European Society of Cardiology, American Academy of Dermatology, and American Society of Clinical Oncology. In addition to the content, the readability of the responses was evaluated using four different readability scales: the Flesch Reading Scale, Gunning Fog Scale Level, Flesch-Kincaid Grade Level, and Dale-Chall Score. To assess the accuracy of the responses in accordance with the medical guidelines, two independent medical professionals rated them on a 3-point Likert scale (0 - incorrect, 1 - partially correct or incomplete, 2 - correct). This allowed us to gauge the compliance of the chatbot responses with the medical guidelines. Results Results: In our study, we posed a total of 45 questions to each of the chatbots. Out of the five chatbots, Microsoft Copilot, PiAI, and ChatGPT were able to respond to all the questions. The length of the responses varied, with PiAI providing the shortest average response length of 7.26 words, and Bard providing the longest at 18.9 words. In terms of readability, the Flesch Reading Ease Scale scores ranged from 17.67 (ChatGPT) to 39.34 (Bard), indicating the relative complexity of the responses. The Flesch-Kincaid Grade Level, which reflects the academic grade level required to comprehend the text, ranged from 14.02 (PiAI) to 15.97 (ChatGPT). The Gunning Fog Scale Level, another measure of readability, varied from 15.77 (Bard) to 19.73 (ChatGPT). Lastly, the Dale-Chall Score, which assesses the understandability of the text, ranged from 10.24 (Bard) to 11.87 (ChatGPT). These results highlight the variability in the readability and complexity of responses generated by different chatbots. Readability analysis is presented in table 1. Conclusion This study indicates that chatbots vary in length, quality and readability. They answer each question in their way, based on data they have pulled from the network. Our data suggests that people who want information from a chatbot need to be careful and verify the answers they get.

Assessing the Responses of Large Language Models (ChatGPT-4, Claude 3, Gemini, and Microsoft Copilot) to Frequently Asked Questions in Retinopathy of Prematurity: A Study on Readability and Appropriateness.

To assess the appropriateness and readability of responses provided by four large language models (LLMs) (ChatGPT-4, Claude 3, Gemini, and Microsoft Co-pilot) to parents' queries pertaining to retinopathy of prematurity (ROP). A total of 60 frequently asked questions were collated and categorized into six distinct sections. The responses generated by the LLMs were evaluated by three experienced ROP specialists to determine their appropriateness and comprehensiveness. Additionally, the readability of the responses was assessed using a range of metrics, including the Flesch-Kincaid Grade Level (FKGL), Gunning Fog (GF) Index, Coleman-Liau (CL) Index, Simple Measure of Gobbledygook (SMOG) Index, and Flesch Reading Ease (FRE) score. ChatGPT-4 demonstrated the highest level of appropriateness (100%) and performed exceptionally well in the Likert analysis, scoring 5 points on 96% of questions. The CL Index and FRE scores identified Gemini as the most readable LLM, whereas the GF Index and SMOG Index rated Microsoft Copilot as the most readable. Nevertheless, ChatGPT-4 exhibited the most intricate text structure, with scores of 18.56 on the GF Index, 18.56 on the CL Index, 17.2 on the SMOG Index, and 9.45 on the FRE score. This suggests that the responses demand a college-level comprehension. ChatGPT-4 demonstrated higher performance than other LLMs in responding to questions related to ROP; however, its texts were more complex. In terms of readability, Gemini and Microsoft Copilot were found to be more successful. [J Pediatr Ophthalmol Strabismus. 20XX;XX(X):XXX-XXX.].

Validity and reliability of a physical literacy knowledge, attitudes, self-efficacy and behaviors questionnaire for early childhood educators (PLKASB-ECE): An exploratory factor analysis.

Nurturing physical literacy in young children offers a unique opportunity to address global physical inactivity trend. Early childhood education and care (ECEC) environments, with their extensive reach into this age group, and early childhood educators, through their daily interactions with children, are strategically positioned to influence children's physical literacy development. However, enhancing educators' ability to foster physical literacy requires valid and reliable assessment tools to measure holistic physical literacy constructs (i.e., cognitive, affective, behavioral) to assess the impact of physical literacy educational interventions. The purpose of this study was to develop a holistic digital tool to measure physical literacy knowledge, attitudes, self-efficacy, and behaviors of early childhood educators (PLKASB-ECE) for both their professional teaching context as well as the educator's own personal physical literacy behaviours. This study was conducted in two phases. Phase 1 involved item generation and content validation. Phase 2 employed a cross-sectional validation study design to assess the psychometric properties of the PLKASB-ECE tool. Based on a literature review an initial 19-item instrument was developed that subsequently underwent three rounds of expert content validation. Six additional items were added, resulting in a final 25-item self-report measure with a readability score equivalent to an 8th grade reading level (Flesch-Kincaid Grade Level: 8.7). This included 1 global rating item, 2 qualitative response items, 7 knowledge items, and 15 items addressing physical literacy attitudes, self-efficacy and behaviors using a 7-point Likert response scale. The PLKASB-ECE tool was administered to 470 educators in Alberta, Canada between 2019 to 2022. The 15 items were subjected to exploratory factor analysis and resulted in a five-factor scale with one item not loading. The five-factor scale held with the final 14 items with loadings ranging from 0.481 to 0.886, Cronbach's alpha ranging from 0.70 to 0.82, with ordinal omega ranging from 0.72 to 0.82. Usability, as assessed by completion time, was 8.15 minutes. These findings demonstrate good indices of reliability and validity for the PLKASB-ECE tool. This tool will be valuable as a research outcome measure for assessing educational interventions aimed at enhancing educators' understanding, confidence, and strategies for fostering holistic concepts of physical literacy in young children within ECECs.

Evaluating the reliability of the responses of large language models to keratoconus-related questions

ABSTRACT Clinical relevance Artificial intelligence has undergone a rapid evolution and large language models (LLMs) have become promising tools for healthcare, with the ability of providing human-like responses to questions. The capabilities of these tools in addressing questions related to keratoconus (KCN) have not been previously explored. Background In this study, the responses were evaluated from three LLMs – ChatGPT-4, Copilot, and Gemini – to common patient questions regarding KCN. Methods Fifty real-life patient inquiries regarding general information, aetiology, symptoms and diagnosis, progression, and treatment of KCN were presented to the LLMs. Evaluations of the answers were conducted by three ophthalmologists with a 5-point Likert scale ranging from ‘strongly disagreed’ to ‘strongly agreed’. The reliability of the responses provided by LLMs was evaluated using the DISCERN and the Ensuring Quality Information for Patients (EQIP) scales. Readability metrics (Flesch Reading Ease Score, Flesch-Kincaid Grade Level, and Coleman-Liau Index) were calculated to evaluate the complexity of responses. Results ChatGPT-4 consistently scored 3 points or higher for all (100%) its responses, while Copilot had five (10%) and Gemini had two (4%) responses scoring 2 points or below. ChatGPT-4 achieved a ‘strongly agree’ rate of 74% across all questions, markedly superior to Copilot at 34% and Gemini at 42% (p < 0.001); and recorded the highest ‘strongly agree’ rates in general information and symptoms & diagnosis categories (90% for both). The median Likert scores differed among LLMs (p < 0.001), with ChatGPT-4 scoring highest and Copilot scoring lowest. Although ChatGPT-4 exhibited more reliability based on the DISCERN scale, it was characterised by lower readability and higher complexity. While all LLMs provided responses categorised as ‘extremely difficult to read’, the responses provided by Copilot showed higher readability. Conclusions Despite the responses provided by ChatGPT-4 exhibiting lower readability and greater complexity, it emerged as the most proficient in answering KCN-related questions.

Improving Biomedical Science Literacy and Patient-Directed Knowledge of Tuberculosis (TB): A Cross-Sectional Infodemiology Study Examining Readability of Patient-Facing TB Information

BackgroundTuberculosis (TB) continues be the leading cause of death globally due to an infectious agent. There is a paucity of data describing the readability of patient-facing TB information for service users. The aim of this study was to calculate the readability of multiple global TB information sources.MethodsInformation on tuberculosis (n = 150 sources) included nine categories, Patient-facing information: WHO publications (n = 17), International governments (n = 19), Hospitals (n = 10), Non-government organisations (NGOs)/charities (n = 20), Cochrane Plain Language Summaries (n = 20); LabTestsOnlineUK (n = 4) and Scientific-facing information: Clinical trials (n = 20), Cochrane abstracts (n = 20), Scientific abstracts (n = 20). Readability was calculated using Readable software, defined by (i) Flesch Reading Ease (FRE), (ii) Flesch-Kincaid Grade Level (FKGL), (iii) Gunning Fog Index and (iv) SMOG Index and two text metrics [words/sentence, syllables/word].ResultsMean readability values for TB information for the FRE and FKGL were 35.6 ± 1.6 (standard error of mean (SEM)) (US Target ≥60; UK Target ≥90) and 12.3 ± 0.3 (US Target ≤8; UK Target ≤6), respectively, with mean words per sentence and syllables per word of 17.2 and 1.8, respectively. Cochrane Plain Language Summaries had similar readability scores to their matching scientific abstract (p = 0.15). LabTestsOnlineUK yielded a mean FRE score of 51.5 ± 1.2, a mean FKGL score of 10.2 ± 0.5 and text metric scores of 16.7 ± 2.3 and 1.6, for words per sentence and syllables per word, respectively. In descending order, TB information from international governments, hospitals and LabTestsOnlineUK were the most readable (FRE = 57.9, 54.1 and 51.5, respectively), whereas scientific abstracts and Cochrane abstracts were the most difficult to read (13.0 and 30.2, respectively).ConclusionPatient-facing TB information analysed had poor readability. Effective communication of biomedical science concepts and information relating to TB is vital for service users to enhance their health literacy of tuberculosis, thereby promoting better clinical outcomes. Biomedical scientists are important custodians of scientific information for their service user populations, including other healthcare professionals within the TB multidisciplinary (MDT) team and patient service users. When preparing TB information, this should be checked and modified in real time employing readability calculators, to align with health readability targets.

Assessing the readability and quality of online written information on epistaxis.

The objective of this study was to assess the readability and quality of online written information on epistaxis. The terms 'epistaxis' and 'nosebleed' were entered into Google. The first six webpages generated for each search term were screened. Readability was assessed using the Flesch-Kincaid Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG) Index and Gunning Fog Index (GFOG). Quality was assessed using the DISCERN instrument. Spearman's correlation between quality and readability was calculated. A total of 37 websites met the inclusion criteria. The mean and 95% confidence intervals for FRES, FKGL, SMOG and GFOG were 58.9 (55.3-62.5), 9.65 (8.74-10.6), 9.18 (8.57-9.8) and 12.5 (11.5-13.5), respectively. The DISCERN score was 34.3 (32.0-36.5). Weak negative correlation was noted between DISCERN and FRES (rs = -0.15, p = 0.36). Online information on epistaxis is generally of poor quality and low readability.

Utilization of Artificial Intelligence to Improve Equitable Healthcare Access for Breast Implant Patients

Abstract The recently mandated US Food and Drug Administration (FDA) patient decision checklists were developed with the goal of improving the informed decision-making process for patients considering breast implants. Yet, prior research has demonstrated that these checklists are written at reading levels far in excess of that recommended by the National Institutes of Health and the American Medical Association. Patients considering breast implants are required to sign these forms prior to surgery, even if they have little to no comprehension of their contents. This study aims to improve the accessibility, and therefore the utility, of the mandated FDA patient literature using the assistance of artificial intelligence (AI). Patient decision checklists were obtained from the three most utilized breast implant manufacturers in the United States– Allergan, Mentor, and Sientra. AI was then asked to synthesize a novel patient decision checklist, written at the 6th grade reading level, using these checklists as source material. The AI-assisted checklist was edited by plastic surgeons for both formatting and content. Readability analysis of all documents was performed using the Flesch Reading-Ease Score, Flesch-Kincaid Grade Level, Gunning-Fog Index, Coleman-Liau Index, Simplified Measure of Gobbledygook, and Automated Readability Index. Textual analysis included sentence count, word count, number and percentage of complex words, average number of words per sentence, and average number of syllables per word. The overall readability of Allergan, Mentor, and Sientra patient checklists correlated with the college reading level. These documents were of a statistically significantly higher reading level the AI-assisted checklist, which was written at the recommended 6th grade level. Text composition analysis similarly demonstrated substantial differences between the AI-assisted and FDA mandated literature. The currently mandated breast implant patient checklists are written at a college reading level and are inaccessible to the average patient. We propose a new patient decision checklist, generated with the assistance of artificial intelligence, in order to improve healthcare access within plastic surgery. This simplified material can be used as an adjunct to the current checklists in order to improve shared decision making.

Development and Validation of an Artificial Intelligence-Assisted Patient Education Material for Ostomy Patients: A Methodological Study.

To develop and test the validity of an artificial intelligence-assisted patient education material for ostomy patients. A methodological study. The study was carried out in two main stages and five steps: (1) determining the information needs of ostomy patients, (2) creating educational content, (3) converting the educational content into patient education material, (4) validation of patient education material based on expert review and (5) measuring the readability of the patient education material. We used ChatGPT 4.0 to determine the information needs and create patient education material content, and Publuu Online Flipbook Maker was used to convert the educational content into patient education material. Understandability and applicability scores were assessed using the Patient Education Materials Assessment Tool submitted to 10 expert reviews. The tool inter-rater reliability was determined via the intraclass correlation coefficient. Readability was analysed using the Flesch-Kincaid Grade Level, Gunning Fog Index and Simple Measure of Gobbledygook formula. The mean Patient Education Materials Assessment Tool understandability score of the patient education material was 81.91%, and the mean Patient Education Materials Assessment Tool actionability score was 85.33%. The scores for the readability indicators were calculated to be Flesch-Kincaid Grade Level: 8.53, Gunning Fog: 10.9 and Simple Measure of Gobbledygook: 7.99. The AI-assisted patient education material for ostomy patients provided accurate information with understandable and actionable responses to patients, but is at a high reading level for patients. Artificial intelligence-assisted patient education materials can significantly increase patient information rates in the health system regarding ease of practice. Artificial intelligence is currently not an option for creating patient education material, and their impact on the patient is not fully known. The study followed the STROBE checklist guidelines. No patient or public contributions.

Retrograde Cricopharyngeal Dysfunction: A Google Search Analysis.

No studies describe what patients search for online in relation to retrograde cricopharyngeal dysfunction (RCPD). Our objectives were to describe the Google search volume for RCPD, identify the most common queries related to RCPD, and evaluate the available online resources. Observational. Google Database. Using Ahrefs and Search Response, Google search volume for RCPD and "People Also Ask" (PAA) questions were documented. PAA questions were categorized based on intent, and the websites were categorized on source. The quality and readability of the sources were determined using the Journal of the American Medical Association (JAMA) benchmark criteria, Flesch Reading Ease score, and Flesch-Kincaid Grade Level. Search volume for RCPD-related content has continually increased since 2021, with a combined average volume of 6287 searches per month. Most PAA questions were related to technical details (61.07%) and treatments (32.06%) for RCPD. Websites provided to answer these questions were most often from academic (25.95%) and commercial (22.14%) sources. None of the sources met the criteria for universal readability, and only 15% met all quality metrics set forth by JAMA. Interest in RCPD is at an all-time high, with information related to its diagnosis and treatment most popular among Google users. Significantly, none of the resources provided by Google met the criteria for universal readability, preventing many patients from fully comprehending the information presented. Future work should aim to address questions related to RCPD in a suitable way for all patient demographics.

Bridging Gaps in Health Literacy for Cleft Lip and Palate: The Role of Artificial Intelligence and Interactive Educational Materials.

Our goal is to address the health literacy gap concerning cleft lip and palate using artificial intelligence (AI), specifically OpenAI's ChatGPT-4, to enhance the readability of patient educational materials (PEMs). Additionally, we aim to explore the integration of interactive educational materials into these resources. A cross-sectional study. Websites of craniofacial teams approved by the American Cleft Palate-Craniofacial Association (ACPA). PEMs collected from these websites. First, we examined the PEMs for interactive educational content such as videos, photos, case examples, and audio clips. Subsequently, we used the WebFX online readability calculator and Grammarly to evaluate the texts' readability and clarity. After that, we used ChatGPT-4 to simplify the PEM texts to a 6th-grade reading level and re-evaluated their readability and clarity. Readability levels (grade level) and clarity scores of PEMs before and after AI modification, and the presence of interactive educational content. Initial evaluations using the Flesch-Kincaid Grade Level formula showed that the texts were written at a median 9th-grade reading level. Modifications using ChatGPT-4 significantly improved readability, achieving a median 6th-grade level and enhanced clarity. Only 27% of the websites featured interactive educational content, indicating a gap in utilizing educational enhancements. The use of AI improved the readability and clarity of patient educational materials, aligning with recommended standards. However, the underutilization of interactive tools suggests a need for broader integration of these resources to optimize patient education and engagement.

Using ChatGPT to Provide Patient-Specific Answers to Parental Questions in the PICU.

To determine if ChatGPT can incorporate patient-specific information to provide high-quality answers to parental questions in the PICU. We hypothesized that ChatGPT would generate high-quality, patient-specific responses. In this cross-sectional study, we generated assessments and plans for 3 PICU patients with respiratory failure, septic shock, and status epilepticus and paired them with 8 typical parental questions. We prompted ChatGPT with instructions, an assessment and plan, and 1 question. Six PICU physicians evaluated the responses for accuracy (1-6), completeness (yes/no), empathy (1-6), and understandability (Patient Education Materials Assessment Tool, PEMAT, 0% to 100%; Flesch-Kincaid grade level). We compared answer quality among scenarios and question types using the Kruskal-Wallis and Fischer's exact tests. We used percent agreement, Cohen's Kappa, and Gwet's agreement coefficient to estimate inter-rater reliability. All answers incorporated patient details, utilizing them for reasoning in 59% of sentences. Responses had high accuracy (median 5.0, [interquartile range (IQR), 4.0-6.0]), empathy (median 5.0, [IQR, 5.0-6.0]), completeness (97% of all questions), and understandability (PEMAT % median 100, [IQR, 87.5-100]; Flesch-Kincaid level 8.7). Only 4/144 reviewer scores were <4/6 in accuracy, and no response was deemed likely to cause harm. There was no difference in accuracy, completeness, empathy, or understandability among scenarios or question types. We found fair, substantial, and almost perfect agreement among reviewers for accuracy, empathy, and understandability, respectively. ChatGPT used patient-specific information to provide high-quality answers to parental questions in PICU clinical scenarios.

Appropriateness and readability of Google Bard and ChatGPT-3.5 generated responses for surgical treatment of glaucoma.

To evaluate the appropriateness and readability of the medical knowledge provided by ChatGPT-3.5 and Google Bard, artificial-intelligence-powered conversational search engines, regarding surgical treatment for glaucoma. In this retrospective, cross-sectional study, 25 common questions related to the surgical management of glaucoma were asked on ChatGPT-3.5 and Google Bard. Glaucoma specialists graded the responses' appropriateness, and different scores assessed readability. Appropriate answers to the posed questions were obtained in 68% of the responses with Google Bard and 96% with ChatGPT-3.5. On average, the responses generated by Google Bard had a significantly lower proportion of sentences, having more than 30 and 20 syllables (23% and 52% respectively) compared to ChatGPT-3.5 (66% and 82% respectively), as noted by readability. Google Bard had significantly (p<0.0001) lower readability grade scores and significantly higher "Flesch Reading ease score", implying greater ease of readability amongst the answers generated by Google Bard. Many patients and their families turn to LLM chatbots for information, necessitating clear and accurate content. Assessments of online glaucoma information have shown variability in quality and readability, with institutional websites generally performing better than private ones. We found that ChatGPT-3.5, while precise, has lower readability than Google Bard, which is more accessible but less precise. For example, the Flesch Reading Ease Score was 57.6 for Google Bard and 22.6 for ChatGPT, indicating Google Bard's content is easier to read. Moreover, the Gunning Fog Index scores suggested that Google Bard's text is more suitable for a broader audience. ChatGPT's knowledge is limited to data up to 2021, whereas Google Bard, trained with real-time data, offers more current information. Further research is needed to evaluate these tools across various medical topics. The answers generated by ChatGPT-3.5™ AI are more accurate than the ones given by Google Bard. However, comprehension of ChatGPT-3.5™ answers may be difficult for the public with glaucoma. This study emphasized the importance of verifying the accuracy and clarity of online information that glaucoma patients rely on to make informed decisions about their ocular health. This is an exciting new area for patient education and health literacy.

7056 Can ChatGPT Educate Multicultural Patients About Metabolic Syndrome?

Abstract Disclosure: G. Wu: None. E. Cheng: None. V.V. Toram: None. K. Rosen: None. A. Wong: None. W. Zhao: None. M. Del Buono: None. Background: Metabolic Syndrome, which affects Asians, is a major risk factor for diabetes. According to the National Institute of Health (NIH), 1 in 3 Americans have Metabolic Syndrome. Diabetes affects 30 million Americans, and 8 million more are undiagnosed. ChatGPT and Bard are Artificial Intelligence (AI) mediated LLMs launched on 11/30/2022 and 3/21/2023, respectively, that generate real-time conversational responses. Purpose: Can ChatGPT educate our multicultural patients about metabolic syndrome? Methods: 1. User prompt into ChatGPT and Bard: What is metabolic syndrome? I am a South Asian male, 40 years old, with a cholesterol level of 250 and a fasting blood sugar level of 120. The exact prompt was used with an Asian male as a substitute. 2. The prompt was translated into Telugu, Hindi, Simplified Chinese, and Traditional Chinese via Google Translate. 3. Flesch-Kincade (FK) method of ascertaining reading level was used. FK is used by the US Department of Education. Results: Flesch-Kincaid reading ease: ChatGPT: English=29.2, Telugu=60.1, Hindi=33.4, Chinese (Trad.)=14.5, Chinese (Simp.)=22.6 Bard: English=35.3, Telugu=52, Hindi=51.7, Chinese (Trad.)=52, Chinese (Simp.)=36.8 Language Group Comparison to English: ChatGPT: Indian Languages p=0.004; Chinese Languages p=0.081. Bard: Indian Languages p=0.007; Chinese Languages p=0.136. Of note, the Telegu response has one incorrect response. Conclusions: In conclusion, ChatGPT and Bard can provide factual responses based on English prompts. Foreign prompts can yield inaccurate information. Physicians and their teams still need to supervise patient usage. Presentation: 6/2/2024

A Critical Assessment of the Quality and Readability of Information on Gastric Cancer Available on the Internet for Patients.

Introduction It is now commonplace for patients to consult the internet with health-related questions. Unfortunately, the quality of information provided to them online is highly variable. Ensuring that patients get high-quality, reliable information is essential for all pathologies. Gastric cancer (GC), with its often subtle early symptoms and signs, is one such pathology where early identification is crucial. Ensuring high-quality information availability online for GC is thus essential to increasing rates of early detection. Aims This study aimed to assess the quality and readability of information posted on websites related to GC. Materials and methods We applied the search term "gastric cancer" or "stomach cancer" to the top three search engines, namely Google, Yahoo, and Bing. Using predefined inclusion and exclusion criteria, we identified 20 unique websites posting information related to gastric cancer (GC). We then assessed the quality and readability of the information posted on these websites. We used recognized tools to complete these assessments, including the JAMA benchmark criteria, the DISCERN tool, the Flesch Reading Ease score (FRES), and the Flesch-Kincaid Grade Level (FKGL). We also developed and used a novel GC-specific content assessment tool. Furthermore, we assessed whether or not each website was awarded the Health on the Internet Seal of Approval. Results The average JAMA score was 1.55, with none of the twenty unique websites scoring the maximum 4 points. The average DISCERN score was 54.8 (68.5%), with no website achieving the maximum of 80. The HON seal was present in only six websites (30%). The average GCSCS score was 11, with only five websites achieving a maximum score of 13 (25%). The average FRES and FKGL were 52.7 and 9.7, respectively. Conclusion Our study underscores the critical need for more high-quality, reliable information about GC online. We also emphasize the importance of ensuring this information is comprehensible to most patients, as it directly impacts their health outcomes.

Comparative Analysis of AI Chatbots ChatGPT, Gemini, and Copilot’s Answers to Common Cataract Questions

Purpose: To compare the readability and quality of the answers given by artificial intelligence (AI) chatbots to the twenty-five most frequent searches about cataracts on Google. Study Design: Cross sectional comparative study. Place and Duration of study: Serdivan State Hospital's Ophthalmology Department from March 2024 to April 2024. Methods: The word 'Cataract' was entered into Google Trends, and 25 trending searches made worldwide since 2004 were identified. These 25 trending searches were entered separately into AI programs. The answers were examined for quality using the Ensuring Quality Information for Patients (EQIP) test, while the readability was evaluated using the Flesch-Kincaid Reading Ease (FKRE) and Flesch-Kincaid Grade Level (FKGL). Results: The keywords searched most frequently were ‘Cataract surgery,’‘Eye cataract and 'After Cataract’. The EQIP category of all three AI chatbots was found to be 'serious quality issues.' When EQIP scores were compared, ChatGPT had a lower median value than Gemini and Copilot (p:0.001, p:0.007, respectively), while there was no significant difference between Gemini and Copilot (p:0.098). When FKRE values were compared, ChatGPT had a lower median value than Gemini and Copilot (p:0.001, p:0.001, respectively), with no significant difference between Gemini and Copilot (p:0.557). When FKGL values were compared, ChatGPT had a higher median value than Gemini and Copilot (p:0.003, p:0.001, respectively), with no significant difference between Gemini and Copilot(p:0.245). Conclusion: All three AI chatbots had an EQIP category of 'serious issues with quality .'The readability of all three was not at the recommended level.

Patient-friendliness of ChatGPT responses: NSCLC targeted therapy.

231 Background: Non-small cell lung cancer (NSCLC) targeted therapies are complex, often requiring extensive patient education beyond what can be conveyed in a single provider visit. Given the average US reading level is at the 8th grade, and many patients seek additional information online, it is vital to evaluate the readability of online resources. This study aims to determine the patient friendliness of information provided by AI chatbots like ChatGPT in explaining NSCLC targeted therapies compared to other commonly accessed sources. Methods: This analysis included targeted therapies approved by the FDA for EGFR-mutated NSCLC: Afatinib, Erlotinib, Osimertinib, Dacomitinib, and Gefitinib. Information sources included ChatGPT 4, ChatGPT 3.5, the Patient Information section of the FDA label, and the Google featured snippet. Each version of ChatGPT was queried 10 times for each individual therapy – “My doctor is suggesting starting therapy with (Drug Name). What can I expect with this medication?”. All responses, labels, and snippets were examined by Microsoft Word for word count (WC) and Flesch Reading Ease Score (FRES), which considers average sentence length and number of syllables per word to assess readability. Data analysis was performed using ANOVA in SAS. Results: Across all therapies, Mean FRES for ChatGPT 4 was 31.21, ChatGPT 3.5 26.69, FDA labels 50.78, and Google snippets 33.1. Mean WC for ChatGPT 4 was 312.54, ChatGPT 3.5 355.78, FDA labels 1220, and Google snippets 42. Significant differences in FRES were found among sources for Afatinib (p-value &lt;0.0001), Erlotinib (p-value 0.0206), Osimertinib (p-value &lt;0.0001), Dacomitinib (p-value 0.0125), and Gefitinib (p-value &lt;0.0001). Significant differences in WC were also found among sources for Afatinib (p-value &lt;0.0001), Erlotinib (p-value &lt;0.0001), Osimertinib (p-value &lt;0.0001), and Gefitinib (p-value 0.0045). No significant difference in WC was found for Dacomitinib (p-value 0.2807). Conclusions: While the FDA label provided the most readable information, none of the sources aligned with the average 8th-grade reading level in the US, indicating a gap in accessible patient education for NSCLC targeted therapies. Notably, readability varies significantly between ChatGPT versions, affecting accessibility for users of the free version, ChatGPT 3.5. Although ChatGPT responses are more succinct compared to the verbose FDA labels, their higher education-level readability presents a barrier to comprehension. This study highlights the need for further research into optimizing AI-generated patient education materials to be both accurate and comprehensible at appropriate reading levels, thereby improving health communication strategies.

Cultural adaptation and validation of the French version of the Antwerp pelvic floor knowledge questionnaire

AimsThe Antwerp Pelvic Floor Knowledge Questionnaire (APFK-Q) is a self-administered tool used for assessing women’s knowledge of the pelvic floor, an essential element of pelvic health education initiatives. The aim of this study was to validate the French version of the APFK-Q and to assess its psychometric properties. MethodsAcceptability of the cultural adaptation of APFK-Q in French (translation/back-translation) was assessed in two groups of women: one group not working in the healthcare sector (non-healthworkers, n = 112), and another of student midwives (n = 29). Twenty-three of the non-healthworkers agreed to take part in a test–retest step to assess reproducibility. Agreement of responses was measured for each question using Cohen’s kappa coefficient. Sensitivity to change was assessed before and after a pelvic-floor health education workshop. The proportion of correct answers was compared before and after the workshop using McNemar’s Chi2 test. Readability and the reading level required for optimal comprehension were assessed using the Flesch Reading Ease Score (FRES) and Flesch-Kincaid Grade Level test, respectively. ResultsSeventy percent (n = 72) of the non-healthworkers and 72 % (n = 21) of the students reported no comprehension difficulties. Agreement was very good (k > 0.8) for 46 % of the items and good (0.6–0.8) for 25 %. The proportion of correct answers was significantly higher for 72 % of the items after the pelvic-floor health education workshop. The French version of the APFK-Q was judged to be easy to understand according to the FRES test, with a score of 60.6. The school level required for optimal comprehension was estimated at 8th grade. ConclusionThe French version of the APFK-Q is an acceptable, reproducible, and easily understandable questionnaire that provides a broad description of women’s knowledge of the pelvic-floor sphere. Sensitivity to change was confirmed for most items, making it a useful tool for assessing the impact of pelvic-floor health education initiatives.