Flesch-Kincaid Research Articles

Abstract 4116844: Simplifying Cardiology Research Abstracts: Assessing ChatGPT's Readability and Comprehensibility for Non-Medical Audiences

Background: Artificial Intelligence (AI)-powered chatbots like ChatGPT are increasingly used in academic medical settings to help with tasks such as evidence synthesis and manuscript drafting. They have shown potential in simplifying complex medical texts for non-medical audiences like patients and journalists. However, less is known about whether simplified texts may exclude important information or be of interest to patients or other non-medically-trained people such as journalists. Objective: This study aims to assess ChatGPT's capacity to simplify cardiology research abstracts by eliminating jargon and enhancing universal comprehension. Methods: We analyzed all abstracts and scientific statements published from July to November 2023 in Circulation (n=113). These abstracts were processed through ChatGPT with the prompt: "Please rewrite the following text to be comprehensible at a 5th-grade reading level. Retain all original information and exclude nothing". We assessed the readability of both original and simplified texts using Flesch-Kincaid Grade Level (FKGL) and Reading Ease (FKRE) scores. Additionally, a panel of five physicians and five laypeople evaluated these texts for completeness, accuracy, and understandability. Results: ChatGPT transformation or abstracts reduced the required reading level from a college graduate to 8-9th grade by both FKGL (18.3 to 8.6; p<.001) and FKRE (14.6 to 68.5; p<.001). The physician panel generally found that the simplified abstracts retained all essential information without introducing errors. Both physicians and laypeople found the simplified texts more comprehensible. Laypeople appreciated the ease of understanding but did not strongly express a desire for more such simplified texts, and some still worried about the simplified abstracts missing important details. Conclusion: ChatGPT simplifies medical research abstracts to a level understandable by those with less than a high school education without excluding important information. This approach may be useful to communicate new research findings to patients and other non-medically-trained people.

ChatGPT for parents' education about early childhood caries: A friend or foe?

With the increasing popularity of online sources for health information, parents may seek information related to early childhood caries (ECC) from artificial intelligence-based chatbots. The aim of this article was to evaluate the usefulness, quality, reliability, and readability of ChatGPT answers to parents' questions about ECC. Eighty questions commonly asked about ECC were compiled from experts and keyword research tools. ChatGPT 3.5 was asked these questions independently. The answers were evaluated by experts in paediatric dentistry. ChatGPT provided "very useful" and "useful" responses to 82.5% of the questions. The mean global quality score was 4.3 ± 1 (good quality). The mean reliability score was 18.5 ± 8.9 (average to very good). The mean understandability score was 59.5% ± 13.8 (not highly understandable), and the mean actionability score was 40.5% ± 12.8 (low actionability). The mean Flesch-Kincaid reading ease score was 32% ± 25.7, and the mean Simple Measure of Gobbledygook index readability score was 15.3 ± 9.1(indicating poor readability for the lay person). Misleading and false information were detected in some answers. ChatGPT has significant potential as a tool for answering parent's questions about ECC. Concerns, however, do exist about the readability and actionability of the answers. The presence of false information should not be overlooked.

Evaluating ChatGPT Responses to Frequently Asked Questions Regarding Rhytidectomy

Importance: Patient education is vital during the assessment process. By informing patients, they are better equipped to make well-informed decisions, potentially leading to greater satisfaction after surgery. Objective: The study aimed to assess accuracy and readability, providing a robust evaluation of ChatGPT 3.5’s ability to address medical inquiries and improve the understanding of the facelift for interested patients. Method: On February 2, 2024, an investigation evaluated ChatGPT 3.5’s performance in answering facelift surgery queries. Four commonly asked questions were posed to the model. Responses were collected in an unguided manner and evaluated for accuracy via a literature review. The responses were evaluated for the accuracy with the senior author. Readability was assessed using the Flesch Reading Ease Score, Automated Readability Index, and Flesch-Kincaid Grade Level, calculated through an open-source platform. Result: The AI chatbot demonstrated its ability to generate comprehensive and accurate information, though some responses occasionally required clarification. The readability score and corresponding levels were consistently assessed at the college level. Conclusion: The study’s findings suggest that AI bots can serve as valuable tools for educating and preparing patients interested in getting a facelift.

Artificial intelligence as a modality to enhance the readability of neurosurgical literature for patients.

In this study the authors assessed the ability of Chat Generative Pretrained Transformer (ChatGPT) 3.5 and ChatGPT4 to generate readable and accurate summaries of published neurosurgical literature. Abstracts published in journal issues released between June 2023 and August 2023 (n = 150) were randomly selected from the top 5 ranked neurosurgical journals according to Google Scholar. ChatGPT models were instructed to generate a readable layperson summary of the original abstract from a statistically validated prompt. Readability results and grade-level indicators (RR-GLIs) scores were calculated for GPT3.5- and GPT4-generated summaries and original abstracts. Two physicians independently rated the accuracy of ChatGPT-generated layperson summaries to assess scientific validity. One-way ANOVA followed by pairwise t-test with Bonferroni correction were performed to compare readability scores. Cohen's kappa was used to assess interrater agreement between the two rater physicians. Analysis of 150 original abstracts showed a statistically significant difference for all RR-GLIs between the ChatGPT-generated summaries and original abstracts. The readability scores are formatted as follows (original abstract mean, GPT3.5 summary mean, GPT4 summary mean, p value): Flesch-Kincaid reading grade (12.55, 7.80, 7.70, p < 0.0001); Gunning fog score (15.46, 10.00, 9.00, p < 0.0001); Simple Measure of Gobbledygook (SMOG) index (11.30, 7.13, 6.60, p < 0.0001); Coleman-Liau index (14.67, 11.32, 10.26, p < 0.0001); automated readability index (10.87, 8.50, 7.75, p < 0.0001); and Flesch-Kincaid reading ease (33.29, 68.45, 69.55, p < 0.0001). GPT4-generated summaries demonstrated higher RR-GLIs than GPT3.5-generated summaries in the following categories: Gunning fog score (0.0003); SMOG index (0.027); Coleman-Liau index (< 0.0001); sentences (< 0.0001); complex words (< 0.0001); and % complex words (0.0035). A total of 68.4% and 84.2% of GPT3.5- and GPT4-generated summaries, respectively, maintained moderate scientific accuracy according to the two physician-reviewers. The findings demonstrate promising potential for application of the ChatGPT in patient education. GPT4 is an accessible tool that can be an immediate solution to enhancing the readability of current neurosurgical literature. Layperson summaries generated by GPT4 would be a valuable addition to a neurosurgical journal and would be likely to improve comprehension for patients using internet resources like PubMed.

Quality of Chatbot Information Related to Benign Prostatic Hyperplasia.

Large language model (LLM) chatbots, a form of artificial intelligence (AI) that excels at prompt-based interactions and mimics human conversation, have emerged as a tool for providing patients with information about urologic conditions. We aimed to examine the quality of information related to benign prostatic hyperplasia surgery from four chatbots and how they would respond to sample patient messages. We identified the top three queries in Google Trends related to "treatment for enlarged prostate." These were entered into ChatGPT (OpenAI), Bard (Google), Bing AI (Microsoft), and Doximity GPT (Doximity), both unprompted and prompted for specific criteria (optimized). The chatbot-provided answers to each query were evaluated for overall quality by three urologists using the DISCERN instrument. Readability was measured with the built-in Flesch-Kincaid reading level tool in Microsoft Word. To assess the ability of chatbots to answer patient questions, we prompted the chatbots with a clinical scenario related to holmium laser enucleation of the prostate, followed by 10 questions that the National Institutes of Health recommends patients ask before surgery. Accuracy and completeness of responses were graded with Likert scales. Without prompting, the quality of information was moderate across all chatbots but improved significantly with prompting (mean [SD], 3.3 [1.2] vs. 4.4 [0.7] out of 5; p < 0.001). When answering simulated patient messages, the chatbots were accurate (mean [SD], 5.6 [0.4] out of 6) and complete (mean [SD], 2.8 [0.3] out of 3). Additionally, 98% (39/40) had a median score of 5 or higher for accuracy, which corresponds to "nearly all correct." The readability was poor, with a mean (SD) Flesch-Kincaid reading level grade of 12.1 (1.3) (unprompted). LLM chatbots hold promise for patient education, but their effectiveness is limited by the need for careful prompting from the user and by responding at a reading level higher than that of most Americans (grade 8). Educating patients and physicians on optimal LLM interaction is crucial to unlock the full potential of chatbots.

Improving health literacy and stakeholder-directed knowledge of One Health through analysis of readability: A cross sectional infodemiology study

BackgroundThe One Health approach involves collaboration across several sectors, including public health, veterinary and environmental sectors in an integrated manner. These sectors may be disparate and unrelated, however to succeed, all stakeholders need to understand what the other stakeholders are communicating. Likewise, it is important that there is public acceptance and support of One Health approaches, which requires effective communication between professional and institutional organisations and the public. To help aid and facilitate such communication, written materials need to be readable by all stakeholders, in order to communicate effectively. There has been an exponential increase in the publication of papers involving One Health, with <5 per year, in the 2000s, to nearly 500 published in 2023. To date, readability of One Health information has not been scrutinised, nor has it been considered as an integral intervention of One Health policy communication. The aim of this study was therefore to examine readability of public-facing One Health information prepared by 24 global organisations. MethodsReadability was calculated using Readable software, to obtain four readability scores [(ⅰ) Flesch Reading Ease (FRE), (ⅱ) Flesch-Kinkaid Grade Level (FKGL), (ⅲ) Gunning Fog Index and (ⅳ) SMOG Index] and two text metrics [words/sentence, syllables/word] for 100 sources of One Health information, from four categories [One Health public information, PubMed abstracts; Science in One Health (SOH) abstracts (articles); SOH abstracts (reviews)]. ResultsReadability of One Health information for the public is poor, not reaching readability reference standards. No information was found that had a readability of less than 9th grade (around 14 years old). Mean values for the Flesch Reading Ease and Flesch Kincaid Grade Level were (19.4 ​± ​1.4) (target >60) and (15.6 ​± ​0.3) (target <8), respectively, with mean words per sentence and syllables per word of 20.5 and 2.0, respectively. Abstracts with “One Health” in the title were more difficult to read than those without “One Health” in the title (FRE: P ​= ​0.0337; FKGL: P ​= ​0.0087). Comparison of FRE and FKGL readability scores for the four categories of One Health information [One Health public information, PubMed abstracts; SOH abstracts (articles); SOH abstracts (reviews)] showed that SOH abstracts from articles were easier to read than those from SOH reviews. No One Health public-facing information from the 100 sources examined met the Flesch Kincaid Grade Level target of ≤8. The most easily read One Health information required a Grade Level of 9th grade (14–15 year old), with a mean Grade Level of 15.5 (university/college level). ConclusionConsiderable work is required in making One Health written materials more readable, particularly for children and adolescents (<14 years of age). It is important that any interventions or mitigations taken to support better public understanding of the One Health approach are not ephemeral, but have longer lasting and legacy value. Authors of One Health information should consider using readability calculators when preparing One Health information for their stakeholders, to check the readability of their work, so that the final material is within recommended readability reference parameters, to support the health literacy and stakeholder-directed knowledge of their readers.

Utilization of Artificial Intelligence in the Creation of Patient Information on Laryngology Topics.

To evaluate and compare the readability and quality of patient information generated by Chat-Generative Pre-Trained Transformer-3.5 (ChatGPT) and the American Academy of Otolaryngology-Head and Neck Surgery (AAO-HNS) using validated instruments including Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease, DISCERN, and Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P). ENTHealth.org and ChatGPT-3.5 were queried for patient information on laryngology topics. ChatGPT-3.5 was queried twice for a given topic to evaluate for reliability. This generated three de-identified text documents for each topic: one from AAO-HNS and two from ChatGPT (ChatGPT Output 1, ChatGPT Output 2). Grade level and reading ease were compared between the three sources using a one-way analysis of variance and Tukey's post hoc test. Independent t-tests were used to compare DISCERN and PEMAT understandability and actionability scores between AAO-HNS and ChatGPT Output 1. Material generated from ChatGPT Output 1 and ChatGPT Output 2 were at least two reading grade levels higher than that of material from AAO-HNS (p < 0.001). Regarding reading ease, ChatGPT Output 1 and ChatGPT Output 2 documents had significantly lower mean scores compared to AAO-HNS (p < 0.001). Moreover, ChatGPT Output 1 material on vocal cord paralysis had a lower PEMAT-P understandability compared to that of AAO-HNS material (p > 0.05). Patient information on the ENTHealth.org website for select laryngology topics was, on average, of a lower grade level and higher reading ease compared to that produced by ChatGPT, but interestingly with largely no difference in the quality of information provided. NA Laryngoscope, 2024.

Performance of Chatgpt in ophthalmology exam; human versus AI.

This cross-sectional study focuses on evaluating the success rate of ChatGPT in answering questions from the 'Resident Training Development Exam' and comparing these results with the performance of the ophthalmology residents. The 75 exam questions, across nine sections and three difficulty levels, were presented to ChatGPT. The responses and explanations were recorded. The readability and complexity of the explanations were analyzed and The Flesch Reading Ease (FRE) score (0-100) was recorded using the program named Readable. Residents were categorized into four groups based on their seniority. The overall and seniority-specific success rates of the residents were compared separately with ChatGPT. Out of 69 questions, ChatGPT answered 37 correctly (53.62%). The highest success was in Lens and Cataract (77.77%), and the lowest in Pediatric Ophthalmology and Strabismus (0.00%). Of 789 residents, overall accuracy was 50.37%. Seniority-specific accuracy rates were 43.49%, 51.30%, 54.91%, and 60.05% for 1st to 4th-year residents. ChatGPT ranked 292nd among residents. Difficulty-wise, 11 questions were easy, 44 moderate, and 14 difficult. ChatGPT's accuracy for each level was 63.63%, 54.54%, and 42.85%, respectively. The average FRE score of responses generated by ChatGPT was found to be 27.56 ± 12.40. ChatGPT correctly answered 53.6% of questions in an exam for residents. ChatGPT has a lower success rate on average than a 3rd year resident. The readability of responses provided by ChatGPT is low, and they are difficult to understand. As difficulty increases, ChatGPT's success decreases. Predictably, these results will change with more information loaded into ChatGPT.

Readability of online government information about welfare rights and benefits: the Israeli case

AbstractThis study examines the readability of government information about social benefits and rights concerning citizens’ engagement. Such information is often obscured and hard to find and, therefore, sometimes less accessible to those who mostly need it. How will readability characteristics affect engagement measures as attesting to information comprehension? This article addresses this question by analyzing year-long usage data records of an Israeli national rights website. We tested readability components to determine how they perform, and whether they affect engagement measures as evidence of comprehension. Analysis using the classic Flesch-Kincaid readability formula shows that most texts on the website are suitable for the college level. Furthermore, as the information was less readable, people spent more time on the page, and there was less bounce rate. Conversely, we found a positive correlation between exit percentage and readability measures. The results raise questions about the Flesch-Kincaid formula and the engagement measures’ relevance to information about benefits and rights, and highlight the need for a Hebrew-adapted readability formula. Further research directions are suggested.

Accuracy of Prospective Assessments of 4 Large Language Model Chatbot Responses to Patient Questions About Emergency Care: Experimental Comparative Study.

Recent surveys indicate that 48% of consumers actively use generative artificial intelligence (AI) for health-related inquiries. Despite widespread adoption and the potential to improve health care access, scant research examines the performance of AI chatbot responses regarding emergency care advice. We assessed the quality of AI chatbot responses to common emergency care questions. We sought to determine qualitative differences in responses from 4 free-access AI chatbots, for 10 different serious and benign emergency conditions. We created 10 emergency care questions that we fed into the free-access versions of ChatGPT 3.5 (OpenAI), Google Bard, Bing AI Chat (Microsoft), and Claude AI (Anthropic) on November 26, 2023. Each response was graded by 5 board-certified emergency medicine (EM) faculty for 8 domains of percentage accuracy, presence of dangerous information, factual accuracy, clarity, completeness, understandability, source reliability, and source relevancy. We determined the correct, complete response to the 10 questions from reputable and scholarly emergency medical references. These were compiled by an EM resident physician. For the readability of the chatbot responses, we used the Flesch-Kincaid Grade Level of each response from readability statistics embedded in Microsoft Word. Differences between chatbots were determined by the chi-square test. Each of the 4 chatbots' responses to the 10 clinical questions were scored across 8 domains by 5 EM faculty, for 400 assessments for each chatbot. Together, the 4 chatbots had the best performance in clarity and understandability (both 85%), intermediate performance in accuracy and completeness (both 50%), and poor performance (10%) for source relevance and reliability (mostly unreported). Chatbots contained dangerous information in 5% to 35% of responses, with no statistical difference between chatbots on this metric (P=.24). ChatGPT, Google Bard, and Claud AI had similar performances across 6 out of 8 domains. Only Bing AI performed better with more identified or relevant sources (40%; the others had 0%-10%). Flesch-Kincaid Reading level was 7.7-8.9 grade for all chatbots, except ChatGPT at 10.8, which were all too advanced for average emergency patients. Responses included both dangerous (eg, starting cardiopulmonary resuscitation with no pulse check) and generally inappropriate advice (eg, loosening the collar to improve breathing without evidence of airway compromise). AI chatbots, though ubiquitous, have significant deficiencies in EM patient advice, despite relatively consistent performance. Information for when to seek urgent or emergent care is frequently incomplete and inaccurate, and patients may be unaware of misinformation. Sources are not generally provided. Patients who use AI to guide health care decisions assume potential risks. AI chatbots for health should be subject to further research, refinement, and regulation. We strongly recommend proper medical consultation to prevent potential adverse outcomes.

Comparing Provider and ChatGPT Responses to Breast Reconstruction Patient Questions in the Electronic Health Record.

Patient-directed Electronic Health Record (EHR) messaging is used as an adjunct to enhance patient-physician interactions but further burdens the physician. There is a need for clear electronic patient communication in all aspects of medicine, including plastic surgery. We can potentially utilize innovative communication tools like ChatGPT. This study assesses ChatGPT's effectiveness in answering breast reconstruction queries, comparing its accuracy, empathy, and readability with healthcare providers' responses. Ten deidentified questions regarding breast reconstruction were extracted from electronic messages. They were presented to ChatGPT3, ChatGPT4, plastic surgeons, and advanced practice providers for response. ChatGPT3 and ChatGPT4 were also prompted to give brief responses. Using 1-5 Likert scoring, accuracy and empathy were graded by 2 plastic surgeons and medical students, respectively. Readability was measured using Flesch Reading Ease. Grades were compared using 2-tailed t tests. Combined provider responses had better Flesch Reading Ease scores compared to all combined chatbot responses (53.3 ± 13.3 vs 36.0 ± 11.6, P < 0.001) and combined brief chatbot responses (53.3 ± 13.3 vs 34.7 ± 12.8, P < 0.001). Empathy scores were higher in all combined chatbot than in those from combined providers (2.9 ± 0.8 vs 2.0 ± 0.9, P < 0.001). There were no statistically significant differences in accuracy between combined providers and all combined chatbot responses (4.3 ± 0.9 vs 4.5 ± 0.6, P = 0.170) or combined brief chatbot responses (4.3 ± 0.9 vs 4.6 ± 0.6, P = 0.128). Amid the time constraints and complexities of plastic surgery decision making, our study underscores ChatGPT's potential to enhance patient communication. ChatGPT excels in empathy and accuracy, yet its readability presents limitations that should be addressed.

Text readability: its impact on reading comprehension and reading time

Recently, the readability of texts has become the focus of reading research because it is believed to have implications for reading comprehension, which is of utmost importance in the field of English as a foreign language (EFL), particularly in the teaching, learning and assessment of reading comprehension. Unfortunately, the influence of text readability on reading comprehension (and reading time) has not been well studied in the EFL context. Most text readability studies are conducted in medical contexts, but these studies are often limited in predicting readability scores for sample texts. To address this gap, the current study aimed to evaluate the influence of text readability levels (based on the Flesch-Kincaid grade level (FKGL)) on students’ reading comprehension and reading time. Data were collected through reading test and analyzed using SPSS version 22. The Friedman test revealed that the distribution of students' reading comprehension score (X2=197.532, p=0.000) and reading time (X2=215.323, p=0.000) are different in each text, suggesting that the readability of texts has a significant influence on both. This study contributed to the practices of reading instruction and assessment. Limitations and suggestions for further research are briefly discussed.

Enhancing patient understanding in obstetrics: The role of generative AI in simplifying informed consent for labor induction with oxytocin.

Informed consent is a cornerstone of ethical medical practice, particularly in obstetrics where procedures like labor induction carry significant risks and require clear patient understanding. Despite legal mandates for patient materials to be accessible, many consent forms remain too complex, resulting in patient confusion and dissatisfaction. This study explores the use of Generative Artificial Intelligence (GAI) to simplify informed consent for labor induction with oxytocin, ensuring content is both medically accurate and comprehensible at an 8th-grade readability level. GAI-generated consent forms streamline the process, automatically tailoring content to meet readability standards while retaining essential details such as the procedure's nature, risks, benefits, and alternatives. Through iterative prompts and expert refinement, the AI produces clear, patient-friendly language that bridges the gap between medical jargon and patient comprehension. Flesch Reading Ease scores show improved readability, meeting recommended levels for health literacy. GAI has the potential to revolutionize healthcare communication by enhancing patient understanding, promoting shared decision-making, and improving satisfaction with the consent process. However, human oversight remains critical to ensure that AI-generated content adheres to legal and ethical standards. This case study demonstrates that GAI can be an effective tool in creating accessible, standardized, yet personalized consent documents, contributing to better-informed patients and potentially reducing malpractice claims.

Exploring the role of AI-driven chatbots in patient care: a critical evaluation amidst healthcare staff shortages

Abstract Background Amidst the backdrop of staff shortages in healthcare systems, patients and their families are increasingly turning to chatbots, powered by Large Language Models (LLMs), for information about their medical conditions. These AI-driven chatbots, capable of generating human-like responses across a broad range of topics, have become a prevalent tool in the healthcare landscape. Given the proliferation of these chatbots, it is crucial to evaluate the quality and accuracy of the responses they provide. Methods We selected five freely accessible chatbots (Bard, Microsoft Copilot, PiAI, ChatGPT, and ChatSpot) for our study. These chatbots were posed questions spanning three medical fields: cardiology, cardio-oncology, and cardio-rheumatology. The responses generated by the chatbots were then compared against established guidelines from the European Society of Cardiology, American Academy of Dermatology, and American Society of Clinical Oncology. In addition to the content, the readability of the responses was evaluated using four different readability scales: the Flesch Reading Scale, Gunning Fog Scale Level, Flesch-Kincaid Grade Level, and Dale-Chall Score. To assess the accuracy of the responses in accordance with the medical guidelines, two independent medical professionals rated them on a 3-point Likert scale (0 - incorrect, 1 - partially correct or incomplete, 2 - correct). This allowed us to gauge the compliance of the chatbot responses with the medical guidelines. Results Results: In our study, we posed a total of 45 questions to each of the chatbots. Out of the five chatbots, Microsoft Copilot, PiAI, and ChatGPT were able to respond to all the questions. The length of the responses varied, with PiAI providing the shortest average response length of 7.26 words, and Bard providing the longest at 18.9 words. In terms of readability, the Flesch Reading Ease Scale scores ranged from 17.67 (ChatGPT) to 39.34 (Bard), indicating the relative complexity of the responses. The Flesch-Kincaid Grade Level, which reflects the academic grade level required to comprehend the text, ranged from 14.02 (PiAI) to 15.97 (ChatGPT). The Gunning Fog Scale Level, another measure of readability, varied from 15.77 (Bard) to 19.73 (ChatGPT). Lastly, the Dale-Chall Score, which assesses the understandability of the text, ranged from 10.24 (Bard) to 11.87 (ChatGPT). These results highlight the variability in the readability and complexity of responses generated by different chatbots. Readability analysis is presented in table 1. Conclusion This study indicates that chatbots vary in length, quality and readability. They answer each question in their way, based on data they have pulled from the network. Our data suggests that people who want information from a chatbot need to be careful and verify the answers they get.

Assessing the Responses of Large Language Models (ChatGPT-4, Claude 3, Gemini, and Microsoft Copilot) to Frequently Asked Questions in Retinopathy of Prematurity: A Study on Readability and Appropriateness.

To assess the appropriateness and readability of responses provided by four large language models (LLMs) (ChatGPT-4, Claude 3, Gemini, and Microsoft Co-pilot) to parents' queries pertaining to retinopathy of prematurity (ROP). A total of 60 frequently asked questions were collated and categorized into six distinct sections. The responses generated by the LLMs were evaluated by three experienced ROP specialists to determine their appropriateness and comprehensiveness. Additionally, the readability of the responses was assessed using a range of metrics, including the Flesch-Kincaid Grade Level (FKGL), Gunning Fog (GF) Index, Coleman-Liau (CL) Index, Simple Measure of Gobbledygook (SMOG) Index, and Flesch Reading Ease (FRE) score. ChatGPT-4 demonstrated the highest level of appropriateness (100%) and performed exceptionally well in the Likert analysis, scoring 5 points on 96% of questions. The CL Index and FRE scores identified Gemini as the most readable LLM, whereas the GF Index and SMOG Index rated Microsoft Copilot as the most readable. Nevertheless, ChatGPT-4 exhibited the most intricate text structure, with scores of 18.56 on the GF Index, 18.56 on the CL Index, 17.2 on the SMOG Index, and 9.45 on the FRE score. This suggests that the responses demand a college-level comprehension. ChatGPT-4 demonstrated higher performance than other LLMs in responding to questions related to ROP; however, its texts were more complex. In terms of readability, Gemini and Microsoft Copilot were found to be more successful. [J Pediatr Ophthalmol Strabismus. 20XX;XX(X):XXX-XXX.].

Validity and reliability of a physical literacy knowledge, attitudes, self-efficacy and behaviors questionnaire for early childhood educators (PLKASB-ECE): An exploratory factor analysis.

Nurturing physical literacy in young children offers a unique opportunity to address global physical inactivity trend. Early childhood education and care (ECEC) environments, with their extensive reach into this age group, and early childhood educators, through their daily interactions with children, are strategically positioned to influence children's physical literacy development. However, enhancing educators' ability to foster physical literacy requires valid and reliable assessment tools to measure holistic physical literacy constructs (i.e., cognitive, affective, behavioral) to assess the impact of physical literacy educational interventions. The purpose of this study was to develop a holistic digital tool to measure physical literacy knowledge, attitudes, self-efficacy, and behaviors of early childhood educators (PLKASB-ECE) for both their professional teaching context as well as the educator's own personal physical literacy behaviours. This study was conducted in two phases. Phase 1 involved item generation and content validation. Phase 2 employed a cross-sectional validation study design to assess the psychometric properties of the PLKASB-ECE tool. Based on a literature review an initial 19-item instrument was developed that subsequently underwent three rounds of expert content validation. Six additional items were added, resulting in a final 25-item self-report measure with a readability score equivalent to an 8th grade reading level (Flesch-Kincaid Grade Level: 8.7). This included 1 global rating item, 2 qualitative response items, 7 knowledge items, and 15 items addressing physical literacy attitudes, self-efficacy and behaviors using a 7-point Likert response scale. The PLKASB-ECE tool was administered to 470 educators in Alberta, Canada between 2019 to 2022. The 15 items were subjected to exploratory factor analysis and resulted in a five-factor scale with one item not loading. The five-factor scale held with the final 14 items with loadings ranging from 0.481 to 0.886, Cronbach's alpha ranging from 0.70 to 0.82, with ordinal omega ranging from 0.72 to 0.82. Usability, as assessed by completion time, was 8.15 minutes. These findings demonstrate good indices of reliability and validity for the PLKASB-ECE tool. This tool will be valuable as a research outcome measure for assessing educational interventions aimed at enhancing educators' understanding, confidence, and strategies for fostering holistic concepts of physical literacy in young children within ECECs.

Evaluating the reliability of the responses of large language models to keratoconus-related questions

ABSTRACT Clinical relevance Artificial intelligence has undergone a rapid evolution and large language models (LLMs) have become promising tools for healthcare, with the ability of providing human-like responses to questions. The capabilities of these tools in addressing questions related to keratoconus (KCN) have not been previously explored. Background In this study, the responses were evaluated from three LLMs – ChatGPT-4, Copilot, and Gemini – to common patient questions regarding KCN. Methods Fifty real-life patient inquiries regarding general information, aetiology, symptoms and diagnosis, progression, and treatment of KCN were presented to the LLMs. Evaluations of the answers were conducted by three ophthalmologists with a 5-point Likert scale ranging from ‘strongly disagreed’ to ‘strongly agreed’. The reliability of the responses provided by LLMs was evaluated using the DISCERN and the Ensuring Quality Information for Patients (EQIP) scales. Readability metrics (Flesch Reading Ease Score, Flesch-Kincaid Grade Level, and Coleman-Liau Index) were calculated to evaluate the complexity of responses. Results ChatGPT-4 consistently scored 3 points or higher for all (100%) its responses, while Copilot had five (10%) and Gemini had two (4%) responses scoring 2 points or below. ChatGPT-4 achieved a ‘strongly agree’ rate of 74% across all questions, markedly superior to Copilot at 34% and Gemini at 42% (p < 0.001); and recorded the highest ‘strongly agree’ rates in general information and symptoms & diagnosis categories (90% for both). The median Likert scores differed among LLMs (p < 0.001), with ChatGPT-4 scoring highest and Copilot scoring lowest. Although ChatGPT-4 exhibited more reliability based on the DISCERN scale, it was characterised by lower readability and higher complexity. While all LLMs provided responses categorised as ‘extremely difficult to read’, the responses provided by Copilot showed higher readability. Conclusions Despite the responses provided by ChatGPT-4 exhibiting lower readability and greater complexity, it emerged as the most proficient in answering KCN-related questions.

Improving Biomedical Science Literacy and Patient-Directed Knowledge of Tuberculosis (TB): A Cross-Sectional Infodemiology Study Examining Readability of Patient-Facing TB Information.

Tuberculosis (TB) continues be the leading cause of death globally due to an infectious agent. There is a paucity of data describing the readability of patient-facing TB information for service users. The aim of this study was to calculate the readability of multiple global TB information sources. Information on tuberculosis (n = 150 sources) included nine categories, Patient-facing information: WHO publications (n = 17), International governments (n = 19), Hospitals (n = 10), Non-government organisations (NGOs)/charities (n = 20), Cochrane Plain Language Summaries (n = 20); LabTestsOnlineUK (n = 4) and Scientific-facing information: Clinical trials (n = 20), Cochrane abstracts (n = 20), Scientific abstracts (n = 20). Readability was calculated using Readable software, defined by (i) Flesch Reading Ease (FRE), (ii) Flesch-Kincaid Grade Level (FKGL), (iii) Gunning Fog Index and (iv) SMOG Index and two text metrics [words/sentence, syllables/word]. Mean readability values for TB information for the FRE and FKGL were 35.6 ± 1.6 (standard error of mean (SEM)) (US Target ≥60; UK Target ≥90) and 12.3 ± 0.3 (US Target ≤8; UK Target ≤6), respectively, with mean words per sentence and syllables per word of 17.2 and 1.8, respectively. Cochrane Plain Language Summaries had similar readability scores to their matching scientific abstract (p = 0.15). LabTestsOnlineUK yielded a mean FRE score of 51.5 ± 1.2, a mean FKGL score of 10.2 ± 0.5 and text metric scores of 16.7 ± 2.3 and 1.6, for words per sentence and syllables per word, respectively. In descending order, TB information from international governments, hospitals and LabTestsOnlineUK were the most readable (FRE = 57.9, 54.1 and 51.5, respectively), whereas scientific abstracts and Cochrane abstracts were the most difficult to read (13.0 and 30.2, respectively). Patient-facing TB information analysed had poor readability. Effective communication of biomedical science concepts and information relating to TB is vital for service users to enhance their health literacy of tuberculosis, thereby promoting better clinical outcomes. Biomedical scientists are important custodians of scientific information for their service user populations, including other healthcare professionals within the TB multidisciplinary (MDT) team and patient service users. When preparing TB information, this should be checked and modified in real time employing readability calculators, to align with health readability targets.

Assessing the readability and quality of online written information on epistaxis.

The objective of this study was to assess the readability and quality of online written information on epistaxis. The terms 'epistaxis' and 'nosebleed' were entered into Google. The first six webpages generated for each search term were screened. Readability was assessed using the Flesch-Kincaid Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG) Index and Gunning Fog Index (GFOG). Quality was assessed using the DISCERN instrument. Spearman's correlation between quality and readability was calculated. A total of 37 websites met the inclusion criteria. The mean and 95% confidence intervals for FRES, FKGL, SMOG and GFOG were 58.9 (55.3-62.5), 9.65 (8.74-10.6), 9.18 (8.57-9.8) and 12.5 (11.5-13.5), respectively. The DISCERN score was 34.3 (32.0-36.5). Weak negative correlation was noted between DISCERN and FRES (rs = -0.15, p = 0.36). Online information on epistaxis is generally of poor quality and low readability.

Utilization of Artificial Intelligence to Improve Equitable Healthcare Access for Breast Implant Patients

Abstract The recently mandated US Food and Drug Administration (FDA) patient decision checklists were developed with the goal of improving the informed decision-making process for patients considering breast implants. Yet, prior research has demonstrated that these checklists are written at reading levels far in excess of that recommended by the National Institutes of Health and the American Medical Association. Patients considering breast implants are required to sign these forms prior to surgery, even if they have little to no comprehension of their contents. This study aims to improve the accessibility, and therefore the utility, of the mandated FDA patient literature using the assistance of artificial intelligence (AI). Patient decision checklists were obtained from the three most utilized breast implant manufacturers in the United States– Allergan, Mentor, and Sientra. AI was then asked to synthesize a novel patient decision checklist, written at the 6th grade reading level, using these checklists as source material. The AI-assisted checklist was edited by plastic surgeons for both formatting and content. Readability analysis of all documents was performed using the Flesch Reading-Ease Score, Flesch-Kincaid Grade Level, Gunning-Fog Index, Coleman-Liau Index, Simplified Measure of Gobbledygook, and Automated Readability Index. Textual analysis included sentence count, word count, number and percentage of complex words, average number of words per sentence, and average number of syllables per word. The overall readability of Allergan, Mentor, and Sientra patient checklists correlated with the college reading level. These documents were of a statistically significantly higher reading level the AI-assisted checklist, which was written at the recommended 6th grade level. Text composition analysis similarly demonstrated substantial differences between the AI-assisted and FDA mandated literature. The currently mandated breast implant patient checklists are written at a college reading level and are inaccessible to the average patient. We propose a new patient decision checklist, generated with the assistance of artificial intelligence, in order to improve healthcare access within plastic surgery. This simplified material can be used as an adjunct to the current checklists in order to improve shared decision making.