Readability Index Research Articles

Evaluating ChatGPT Responses to Frequently Asked Questions Regarding Rhytidectomy

Importance: Patient education is vital during the assessment process. By informing patients, they are better equipped to make well-informed decisions, potentially leading to greater satisfaction after surgery. Objective: The study aimed to assess accuracy and readability, providing a robust evaluation of ChatGPT 3.5’s ability to address medical inquiries and improve the understanding of the facelift for interested patients. Method: On February 2, 2024, an investigation evaluated ChatGPT 3.5’s performance in answering facelift surgery queries. Four commonly asked questions were posed to the model. Responses were collected in an unguided manner and evaluated for accuracy via a literature review. The responses were evaluated for the accuracy with the senior author. Readability was assessed using the Flesch Reading Ease Score, Automated Readability Index, and Flesch-Kincaid Grade Level, calculated through an open-source platform. Result: The AI chatbot demonstrated its ability to generate comprehensive and accurate information, though some responses occasionally required clarification. The readability score and corresponding levels were consistently assessed at the college level. Conclusion: The study’s findings suggest that AI bots can serve as valuable tools for educating and preparing patients interested in getting a facelift.

Artificial intelligence as a modality to enhance the readability of neurosurgical literature for patients.

In this study the authors assessed the ability of Chat Generative Pretrained Transformer (ChatGPT) 3.5 and ChatGPT4 to generate readable and accurate summaries of published neurosurgical literature. Abstracts published in journal issues released between June 2023 and August 2023 (n = 150) were randomly selected from the top 5 ranked neurosurgical journals according to Google Scholar. ChatGPT models were instructed to generate a readable layperson summary of the original abstract from a statistically validated prompt. Readability results and grade-level indicators (RR-GLIs) scores were calculated for GPT3.5- and GPT4-generated summaries and original abstracts. Two physicians independently rated the accuracy of ChatGPT-generated layperson summaries to assess scientific validity. One-way ANOVA followed by pairwise t-test with Bonferroni correction were performed to compare readability scores. Cohen's kappa was used to assess interrater agreement between the two rater physicians. Analysis of 150 original abstracts showed a statistically significant difference for all RR-GLIs between the ChatGPT-generated summaries and original abstracts. The readability scores are formatted as follows (original abstract mean, GPT3.5 summary mean, GPT4 summary mean, p value): Flesch-Kincaid reading grade (12.55, 7.80, 7.70, p < 0.0001); Gunning fog score (15.46, 10.00, 9.00, p < 0.0001); Simple Measure of Gobbledygook (SMOG) index (11.30, 7.13, 6.60, p < 0.0001); Coleman-Liau index (14.67, 11.32, 10.26, p < 0.0001); automated readability index (10.87, 8.50, 7.75, p < 0.0001); and Flesch-Kincaid reading ease (33.29, 68.45, 69.55, p < 0.0001). GPT4-generated summaries demonstrated higher RR-GLIs than GPT3.5-generated summaries in the following categories: Gunning fog score (0.0003); SMOG index (0.027); Coleman-Liau index (< 0.0001); sentences (< 0.0001); complex words (< 0.0001); and % complex words (0.0035). A total of 68.4% and 84.2% of GPT3.5- and GPT4-generated summaries, respectively, maintained moderate scientific accuracy according to the two physician-reviewers. The findings demonstrate promising potential for application of the ChatGPT in patient education. GPT4 is an accessible tool that can be an immediate solution to enhancing the readability of current neurosurgical literature. Layperson summaries generated by GPT4 would be a valuable addition to a neurosurgical journal and would be likely to improve comprehension for patients using internet resources like PubMed.

Utilization of Artificial Intelligence to Improve Equitable Healthcare Access for Breast Implant Patients

Abstract The recently mandated US Food and Drug Administration (FDA) patient decision checklists were developed with the goal of improving the informed decision-making process for patients considering breast implants. Yet, prior research has demonstrated that these checklists are written at reading levels far in excess of that recommended by the National Institutes of Health and the American Medical Association. Patients considering breast implants are required to sign these forms prior to surgery, even if they have little to no comprehension of their contents. This study aims to improve the accessibility, and therefore the utility, of the mandated FDA patient literature using the assistance of artificial intelligence (AI). Patient decision checklists were obtained from the three most utilized breast implant manufacturers in the United States– Allergan, Mentor, and Sientra. AI was then asked to synthesize a novel patient decision checklist, written at the 6th grade reading level, using these checklists as source material. The AI-assisted checklist was edited by plastic surgeons for both formatting and content. Readability analysis of all documents was performed using the Flesch Reading-Ease Score, Flesch-Kincaid Grade Level, Gunning-Fog Index, Coleman-Liau Index, Simplified Measure of Gobbledygook, and Automated Readability Index. Textual analysis included sentence count, word count, number and percentage of complex words, average number of words per sentence, and average number of syllables per word. The overall readability of Allergan, Mentor, and Sientra patient checklists correlated with the college reading level. These documents were of a statistically significantly higher reading level the AI-assisted checklist, which was written at the recommended 6th grade level. Text composition analysis similarly demonstrated substantial differences between the AI-assisted and FDA mandated literature. The currently mandated breast implant patient checklists are written at a college reading level and are inaccessible to the average patient. We propose a new patient decision checklist, generated with the assistance of artificial intelligence, in order to improve healthcare access within plastic surgery. This simplified material can be used as an adjunct to the current checklists in order to improve shared decision making.

How Readable Is the Information the United Kingdom's Statutory Health and Social Care Professional Regulators Provide for the Public to Engage With Fitness to Practise Processes?

The public are an important source of notifications and evidence for the investigation of concerns by regulators of professionals. The website is an important source of information for the public, but the complexity of information presented to engage with the public is unknown. This study explored the readability of information provided for the public to engage with fitness to practise processes by examining the websites of the 13 UK statutory health and social care professional regulators. Six readability algorithms were utilised to calculate the readability scores of 180 general and 8 easy-read documents published for the 15 sites of the United Kingdom's 13 health and social care statutory professional regulatory bodies. These tests were the Flesch Kincaid Reading Ease, the Flesch Kincaid Grade Level, the Gunning Fog Score, the Simple Measure of Gobbledygook (SMOG) Index, the Coleman Liau Index and the Automated Readability Index (ARI). All the fitness to practise documents analysed in this study are written at a level too difficult for most of the general population to read, except one easy-read document. There was also considerable variation in readability across resources for the same regulator, which could be confusing. Regulatory bodies risk excluding a large proportion of UK adults who may want to engage with professional regulatory proceedings. This is the first comparative analysis of readability conducted independent of the regulators of the fitness to practise website documents of health and social care regulators. The public are a key source of evidence in regulatory proceedings. Regulators could improve public engagement by addressing the complexity of language used. Our advisory group of people with lived experience of involvement as members of the public in fitness to practise proceedings discussed the findings and contributed to the recommendations.

Evaluating the readability of online patient information for cholangiocarcinoma.

264 Background: The digital age has facilitated the dissemination of free comprehensive information regarding the diagnosis and treatment of various cancers. Especially for rarer cancers, such as cholangiocarcinoma, patients heavily rely on online patient information (OPI) as a primary source of information. However, the usefulness of these websites is dependent on the readability of the information. The National Institute of Health (NIH) recommends OPI have a readability level of grade 8 or lower. We sought to assess the readability of OPI for cholangiocarcinoma. Methods: We utilized Google to identify the top ten websites utilizing the search query "cholangiocarcinoma". The readability score of each website was defined as the median of ten popular standardized readability scales - Flesch–Kincaid, Gunning Fog, Coleman–Liau, Simple Measure of Gobbledygook, Automated Readability Index, FORCAST, Raygor, Fry Graph, and New Dale Chall. These readability scales were obtained using www.readable.com. Group comparisons of readability score were performed for affiliation (i.e., academic, non-profit, or government) and website content (i.e., treatment modalities, complications, clinical trials, supportive care, and survivorship). Statistical analysis was done in RStudio. Pairwise comparisons were performed using the Wilcoxon rank sum test. Results: A total of 10 websites were analyzed with 4/10 (40%) affiliated with academic institutions, 3/10 (30%) affiliated with non-profit organization, and 3/10 (30%) affiliated with government websites. The median readability score for all 10 websites was 9.95 (IQR 9.15-10.75). The majority of the websites (9/10, 90%) websites had readability scores that did not satisfy the NIH recommendation for readability. Readability scores did not differ significantly between academic, government, and nonprofit websites (p=0.68). Readability also did not differ by website content. The correlation between word count and readability score was not statistically significant (r = 0.03, p = 0.92). Conclusions: We performed a first of its kind analysis to assess the readability of OPI for cholangiocarcinoma. The median readability of OPI for cholangiocarcinoma was at a 10th grade level and the majority of the websites did not satisfy the NIH guideline for readability. Surprisingly, government websites were not simpler to read than their academic and nonprofit counterparts. Our study underscores the need to develop OPI for cholangiocarcinoma that is easily readable for patients and their caregivers. Organizations publishing OPI may use readability scores to ensure their content is accessible for the general population.

Evaluation of Rhinoplasty Information from ChatGPT, Gemini, and Claude for Readability and Accuracy.

Assessment of the readability, accuracy, quality, and completeness of ChatGPT (Open AI, San Francisco, CA), Gemini (Google, Mountain View, CA), and Claude (Anthropic, San Francisco, CA) responses to common questions about rhinoplasty. Ten questions commonly encountered in the senior author's (SPM) rhinoplasty practice were presented to ChatGPT-4, Gemini and Claude. Seven Facial Plastic and Reconstructive Surgeons with experience in rhinoplasty were asked to evaluate these responses for accuracy, quality, completeness, relevance, and use of medical jargon on a Likert scale. The responses were also evaluated using several readability indices. ChatGPT achieved significantly higher evaluator scores for accuracy, and overall quality but scored significantly lower on completeness compared to Gemini and Claude. All three chatbot responses to the ten questions were rated as neutral to incomplete. All three chatbots were found to use medical jargon and scored at a college reading level for readability scores. Rhinoplasty surgeons should be aware that the medical information found on chatbot platforms is incomplete and still needs to be scrutinized for accuracy. However, the technology does have potential for use in healthcare education by training it on evidence-based recommendations and improving readability. This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266 .

Comparing the Ability of Google and ChatGPT to Accurately Respond to Oculoplastics-Related Patient Questions and Generate Customized Oculoplastics Patient Education Materials.

To compare the accuracy and readability of responses to oculoplastics patient questions provided by Google and ChatGPT. Additionally, to assess the ability of ChatGPT to create customized patient education materials. We executed a Google search to identify the 3 most frequently asked patient questions (FAQs) related to 10 oculoplastics conditions. FAQs were entered into both the Google search engine and the ChatGPT tool and responses were recorded. Responses were graded for readability using five validated readability indices and for accuracy by six oculoplastics surgeons. ChatGPT was instructed to create patient education materials at various reading levels for 8 oculoplastics procedures. The accuracy and readability of ChatGPT-generated procedural explanations were assessed. ChatGPT responses to patient FAQs were written at a significantly higher average grade level than Google responses (grade 15.6 vs 10.0, p < 0.001). ChatGPT responses (93% accuracy) were significantly more accurate (p < 0.001) than Google responses (78% accuracy) and were preferred by expert panelists (79%). ChatGPT accurately explained oculoplastics procedures at an above average reading level. When instructed to rewrite patient education materials at a lower reading level, grade level was reduced by approximately 4 (15.7 vs 11.7, respectively, p < 0.001) without sacrificing accuracy. ChatGPT has the potential to provide patients with accurate information regarding their oculoplastics conditions. ChatGPT may also be utilized by oculoplastic surgeons as an accurate tool to provide customizable patient education for patients with varying health literacy. A better understanding of oculoplastics conditions and procedures amongst patients can lead to informed eye care decisions.

Quality and Readability Analysis of Information on Cleft Lip and Palate: Sample of Online Resources in Turkish

Aim: Cleft lip and palate is one of the most common congenital craniofacial conditions with follow-up of many different professionals usually beginning at birth and continuing throughout life. Internet resources can influence people's decisions on health-related issues. The purpose of this study was to examine the information quality, verifiability, standard compliance, and readability of websites that offer information on cleft lip and palate. Method: The first 100 websites accessed through "http://www.google.com.tr/" using the keyword "cleft lip and palate (“dudak damak yarıkları” in Turkish)" were evaluated and 64 websites were included in the analysis. The information quality, conformity to standards, and verifiability of the websites were scored according to HONcode criteria (0=lowest; 16=highest); Ateşman (0–29=very difficult; 90–100=very easy) and Çetinkaya (0–34=disabled level; 51+=independent level) readability indices were applied. Results: The mean HONcode score of the websites was 4.79±2.07 (min=0; max=9). The average word length was 2.77±0.12 (min=2.44; max=3.13) and the average sentence length was 11.46±2.44 (min=5.4; max=19.1). The mean of Ateşman readability analysis was 57.57±8.92 (min=32.9; max=77.4) and the mean of Çetinkaya readability analysis was 57.93±3.27 (min=49.72; max=64.64). Conclusion: The results indicate that the web pages have limited standards compliance, verifiability, and information quality. The Ateşman index places readability at a medium difficulty level, while the Çetinkaya index places it at an independent level. It is emphasized that content producers should pay attention to quality standards while developing the reading material on websites related to cleft lip and palate and pay attention to readability features so that visitors can easily access information.

Using constructions to measure developmental language complexity

Abstract Models used to explain phenomena are necessarily finer grained than the models used to measure them. In language study, the measures used to assess development (e.g., readability indices) rely on models of language that are too coarse grained to be interpreted in a linguistic framework and so do not participate in linguistic accounts of development. This study argues that the constructionist approaches provide a framework for the development of a practical and interpretable measure of developmental complexity because these approaches feature affordances from which a measurement model may be derived: they describe language knowledge as a comprehensive network of enumerable entities that do not require the imputation of external processes, are extensible to early child language, and hold that the drivers of language development are the learning and generalization of constructions. It is argued here that treating schematic constructions as the unit of language knowledge supports a complexity measure that can reflect developmental changes arising from the learning and productive generalization of these units.

In the Belly of Text Complexity: Unravelling the Nexus between Lexical Density and Readability

This study set out to explore the link between lexical density (LD) as a tool for determining text complexity and the general readability of texts (in this study, texts assigned to Senior High School learners in Ghana). To clearly establish this link, the levels of LD of assigned texts have to be first determined, and then, compared with their corresponding readability values. To that end, Ure’s (1971) LD formula and Halliday’s (1985b) LD formula have been used to determine the LD values of the sampled texts. Gunning’s (1952) and Flesch’s (1948) readability indices have been used to determine the readability values of same assigned texts. LD values for the sampled texts were found to be generally very high (above the widely agreed minimum of 40%); as their corresponding readability values were low. The correlation between LD and readability is therefore adequately revealed: high LD translates directly into low readability while low LD translates directly into high readability. Implicatively, low readability of texts definitely affects learner progress negatively. Keywords: lexical density, readability, Senior High School learners in Ghana, Ure’s (1971) LD formula, Halliday’s (1985) LD formula, Gunning’s (1952) readability index, Flesch’s (1948) readability index

Assessment of the Quality and Readability of Web-Based Arabic Health Information on Halitosis: Infodemiological Study.

Halitosis, characterized by an undesirable mouth odor, represents a common concern. This study aims to assess the quality and readability of web-based Arabic health information on halitosis as the internet is becoming a prominent global source of medical information. A total of 300 Arabic websites were retrieved from Google using 3 commonly used phrases for halitosis in Arabic. The quality of the websites was assessed using benchmark criteria established by the Journal of the American Medical Association, the DISCERN tool, and the presence of the Health on the Net Foundation Code of Conduct (HONcode). The assessment of readability (Flesch Reading Ease [FRE], Simple Measure of Gobbledygook, and Flesch-Kincaid Grade Level [FKGL]) was conducted using web-based readability indexes. A total of 127 websites were examined. Regarding quality assessment, 87.4% (n=111) of websites failed to fulfill any Journal of the American Medical Association requirements, highlighting a lack of authorship (authors' contributions), attribution (references), disclosure (sponsorship), and currency (publication date). The DISCERN tool had a mean score of 34.55 (SD 7.46), with the majority (n=72, 56.6%) rated as moderate quality, 43.3% (n=55) as having a low score, and none receiving a high DISCERN score, indicating a general inadequacy in providing quality health information to make decisions and treatment choices. No website had HONcode certification, emphasizing the concern over the credibility and trustworthiness of these resources. Regarding readability assessment, Arabic halitosis websites had high readability scores, with 90.5% (n=115) receiving an FRE score ≥80, 98.4% (n=125) receiving a Simple Measure of Gobbledygook score <7, and 67.7% (n=86) receiving an FKGL score <7. There were significant correlations between the DISCERN scores and the quantity of words (P<.001) and sentences (P<.001) on the websites. Additionally, there was a significant relationship (P<.001) between the number of sentences and FKGL and FRE scores. While readability was found to be very good, indicating that the information is accessible to the public, the quality of Arabic halitosis websites was poor, reflecting a significant gap in providing reliable and comprehensive health information. This highlights the need for improving the availability of high-quality materials to ensure Arabic-speaking populations have access to reliable information about halitosis and its treatment options, tying quality and availability together as critical for effective health communication.

ChatGPT for improving postoperative instructions in multiple fields of plastic surgery

BackgroundClear discharge instructions are vital for patients and caregivers to manage post-op care at home. However, they often exceed the sixth-grade reading level recommended by national associations. We hypothesize that ChatGPT could help rewrite instructions to this level for increased accessibility and comprehension. We aim to assess readability, understandability, actionability, and safety of ChatGPT rewritten post-op instructions in four Plastic Surgery subspecialties: Breast, Craniofacial, Hand, and Aesthetic surgery. MethodsWe obtained post-op instructions from four index procedures in Plastic Surgery. We utilized ChatGPT to rewrite at the sixth- and fourth-grade reading levels. Readability was determined by seven readability indexes, understandability and actionability by the PEMAT-P questionnaire, and safety by the primary surgeons. ResultsOverall, average readability of the original post-op instructions ranged between seventh-eighth grade. Of the sixth-grade ChatGPT instructions, only one was lowered to the sixth-grade level. Of the fourth-grade ChatGPT instructions, all were reduced to sixth-grade or below, but none achieved fourth-grade level. Understandability scores increased as reading level decreased while actionability scores decreased for fourth-grade rewrites. Safety was not compromised in all rewrites. ConclusionsOur findings suggest that ChatGPT can adapt post-op instructions to a more readable sixth-grade level without compromising safety. We suggest prompting ChatGPT to write 1-2 grade levels lower than the desired reading level. While understandability increased for all ChatGPT rewrites, actionability decreased for fourth-grade level instructions. Sixth-grade remains as the optimal reading level for post-op instructions. This study demonstrates that ChatGPT can help improve patient care by improving readability of post-op instructions.

Generating colloquial radiology reports with large language models.

Patients are increasingly being given direct access to their medical records. However, radiology reports are written for clinicians and typically contain medical jargon, which can be confusing. One solution is for radiologists to provide a "colloquial" version that is accessible to the layperson. Because manually generating these colloquial translations would represent a significant burden for radiologists, a way to automatically produce accurate, accessible patient-facing reports is desired. We propose a novel method to produce colloquial translations of radiology reports by providing specialized prompts to a large language model (LLM). Our method automatically extracts and defines medical terms and includes their definitions in the LLM prompt. Using our method and a naive strategy, translations were generated at 4 different reading levels for 100 de-identified neuroradiology reports from an academic medical center. Translations were evaluated by a panel of radiologists for accuracy, likability, harm potential, and readability. Our approach translated the Findings and Impression sections at the 8th-grade level with accuracies of 88% and 93%, respectively. Across all grade levels, our approach was 20% more accurate than the baseline method. Overall, translations were more readable than the original reports, as evaluated using standard readability indices. We find that our translations at the eighth-grade level strike an optimal balance between accuracy and readability. Notably, this corresponds to nationally recognized recommendations for patient-facing health communication. We believe that using this approach to draft patient-accessible reports will benefit patients without significantly increasing the burden on radiologists.

Assessment of readability, reliability, and quality of ChatGPT®, BARD®, Gemini®, Copilot®, Perplexity® responses on palliative care.

There is no study that comprehensively evaluates data on the readability and quality of "palliative care" information provided by artificial intelligence (AI) chatbots ChatGPT®, Bard®, Gemini®, Copilot®, Perplexity®. Our study is an observational and cross-sectional original research study. In our study, AI chatbots ChatGPT®, Bard®, Gemini®, Copilot®, and Perplexity® were asked to present the answers of the 100 questions most frequently asked by patients about palliative care. Responses from each 5 AI chatbots were analyzed separately. This study did not involve any human participants. Study results revealed significant differences between the readability assessments of responses from all 5 AI chatbots (P < .05). According to the results of our study, when different readability indexes were evaluated holistically, the readability of AI chatbot responses was evaluated as Bard®, Copilot®, Perplexity®, ChatGPT®, Gemini®, from easy to difficult (P < .05). In our study, the median readability indexes of each of the 5 AI chatbots Bard®, Copilot®, Perplexity®, ChatGPT®, Gemini® responses were compared to the "recommended" 6th grade reading level. According to the results of our study answers of all 5 AI chatbots were compared with the 6th grade reading level, statistically significant differences were observed in the all formulas (P < .001). The answers of all 5 artificial intelligence robots were determined to be at an educational level well above the 6th grade level. The modified DISCERN and Journal of American Medical Association scores was found to be the highest in Perplexity® (P < .001). Gemini® responses were found to have the highest Global Quality Scale score (P < .001). It is emphasized that patient education materials should have a readability level of 6th grade level. Of the 5 AI chatbots whose answers about palliative care were evaluated, Bard®, Copilot®, Perplexity®, ChatGPT®, Gemini®, their current answers were found to be well above the recommended levels in terms of readability of text content. Text content quality assessment scores are also low. Both the quality and readability of texts should be brought to appropriate recommended limits.

Evaluation of the Appropriateness and Readability of ChatGPT-4 Responses to Patient Queries on Uveitis

To compare the utility of ChatGPT-4 as an online uveitis patient education resource with existing patient education websites. Evaluation of technology. Not applicable. The term "uveitis" was entered into the Google search engine, and the first 8 nonsponsored websites were selected to be enrolled in the study. Information regarding uveitis for patients was extracted from Healthline, Mayo Clinic, WebMD, National Eye Institute, Ocular Uveitis and Immunology Foundation, American Academy of Ophthalmology, Cleveland Clinic, and National Health Service websites. ChatGPT-4 was then prompted to generate responses about uveitis in both standard and simplified formats. To generate the simplified response, the following request was added to the prompt: 'Please provide a response suitable for the average American adult, at a sixth-grade comprehension level.' Three dual fellowship-trained specialists, all masked to the sources, graded the appropriateness of the contents (extracted from the existing websites) and responses (generated responses by ChatGPT-4) in terms of personal preference, comprehensiveness, and accuracy. Additionally, 5 readability indices, including Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook index were calculated using an online calculator, Readable.com, to assess the ease of comprehension of each answer. Personal preference, accuracy, comprehensiveness, and readability of contents and responses about uveitis. A total of 497 contents and responses, including 71 contents from existing websites, 213 standard responses, and 213 simplified responses from ChatGPT-4 were recorded and graded. Standard ChatGPT-4 responses were preferred and perceived to be more comprehensive by dually trained (uveitis and retina) specialist ophthalmologists while maintaining similar accuracy level compared with existing websites. Moreover, simplified ChatGPT-4 responses matched almost all existing websites in terms of personal preference, accuracy, and comprehensiveness. Notably, almost all readability indices suggested that standard ChatGPT-4 responses demand a higher educational level for comprehension, whereas simplified responses required lower level of education compared with the existing websites. This study shows that ChatGPT can provide patients with an avenue to access comprehensive and accurate information about uveitis, tailored to their educational level. The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Comparative Analysis of Large Language Models in Simplifying Turkish Ultrasound Reports to Enhance Patient Understanding

Objective: To evaluate and compare the abilities of Language Models (LLMs) in simplifying Turkish ultrasound (US) findings for patients. Methods: We assessed the simplification performance of four LLMs: ChatGPT 4, Gemini 1.5 Pro, Claude 3 Opus, and Perplexity, using fifty fictional Turkish US findings. Comparison was based on Ateşman’s Readability Index and word count. Three radiologists rated medical accuracy, consistency, and comprehensibility on a Likert scale from 1 to 5. Statistical tests (Friedman, Wilcoxon, and Spearman correlation) examined differences in LLMs' performance. Results: Gemini 1.5 Pro, ChatGPT-4, and Claude 3 Opus received high Likert scores for medical accuracy, consistency, and comprehensibility (mean: 4.7–4.8). Perplexity scored significantly lower (mean: 4.1, p&lt;0.001). Gemini 1.5 Pro achieved the highest readability score (mean: 61.16), followed by ChatGPT-4 (mean: 58.94) and Claude 3 Opus (mean: 51.16). Perplexity had the lowest readability score (mean: 47.01). Gemini 1.5 Pro and ChatGPT-4 used significantly more words compared to Claude 3 Opus and Perplexity (p&lt;0.001). Linear correlation analysis revealed a positive correlation between word count of fictional US findings and responses generated by Gemini 1.5 Pro (correlation coefficient = 0.38, p&lt;0.05) and ChatGPT-4 (correlation coefficient = 0.43, p&lt;0.001). Conclusion: This study highlights strong potential of LLMs in simplifying Turkish US findings, improving accessibility and clarity for patients. Gemini 1.5 Pro, ChatGPT-4, and Claude 3 Opus performed well, highlighting their effectiveness in healthcare communication. Further research is required to fully understand the integration of LLMs into clinical practice and their influence on patient comprehension and decision-making.

Enhancing Health Literacy: Evaluating the Readability of Patient Handouts Revised by ChatGPT's Large Language Model.

To use an artificial intelligence (AI)-powered large language model (LLM) to improve readability of patient handouts. Review of online material modified by AI. Academic center. Five handout materials obtained from the American Rhinologic Society (ARS) and the American Academy of Facial Plastic and Reconstructive Surgery websites were assessed using validated readability metrics. The handouts were inputted into OpenAI's ChatGPT-4 after prompting: "Rewrite the following at a 6th-grade reading level." The understandability and actionability of both native and LLM-revised versions were evaluated using the Patient Education Materials Assessment Tool (PEMAT). Results were compared using Wilcoxon rank-sum tests. The mean readability scores of the standard (ARS, American Academy of Facial Plastic and Reconstructive Surgery) materials corresponded to "difficult," with reading categories ranging between high school and university grade levels. Conversely, the LLM-revised handouts had an average seventh-grade reading level. LLM-revised handouts had better readability in nearly all metrics tested: Flesch-Kincaid Reading Ease (70.8 vs 43.9; P < .05), Gunning Fog Score (10.2 vs 14.42; P < .05), Simple Measure of Gobbledygook (9.9 vs 13.1; P < .05), Coleman-Liau (8.8 vs 12.6; P < .05), and Automated Readability Index (8.2 vs 10.7; P = .06). PEMAT scores were significantly higher in the LLM-revised handouts for understandability (91 vs 74%; P < .05) with similar actionability (42 vs 34%; P = .15) when compared to the standard materials. Patient-facing handouts can be augmented by ChatGPT with simple prompting to tailor information with improved readability. This study demonstrates the utility of LLMs to aid in rewriting patient handouts and may serve as a tool to help optimize education materials. Level VI.

Readability and Accessibility of Patient-Education Materials for Heart Failure in the United States

Online education materials are widely used by patients and caregivers to understand the management of complex chronic diseases such as heart failure (HF). Organizations such as the American Medical Association and National Institutes of Health recommend that materials be written at a 6th grade reading level. The current study examined the readability and accessibility of online education materials for patients with HF. Whole page text from each included website was entered into an online readability calculator. Five validated readability indices (Flesch-Kincaid Grade Level, Flesch Reading Ease Scale, Gunning Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook (SMOG Index)) were used to evaluate each source. Websites were categorized by source (government, public, and private). The availability of audiovisual accessibility features and content in non-English languages were assessed for each website. Of the 36 online resources analyzed, the median readability level was 9-10th grade by the Flesch-Kincaid Grade Level and college level using the Flesch Reading Ease Scale. The Gunning Fog Index and Coleman-Liau Index both showed median readability scores corresponding to a 12th grade reading level, while the SMOG Index showed a median score corresponding to that of the 9th grade. Only 10 websites (28%) offered information in languages other than English, and none provided comprehensive accessibility features for users with disabilities. Common online educational materials for patients with HF are characterized by a higher readability level than that recommended by the National Institutes of Health and American Medical Association with limited multilingual and accessibility options, potentially limiting the accessibility of resources to patients and caregivers.

Readability analysis of ChatGPT's responses on lung cancer

For common diseases such as lung cancer, patients often use the internet to obtain medical information. As a result of advances in artificial intelligence and large language models such as ChatGPT, patients and health professionals use these tools to obtain medical information. The aim of this study was to evaluate the readability of ChatGPT-generated responses with different readability scales in the context of lung cancer. The most common questions in the lung cancer section of Medscape® were reviewed, and questions on the definition, etiology, risk factors, diagnosis, treatment, and prognosis of lung cancer (both NSCLC and SCLC) were selected. A set of 80 questions were asked 10 times to ChatGPT via the OpenAI API. ChatGPT's responses were tested using various readability formulas. The mean Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning FOG Scale, SMOG Index, Automated Readability Index, Coleman-Liau Index, Linsear Write Formula, Dale-Chall Readability Score, and Spache Readability Formula scores are at a moderate level (mean and standard deviation: 40.52 ± 9.81, 12.56 ± 1.66, 13.63 ± 1.54, 14.61 ± 1.45, 15.04 ± 1.97, 14.24 ± 1.90, 11.96 ± 2.55, 10.03 ± 0.63 and 5.93 ± 0.50, respectively). The readability levels of the answers generated by ChatGPT are "collage" and above and are difficult to read. Perhaps in the near future, the ChatGPT can be programmed to produce responses that are appropriate for people of different educational and age groups.

Lessons to be learned when designing comprehensible patient-oriented online information about temporomandibular disorders.

Temporomandibular disorders (TMD) are a prevalent ailment with a global impact, affecting a substantial number of individuals. While some individuals are receiving treatment from orthodontists for TMD, a significant proportion of individuals obtain knowledge through websites. Our purpose had been to evaluate, from a patient-oriented perspective, the readability of home pages of websites scored in the 10 most prominent devoted to TMD. We also determined what level of education would have been needed to get an overview of the information on the websites under scrutiny. This approach ensures that our findings are centred on the patient experience, providing insights into how accessible and understandable websites about TMD. We determined the top 10 patient-focused English language websites by searching for 'temporomandibular disorders' in the 'no country redirect' plugin of the Google Chrome browser (www.google.com/ncr). The readability of the texts was assessed using the Gunning fog index (GFI), Coleman Liau index (CLI), Automated readability index (ARI) Simple Measure of Gobbledygook (SMOG), Flesch Kincald grade level (FKGL), Flesh reasing ease (FRE) (https://readabilityformulas.com). The mean Flesch reading ease index score was determined to be 48.67, accompanied by a standard deviation of 15.04 and these websites require an average of 13.49 years of formal education (GFI), with a standard deviation of 2.62, for ease of understanding. Our research indicates that a significant proportion of websites related to TMD can be defined as a level of complexity that exceeds the ability to read comprehension of the general population.