Average Reading Level Research Articles

Comparing ChatGPT vs. Surgeon-Generated Informed Consent Documentation for Plastic Surgery Procedures

Abstract Background Informed consent is a crucial requirement of a patient’s surgical care but can be a burdensome task. Artificial intelligence (AI) and machine learning language models may provide an alternative approach to writing detailed, readable consent forms in an efficient manner. No studies have assessed the accuracy and completeness of AI-generated consents for aesthetic plastic surgeries. Objectives This study aims to compare the length, reading level, accuracy, and completeness of informed consent forms that are AI chatbot (ChatGPT-4, OpenAI, San Francisco, CA)-generated versus plastic surgeon-generated for the most commonly performed aesthetic plastic surgeries. Methods This study is a cross-sectional design comparing informed consent forms created by the American Association of Plastic Surgeons (ASPS) with informed consent forms generated by ChatGPT-4 for the five most commonly performed plastic surgery procedures: liposuction, breast augmentation, abdominoplasty, breast lift, and blepharoplasty. Results The average word count of ChatGPT forms was lower than for the ASPS generated forms (1023 vs 2901, p=0.01). Average reading level for ChatGPT forms was also lower than ASPS forms (11.2 vs 12.5, p=0.02). There was no difference between accuracy and completeness scores for general descriptions of the surgery, risks, benefits, or alternatives. The mean overall impression score for ChatGPT consents was 2.33, whereas it was 2.23 for ASPS consent forms (p=0.18). Conclusions Our study demonstrates that informed consent forms generated by ChatGPT were significantly shorter and more readable than ASPS forms with no significant difference in completeness and accuracy.

Standard HIPAA Authorization Forms Decreased Response Rates for a Multi-site Pragmatic Trial.

The Health Insurance Portability and Accountability Act (HIPAA) aims to safeguard patient information; however, complex legal language may lead to confusion and mistrust, and hinder enrollment in clinical trials. To evaluate the effect of a standard HIPAA authorization included in mailed survey packets on study enrollment for a multi-site pragmatic trial. This study is nested within an advance care planning pragmatic trial at 50 primary care clinics across three University of California (UC) Health Systems. We included English and Spanish-speaking seriously ill patients. One third of eligible patients received and 2/3 did not receive the HIPAA authorization in their enrollment packet. We compared enrollment rates at 3months and assessed the readability, understandability, and actionability of the standard HIPAA form using the Federal Plain Language Guidelines Checklist for Plain Language, the Automatic Readability Checker consensus calculator (grade 8 is the average reading level for US adults), and the Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P, 0-100%, 70% considered the minimum). Of 4632 eligible patients (mean age 71, 48% women, 11% Spanish-speaking, 40% racial/ethnic minority); 1543 received a mailed enrollment packet with a HIPAA form and 3089 did not. Patients mailed the HIPAA form were less likely to enroll (10.2% vs. 14.8%, p < 0.001). The standard HIPAA form scored at the 12th grade reading level, had a PEMAT-P Understandability score of 42%, had an Actionability score of 40%, and only met 50% of Federal Plain Language Guideline Checklist items. The inclusion of a standard HIPAA authorization in mailed enrollment packets for a large pragmatic trial led to lower rates of study enrollment. This study informs how HIPAA authorization forms should be redesigned to be more accessible to patients to prevent unnecessary barriers to research enrollment.

Patient-friendliness of ChatGPT responses: NSCLC targeted therapy.

231 Background: Non-small cell lung cancer (NSCLC) targeted therapies are complex, often requiring extensive patient education beyond what can be conveyed in a single provider visit. Given the average US reading level is at the 8th grade, and many patients seek additional information online, it is vital to evaluate the readability of online resources. This study aims to determine the patient friendliness of information provided by AI chatbots like ChatGPT in explaining NSCLC targeted therapies compared to other commonly accessed sources. Methods: This analysis included targeted therapies approved by the FDA for EGFR-mutated NSCLC: Afatinib, Erlotinib, Osimertinib, Dacomitinib, and Gefitinib. Information sources included ChatGPT 4, ChatGPT 3.5, the Patient Information section of the FDA label, and the Google featured snippet. Each version of ChatGPT was queried 10 times for each individual therapy – “My doctor is suggesting starting therapy with (Drug Name). What can I expect with this medication?”. All responses, labels, and snippets were examined by Microsoft Word for word count (WC) and Flesch Reading Ease Score (FRES), which considers average sentence length and number of syllables per word to assess readability. Data analysis was performed using ANOVA in SAS. Results: Across all therapies, Mean FRES for ChatGPT 4 was 31.21, ChatGPT 3.5 26.69, FDA labels 50.78, and Google snippets 33.1. Mean WC for ChatGPT 4 was 312.54, ChatGPT 3.5 355.78, FDA labels 1220, and Google snippets 42. Significant differences in FRES were found among sources for Afatinib (p-value &lt;0.0001), Erlotinib (p-value 0.0206), Osimertinib (p-value &lt;0.0001), Dacomitinib (p-value 0.0125), and Gefitinib (p-value &lt;0.0001). Significant differences in WC were also found among sources for Afatinib (p-value &lt;0.0001), Erlotinib (p-value &lt;0.0001), Osimertinib (p-value &lt;0.0001), and Gefitinib (p-value 0.0045). No significant difference in WC was found for Dacomitinib (p-value 0.2807). Conclusions: While the FDA label provided the most readable information, none of the sources aligned with the average 8th-grade reading level in the US, indicating a gap in accessible patient education for NSCLC targeted therapies. Notably, readability varies significantly between ChatGPT versions, affecting accessibility for users of the free version, ChatGPT 3.5. Although ChatGPT responses are more succinct compared to the verbose FDA labels, their higher education-level readability presents a barrier to comprehension. This study highlights the need for further research into optimizing AI-generated patient education materials to be both accurate and comprehensible at appropriate reading levels, thereby improving health communication strategies.

Comparing the Ability of Google and ChatGPT to Accurately Respond to Oculoplastics-Related Patient Questions and Generate Customized Oculoplastics Patient Education Materials.

To compare the accuracy and readability of responses to oculoplastics patient questions provided by Google and ChatGPT. Additionally, to assess the ability of ChatGPT to create customized patient education materials. We executed a Google search to identify the 3 most frequently asked patient questions (FAQs) related to 10 oculoplastics conditions. FAQs were entered into both the Google search engine and the ChatGPT tool and responses were recorded. Responses were graded for readability using five validated readability indices and for accuracy by six oculoplastics surgeons. ChatGPT was instructed to create patient education materials at various reading levels for 8 oculoplastics procedures. The accuracy and readability of ChatGPT-generated procedural explanations were assessed. ChatGPT responses to patient FAQs were written at a significantly higher average grade level than Google responses (grade 15.6 vs 10.0, p < 0.001). ChatGPT responses (93% accuracy) were significantly more accurate (p < 0.001) than Google responses (78% accuracy) and were preferred by expert panelists (79%). ChatGPT accurately explained oculoplastics procedures at an above average reading level. When instructed to rewrite patient education materials at a lower reading level, grade level was reduced by approximately 4 (15.7 vs 11.7, respectively, p < 0.001) without sacrificing accuracy. ChatGPT has the potential to provide patients with accurate information regarding their oculoplastics conditions. ChatGPT may also be utilized by oculoplastic surgeons as an accurate tool to provide customizable patient education for patients with varying health literacy. A better understanding of oculoplastics conditions and procedures amongst patients can lead to informed eye care decisions.

Assessment of printed lung cancer surgery patient education materials in the United States

ObjectiveWell-designed patient education materials (PEMs) increase health literacy, which has been linked to better surgical patient outcomes. The quality of lung cancer surgery PEMs is unknown, however. Here we assessed printed lung cancer surgery PEMs for readability, understandability, actionability, and accessibility. MethodsVarious lung cancer programs throughout the United States were contacted for their lung cancer surgery PEMs. The readability of the received materials was calculated using 6 readability tests. Four thoracic surgeon–advanced practice practitioner dyads scored the PEMs for understandability, actionability, and accessibility using the Patient Education Material Assessment Tool and the Accessibility Assessment Tool, with the recommended minimum threshold of 70%. One-sample t tests were performed to compare each parameter against its recommended threshold. ResultsOut of 34 institutions contacted, 18 (52.9%) provided PEMs. The average reading level of the PEMs ranged from 7th grade to 11th grade, significantly exceeding the recommended 6th grade health literacy threshold (P < .01). Although mean understandability (73.7 ± 13.2%) and actionability (70.2 ± 17.8%) scores were not significantly different from the minimum threshold, and the mean accessibility score (81.8 ± 13.5%) was significantly higher than the threshold (P < .05), there was wide variation in the scores. Most PEMs scored well in organization and writing but lacked other features that can enhance patient understanding, such as visual aids and summaries. ConclusionsPEMs are written at reading levels that are too advanced for patients. Although PEMs scored well in understandability, actionability, and accessibility, analysis of individual items revealed the need for improvement, including the use of shorter sentences, more visual aids and summaries, and expansion of language translations.

Prompt matters: evaluation of large language model chatbot responses related to Peyronie's disease.

Despite direct access to clinicians through the electronic health record, patients are increasingly turning to the internet for information related to their health, especially with sensitive urologic conditions such as Peyronie's disease (PD). Large language model (LLM) chatbots are a form of artificial intelligence that rely on user prompts to mimic conversation, and they have shown remarkable capabilities. The conversational nature of these chatbots has the potential to answer patient questions related to PD; however, the accuracy, comprehensiveness, and readability of these LLMs related to PD remain unknown. To assess the quality and readability of information generated from 4 LLMs with searches related to PD; to see if users could improve responses; and to assess the accuracy, completeness, and readability of responses to artificial preoperative patient questions sent through the electronic health record prior to undergoing PD surgery. The National Institutes of Health's frequently asked questions related to PD were entered into 4 LLMs, unprompted and prompted. The responses were evaluated for overall quality by the previously validated DISCERN questionnaire. Accuracy and completeness of LLM responses to 11 presurgical patient messages were evaluated with previously accepted Likert scales. All evaluations were performed by 3 independent reviewers in October 2023, and all reviews were repeated in April 2024. Descriptive statistics and analysis were performed. Without prompting, the quality of information was moderate across all LLMs but improved to high quality with prompting. LLMs were accurate and complete, with an average score of 5.5 of 6.0 (SD, 0.8) and 2.8 of 3.0 (SD, 0.4), respectively. The average Flesch-Kincaid reading level was grade 12.9 (SD, 2.1). Chatbots were unable to communicate at a grade 8 reading level when prompted, and their citations were appropriate only 42.5% of the time. LLMs may become a valuable tool for patient education for PD, but they currently rely on clinical context and appropriate prompting by humans to be useful. Unfortunately, their prerequisite reading level remains higher than that of the average patient, and their citations cannot be trusted. However, given their increasing uptake and accessibility, patients and physicians should be educated on how to interact with these LLMs to elicit the most appropriate responses. In the future, LLMs may reduce burnout by helping physicians respond to patient messages.

Enhancing Health Literacy: Evaluating the Readability of Patient Handouts Revised by ChatGPT's Large Language Model.

To use an artificial intelligence (AI)-powered large language model (LLM) to improve readability of patient handouts. Review of online material modified by AI. Academic center. Five handout materials obtained from the American Rhinologic Society (ARS) and the American Academy of Facial Plastic and Reconstructive Surgery websites were assessed using validated readability metrics. The handouts were inputted into OpenAI's ChatGPT-4 after prompting: "Rewrite the following at a 6th-grade reading level." The understandability and actionability of both native and LLM-revised versions were evaluated using the Patient Education Materials Assessment Tool (PEMAT). Results were compared using Wilcoxon rank-sum tests. The mean readability scores of the standard (ARS, American Academy of Facial Plastic and Reconstructive Surgery) materials corresponded to "difficult," with reading categories ranging between high school and university grade levels. Conversely, the LLM-revised handouts had an average seventh-grade reading level. LLM-revised handouts had better readability in nearly all metrics tested: Flesch-Kincaid Reading Ease (70.8 vs 43.9; P < .05), Gunning Fog Score (10.2 vs 14.42; P < .05), Simple Measure of Gobbledygook (9.9 vs 13.1; P < .05), Coleman-Liau (8.8 vs 12.6; P < .05), and Automated Readability Index (8.2 vs 10.7; P = .06). PEMAT scores were significantly higher in the LLM-revised handouts for understandability (91 vs 74%; P < .05) with similar actionability (42 vs 34%; P = .15) when compared to the standard materials. Patient-facing handouts can be augmented by ChatGPT with simple prompting to tailor information with improved readability. This study demonstrates the utility of LLMs to aid in rewriting patient handouts and may serve as a tool to help optimize education materials. Level VI.

Readability and Higher Education Access: Exploring Community College Admissions Information on Websites

ABSTRACT Although United States community college enrollment rose in Fall 2023, the community college sector as a whole has contracted by 37% since 2010, representing over 2.6 million students. As a result, community college practitioners, specifically, those working in admissions and enrollment management should ensure that admissions instructions are simple enough so that students can read and comprehend how to apply for admission to community college. However, research at the four-year institutional level has found that admission information for prospective students is complex, often written above the 14th-grade English reading comprehension level. Here, this study gathered admissions application instructions from 949 community colleges in the United States to evaluate the grade-level readability of these instructions. The results suggest that the average readability level of community college admission information is near the 14th-grade level, with negligible differences between institutional sectors. Implications for practice and community college student access are addressed.

Using artificial intelligence to generate medical literature for urology patients: a comparison of three different large language models

PurposeLarge language models (LLMs) are a form of artificial intelligence (AI) that uses deep learning techniques to understand, summarize and generate content. The potential benefits of LLMs in healthcare is predicted to be immense. The objective of this study was to examine the quality of patient information leaflets (PILs) produced by 3 LLMs on urological topics.MethodsPrompts were created to generate PILs from 3 LLMs: ChatGPT-4, PaLM 2 (Google Bard) and Llama 2 (Meta) across four urology topics (circumcision, nephrectomy, overactive bladder syndrome, and transurethral resection of the prostate). PILs were evaluated using a quality assessment checklist. PIL readability was assessed by the Average Reading Level Consensus Calculator.ResultsPILs generated by PaLM 2 had the highest overall average quality score (3.58), followed by Llama 2 (3.34) and ChatGPT-4 (3.08). PaLM 2 generated PILs were of the highest quality in all topics except TURP and was the only LLM to include images. Medical inaccuracies were present in all generated content including instances of significant error. Readability analysis identified PaLM 2 generated PILs as the simplest (age 14–15 average reading level). Llama 2 PILs were the most difficult (age 16–17 average).ConclusionWhile LLMs can generate PILs that may help reduce healthcare professional workload, generated content requires clinician input for accuracy and inclusion of health literacy aids, such as images. LLM-generated PILs were above the average reading level for adults, necessitating improvement in LLM algorithms and/or prompt design. How satisfied patients are to LLM-generated PILs remains to be evaluated.

Both English- and Spanish-Language Anterior Cruciate Ligament Reconstruction Online Patient Education Materials Are Written at Higher-Than-Recommended Reading Levels

PurposeThe purpose of this study is to examine the overall reading levels of anterior cruciate ligament reconstruction online patient education materials (OPEMs) written in English and Spanish. MethodsInvestigators conducted Google searches for OPEMs utilizing “ACL surgery” and “cirugía LCA” as English and Spanish search terms. Several measures of readability were used to analyze 25 English (Flesch Reading Ease, Flesch Reading Ease Grade Level, Flesch-Kincaid Grade Level, Coleman-Liau Index, Gunning-Fox Index, and the Simple Measure of Gobbledygook) and 25 Spanish (Fernández-Huerta Index, Fernández-Huerta Grade Level, and INFLESZ) OPEMs. English and Spanish OPEMs were compared based on mean overall grade level and number of OPEMs written below a seventh or ninth-grade reading level. ResultsEnglish OPEMs demonstrated a higher mean overall grade level compared to Spanish OPEMs (10.48 ± 1.86 vs 8.64 ± 1.22, p<.001). No significant differences were noted in the number of OPEMs written below a seventh-grade reading level. However, significantly more Spanish OPEMs were written below a ninth-grade reading level, compared to English OPEMs (56% vs 16%, p=.003). ConclusionAlthough Spanish OPEMs were written at a lower reading level, average readability for both English and Spanish OPEMs was significantly higher than the recommended level. Across both languages, only a single English webpage met the AMA-recommended sixth-grade reading level. More Spanish articles were written at or below the average adult reading level in the United States.

Understanding the cancer health literacy gap: NCCN member institutions’ website readability across the high mortality cancers in the US

BackgroundThe readability of cancer educational content often exceeds the average reading level of the U.S. population. This study evaluated the readability of online educational material for the five deadliest cancers in the US. MethodsWe assessed the readability of diagnosis and treatment information across 34 NCCN member institution websites. Readability was analyzed using four validated formulas: Fry Readability Score, Flesch-Kincaid Grade Level, Gunning-Fog Index, and Simple Measure of Gobbledygook. The primary outcome was the mean organizational readability level for each resource, and secondary outcomes included differences in readability between diagnosis and treatment materials and an analysis of sentence and word complexity. ResultsThe mean readability level across 34 websites was 13 ± 1.96, equivalent to a university freshman level. Treatment materials were significantly harder to read than diagnosis materials (14 ± 1.8 vs. 12 ± 1.8, p < 0.01) across all cancers. Readability varied significantly depending on the type of cancer, with material relating to the diagnosis of pancreatic cancer requiring the highest educational level (13 ± 1.9) and material relating to treating breast cancer requiring the lowest reading proficiency (12 ± 1.7). Furthermore, the analysis demonstrated that cancer diagnosis materials contained a lower median percentage of complex and long words than treatment materials but had a higher percentage of long sentences. ConclusionsThese findings underscore the need for organizations to prioritize matching the nation’s literacy levels. Despite the growing reliance on online resources for patient education, the current readability levels hinder patient comprehension

Readability of Online Spanish Materials for Breast Reconstruction Using Deep Inferior Epigastric Perforator Flaps.

Background The internet has become an increasingly popular tool for patients to find information pertaining to medical procedures. Although the information is easily accessible, data shows that many online educational materials pertaining to surgical subspecialties are far above the average reading level in the United States. The aim of this study was to evaluate the English and Spanish online materials for the deep inferior epigastric perforator (DIEP) flap reconstruction procedure. Methods The first eight institutional or organizational websites that provided information on the DIEP procedure in English and Spanish were included. Each website was evaluated using the Patient Education and Materials Assessment Tool (PEMAT), Cultural Sensitivity Assessment Tool (CSAT), and either Simplified Measure of Gobbledygook (SMOG) for English websites or Spanish Orthographic Length (SOL) for Spanish websites. Results The English websites had a statistically lower CSAT score compared to the Spanish websites (p=0.006). However, Spanish websites had a statistically higher percentage of complex words compared to English sources (p<0.001). An analysis of reading grade levels through SMOG and SOL scores revealed that Spanish websites had statistically lower scores (p<0.001). There were no statistically significant differences in the understandability or actionability scores between the English and Spanish websites. Conclusions Online educational materials on the DIEP flap reconstruction procedure should be readable, understandable, actionable, and culturally sensitive. Our analysis revealed that improvements can be made in understandability and actionability on these websites.Plastic surgeons should be aware of what constitutes a great online educational resource and what online educational materials their patients will have access to.

Readability and Suitability of Local Disaster Preparedness Materials in Predominantly Hispanic Border Communities: A Case Study of Cameron County, Texas

Abstract Community awareness is a key component of disaster resilience and numerous agencies developed a plethora of preparedness materials. Previous studies evaluated national-level resources and social media content indicating that these efforts are not effectively reaching vulnerable populations, as the information is provided at or above the secondary education level. However, the local-specific content has been mostly overlooked in the analyses, subsequently neglecting the needs of areas with specific demographics and hazards, like predominantly Hispanic border communities. Thus, this study turns to local preparedness and mitigation materials, rather than general internet searches or national level documents. We analyze information provided on 239 webpages of local authorities in Cameron county in the southernmost tip of Texas. We evaluate: 1) the availability of languages and visual materials, 2) the reading difficulty level, and 3) the suitability of content. Our results demonstrate the lack of suitable and readable information provided on the local authorities’ websites with the average readability level implying “difficult” to “very confusing”, and the average suitability score equivalent to “below average/adequate”. Moreover, the lack of Spanish-language content indicates an under-utilization of the otherwise vernacular language. We discuss those results in the socio-cultural context of the border community.

Optimizing Readability of Patient-Facing Hand Surgery Education Materials Using Chat Generative Pretrained Transformer 3.5

To address patient health literacy, the American Medical Association and the National Institutes of Health recommend that readability of patient education materials should not exceed an eighth grade reading level. However, patient-facing materials often remain above the recommended average reading level. Current online calculators provide readability scores; however, they lack the ability to provide text-specific feedback, which may streamline the process of simplifying patient materials. The purpose of this study was to evaluate Chat Generative Pretrained Transformer (ChatGPT) 3.5 as a tool for optimizing patient-facing hand surgery education materials through reading level analysis and simplification. The readability of 18 patient-facing hand surgery education materials was compared by a traditional online calculator for reading level and ChatGPT 3.5. The original excerpts were then entered into ChatGPT 3.5 and simplified by the artificial intelligence tool. The simplified excerpts were scored by the same calculators. The readability scores for the original excerpts from the online calculator and ChatGPT 3.5 were similar. The simplified excerpts' scores were lower than the originals, with a mean of 7.28, less than the maximum recommended8. The use of ChatGPT 3.5 for the purpose of simplification and readability analysis of patient-facing hand surgery materials is efficient and may help facilitate the conveyance of important health information. ChatGPT 3.5 rendered readability scores comparable with traditional readability calculators, in addition to excerpt-specific feedback. It was also able to simplify materials to the recommended grade levels. By confirming ChatGPT3.5's ability to assess and simplify patient education materials, this study offers a practical solution for potentially improving patient comprehension, engagement, and health outcomes in clinical settings.

Barriers to Health Care Communication: Patient Education Resource Readability and Spanish Translation for Slipped Capital Femoral Epiphysis

BackgroundThe American Medical Association (AMA) and the National Institutes of Health (NIH) recommend that patient education resources be written at 6th and 8th grade reading levels, respectively. However, many existing resources are written above this level. In addition, the majority of patient education resources are only available in English, despite the significant Spanish-speaking population in the United States. The purpose of this study was to determine the availability and readability of online patient education resources for Slipped Capital Femoral Epiphysis (SCFE) provided by the top-ranked pediatric orthopaedic hospitals and major pediatric orthopedic professional societies. MethodsWe performed a cross-sectional analysis to determine the availability of English and Spanish language resources for SCFE provided by the top pediatric orthopaedic hospitals and major professional societies. We then assessed the readability of available patient education materials via three measures of readability. Statistical analysis was performed using descriptive, univariate, and correlation analyses. ResultsIn total, 79.6% (39/49) of hospitals and 66.7% (2/3) of professional societies provided English language resources for SCFE. In contrast, Spanish language resources were provided by 34.7% (17/49) and 33% (1/3) hospitals and professional societies. English language resources scored significantly lower than Spanish language resources using the Flesch-Kincaid and Fernandez-Huerta reading ease scales, respectively (60.32 vs. 65.89, p<0.05). There was no difference in grade level calculated by Fry Graph/Gilliam-Peña-Mountain (9.5 vs. 8.5, p>0.05), but English language resources had a significantly higher grade level calculated by SMOG/SOL (11.9 vs. 10.4, p<0.01). There was greater availability of Spanish language resources in states with >20% primary Spanish- speakers (p<0.05). ConclusionThe majority of top-ranked hospitals and professional societies provide online educational materials on SCFE in English; however, only a third provide similar materials in Spanish. Additionally, the average reading level of English and Spanish language resources is above the recommendations from the AMA and NIH. Additional effort and resources should be allocated toward improving SCFE patient education materials to increase accessibility to broader patient and caregiver populations. LEVEL OF EVIDENCELevel III KEY CONCEPTS(1)Online patient education resources for Slipped Capital Femoral Epiphysis (SCFE) are written above the recommended reading grade level for both English and Spanish.(2)There is limited availability of Spanish-language patient education recourses for SCFE, despite a significant Spanish-speaking population in the United States.(3)There is greater availability of Spanish-language resources for SCFE in states with a primary Spanish-speaking population greater than 20%.

What Questions Do Patients Ask About Biologic Therapies for Chronic Sinusitis With Nasal Polyps?

Objective: The objective of this study is to explore the questions commonly asked online about biologic therapies for use in treatment of chronic sinusitis with nasal polyps (CRSwNP) and the quality of the available content. Methods: Most common search terms were identified via Google Trends. People Also Ask (PAA) questions were identified and extracted with their associated website using an online data-scraping program [Search Engine Optimization (SEO) Minion, Keywords Everywhere]. Sources were evaluated using Flesch Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE, higher number = better) score for readability; Journal of American Medical Association (JAMA) Benchmark criteria for quality assessment (0-4, 4 = all criteria met). Results: A total of 143 unique PAA questions and unique websites were identified. Questions were organized into 3 categories: questions about biologic therapies overall (38.46%), about CRSwNP (37.76%), and about treatment options for CRSwNP (23.78%). Websites answering PAA questions were from commercial (60.14%), medical practice (13.99%), academic (13.29%), and government (12.59%) sources. FKGL scores found the average reading level to be at approximately a 12th grade level (SD = 3.297) alongside a low reading ease FRE score of 37.6 (SD = 16.77). Mean JAMA criteria scores were 0.9895 (SD = 0.848), indicating largely low-quality materials. Conclusion: Biologic therapies are a novel treatment option for CRSwNP, and participants are seeking more information about these treatments and disease state. Online resources regarding biologics should be presented at a lower reading level. Sources with evidence-based information are needed. Physicians should be aware of these limitations in online material and counsel accordingly by curating and directing patients to good sources.Level of Evidence: Step 4.

Enhancing Readability of Online Patient-Facing Content: The Role of AI Chatbots in Improving Cancer Information Accessibility.

Internet-based health education is increasingly vital in patient care. However, the readability of online information often exceeds the average reading level of the US population, limiting accessibility and comprehension. This study investigates the use of chatbot artificial intelligence to improve the readability of cancer-related patient-facing content. We used ChatGPT 4.0 to rewrite content about breast, colon, lung, prostate, and pancreas cancer across 34 websites associated with NCCN Member Institutions. Readability was analyzed using Fry Readability Score, Flesch-Kincaid Grade Level, Gunning Fog Index, and Simple Measure of Gobbledygook. The primary outcome was the mean readability score for the original and artificial intelligence (AI)-generated content. As secondary outcomes, we assessed the accuracy, similarity, and quality using F1 scores, cosine similarity scores, and section 2 of the DISCERN instrument, respectively. The mean readability level across the 34 websites was equivalent to a university freshman level (grade 13±1.5). However, after ChatGPT's intervention, the AI-generated outputs had a mean readability score equivalent to a high school freshman education level (grade 9±0.8). The overall F1 score for the rewritten content was 0.87, the precision score was 0.934, and the recall score was 0.814. Compared with their original counterparts, the AI-rewritten content had a cosine similarity score of 0.915 (95% CI, 0.908-0.922). The improved readability was attributed to simpler words and shorter sentences. The mean DISCERN score of the random sample of AI-generated content was equivalent to "good" (28.5±5), with no significant differences compared with their original counterparts. Our study demonstrates the potential of AI chatbots to improve the readability of patient-facing content while maintaining content quality. The decrease in requisite literacy after AI revision emphasizes the potential of this technology to reduce health care disparities caused by a mismatch between educational resources available to a patient and their health literacy.

아동양육시설 초등학생들의 기초문해력 실태 조사

The purpose of this study was to examine the level of basic literacy skills among elementary school students in residential child care institution based on their reading development. A total of 118 students participated in the reading screening test. According to the results of the screening test, students with severe reading difficulties were classified as ‘at risk’, students with reading difficulties were classified as ‘some risk’, and students with average and above average reading levels were classified as ‘typical’. The ‘at risk’ students were administered the Korean language acquisition and word recognition tests, while students in the ‘some risk’, and ‘typical’, were administered the reading fluency and comprehension tests. The main research findings are as follows. First, elementary school students in residential child care institution generally showed low levels of basic literacy skills. Second, almost students in the ‘at risk’ group were found to have difficulty in word recognition. Third, students in the ‘some risk’ and ‘typical’ groups were found to have greater difficulty in reading comprehension than reading fluency. Based on these research results, implications for enhancing basic literacy skills of elementary school students in residential child care institution were proposed.

Can Generative Artificial Intelligence Enhance Health Literacy About Lateral Epicondylitis?

Health literacy is a critical determinant of a patient's overall health status, and studies have demonstrated a consistent link between poor health literacy and negative health outcomes. The Centers for Disease Control and Prevention (CDC) and the National Institutes of Health (NIH) advise that patient educational materials (PEMs) should be written at an eighth-grade reading level or lower, matching the average reading level of adult Americans. The purpose of this study was to investigate the ability of generative artificial intelligence (AI) to edit PEMs from orthopaedic institutions to meet the CDC and NIH guidelines. PEMs about lateral epicondylitis (LE) from the top 25 ranked orthopaedic institutions from the 2022 U.S. News & World Report Best Hospitals Specialty Ranking were gathered. ChatGPT Plus (version 4.0) was then instructed to rewrite PEMs on LE from these institutions to comply with CDC and NIH-recommended guidelines. Readability scores were calculated for the original and rewritten PEMs, and paired t-tests were used to determine statistical significance. Analysis of the original and edited PEMs about LE revealed significant reductions in reading grade level and word count of 3.70 ± 1.84 (p<0.001) and 346.72 ± 364.63 (p<0.001), respectively. Our study demonstrated generative AI's ability to rewrite PEM about LE at a reading comprehension level that conforms to the CDC and NIH guidelines. Hospital administrators and orthopaedic surgeons should consider the findings of this studyand the potential utility of artificial intelligence when crafting PEMs of their own.

A mobile app providing individually-tailored psychoeducation about sleep for older adults with chronic health conditions and low health literacy.

This paper reports on a study of a mobile app that provides tailored information about sleep to individuals aged 40 and older who have chronic health conditions and low health literacy. The sleep module was a part of a multitopic app focused on chronic disease self-management. Participants were randomly assigned to receive sleep psychoeducation at reading levels equivalent to 3rd, 6th or 8th grade. The primary outcome measure was the Pittsburgh Sleep Quality Index (PSQI), which was completed at baseline, after the intervention, and again three months later. Outcomes were assessed using repeated measures mixed effects models. Most participants were Black, Indigenous, or Other Persons of Color (BIPOC; 87%); they had average reading level at the 7th grade. Health literacy, socioeconomic status, and number of health conditions were related to the PSQI. The PSQI score decreased over the course of the three study visits for all groups, consistent with a small to medium effect size (d = 0.40). No effect of treatment group was observed. Participants were positive about the usefulness and helpfulness of the app. Results suggest that a brief tailored information intervention may be beneficial for individuals aged 40 and older who have low health literacy and chronic health conditions. Further development of the intervention may enhance its clinical effectiveness.