Sixth-grade Reading Level Research Articles

Online resources for strabismus: an evaluation of readability, complexity, and suitability.

Over one-third of US adults have never attended college, creating a large disparity in the readability of online health materials. Decreased health literacy and accessibility to medical information negatively affect patients and well-informed patients are more likely to experience better health outcomes (1). The NIH and AMA recommend patient-intended education materials be written at a sixth-grade reading level (2), therefore, this study analyzed the accessibility of the top ten web pages for "strabismus." The first ten online resources returned in a Google search for "strabismus" were analyzed. Web pages were then assessed for the readability level (Simple Measure of Gobbledygook), complexity (PMOSE/IKIRSCH), and suitability (Suitability Assessment of Materials). Two independent raters assessed the complexity and suitability. Readability analysis of the strabismus resources revealed an average reading grade level of 11.4 ± 1.07. There was a statistical difference in the reading grade level between the .com and .gov, and the .org and .com websites (p = .029 and p = .031, respectively). Complexity analysis revealed a mean score of 6.50 ± 2.29, corresponding to an 8th-12th grade reading level. The suitability assessment showed a mean value of 70.3 10.1%, representing a "superior" score for the information provided to the reader. The inter-rater agreement was similar for the complexity and fair for the suitability analysis. On average, online resources for strabismus have a low complexity level. However, the majority of the top ten articles reviewed are above the recommended literacy level, indicating a need for revision. The vast amount of available online health resources have significantly affected the field of medicine. Most patients research their disease process using online sources and many reference this material before their initial ophthalmologic consultation. Considering that more than half of Americans read below the equivalent of a sixth-grade level and that the AMA/NIH recommend all patient-intended materials to be written above this level, there is a health literacy disconnect. This limits patients' ability to educate themselves about their medical conditions and participate in informed conversations regarding their healthcare. Patients who are unable to interpret health information accurately have increased rates of hospitalization, develop more medical conditions, and experience a higher rate of mortality. This preventable impediment to informed healthcare care magnifies the urgency for easily readable online resources that are formatted in a manner that is clear to understand and suitable for patients with lower health literacy.

Improving Patient Understanding of Emergency Department Discharge Instructions

Introduction: Previous studies have shown that patients in the emergency department (ED) are frequently given incomplete discharge instructions that are written at least four grade levels above the recommended sixth-grade reading level, leading to poor understanding. Our aims in this study were to implement standardized discharge instructions containing six key components written at a more appropriate reading level for common emergency department (ED) diagnoses to improve patient understanding. Methods: We conducted this study in a 20-bed ED at an urban Veteran’s Administration hospital. Data was collected via in-person patient and clinician interviews. Patient interviews were conducted after patients received their discharge instructions. We compared patient responses to clinician responses and marked them as incorrect, partially correct, or correct with a score of 0, 0.5, or 1, respectively. The maximum possible score for each interview was six. Six key components of discharge instructions were asked about: diagnosis; new medications; at-home care; duration of illness; reasons to return; and follow-up. There were 25 patients in the pre-intervention group and 20 in the intervention group with the standardized set of instructions. We performed a Mann-Whitney U test on the total interview scores in the control and intervention groups and conducted a sub-analysis on the individual scores for each of the six key components. Results: The patients in the intervention demonstrated a statistically significant increase in patient-clinician correlation when compared to the patients in the pre-intervention group overall (P < 0.05), and two of the six key components of the discharge instructions individually showed statistically significant increase in patient-clinician correlation when standardized discharge instructions were used. Conclusion: Patients who received the standardized discharge instructions had improved understanding of their discharge instructions. Future opportunities extending off this pilot study include expanding the number of diagnoses for which standardized instructions are used and investigating patient-centered outcomes related to these instructions.

Large language models: a new frontier in paediatric cataract patient education

Background/aimsThis was a cross-sectional comparative study. We evaluated the ability of three large language models (LLMs) (ChatGPT-3.5, ChatGPT-4, and Google Bard) to generate novel patient education materials (PEMs) and improve...

Expanding Accessibility in Cleft Care: The Role of Artificial Intelligence in Improving Literacy of Alveolar Bone Grafting Information.

The American Medical Association (AMA) recommends patient education materials (PEMs) be written at or below a sixth grade reading level. This study seeks to determine the quality, readability, and content of available alveolar bone grafting (ABG) PEMs and determine if artificial intelligence can improve PEM readability. Review of free online PEMs. Online ABG PEMs were retrieved from different authoring body types (hospital/academic center, medical society, or private practice). None. Content was assessed by screening PEMs for specific ABG-related topics. Quality was evaluated with the Patient Education Material Assessment Tool (PEMAT), which has measures of understandability and actionability. Open-access readability software (WebFX) determined readability with Flesch Reading Ease, Flesch-Kincaid Grade Level, and Gunning-Fog Index. PEMs were rewritten with ChatGPT, and readability metrics were reassessed. Quality, readability, and content of ABG PEMs. 34 PEMs were analyzed. Regarding quality, the average PEMAT-understandability score was 67.0 ± 16.2%, almost at the minimum acceptable score of 70.0% (p = 0.281). The average PEMAT-actionability score was low at 33.0 ± 24.1%. Regarding readability, the average Flesch Reading Ease score was 64.6 ± 12.8, categorized as "standard/plain English." The average Flesch-Kincaid Grade Level was 8.0 ± 2.3, significantly higher than AMA recommendations (p < 0.0001). PEM rewriting with ChatGPT improved Flesch-Kincaid Grade Level to 6.1 ± 1.3 (p < 0.0001). Available ABG PEMs are above the recommended reading level, yet ChatGPT can improve PEM readability. Future studies should improve areas of ABG PEMs that are most lacking, such as actionability.

ChatGPT for improving postoperative instructions in multiple fields of plastic surgery

BackgroundClear discharge instructions are vital for patients and caregivers to manage post-op care at home. However, they often exceed the sixth-grade reading level recommended by national associations. We hypothesize that ChatGPT could help rewrite instructions to this level for increased accessibility and comprehension. We aim to assess readability, understandability, actionability, and safety of ChatGPT rewritten post-op instructions in four Plastic Surgery subspecialties: Breast, Craniofacial, Hand, and Aesthetic surgery. MethodsWe obtained post-op instructions from four index procedures in Plastic Surgery. We utilized ChatGPT to rewrite at the sixth- and fourth-grade reading levels. Readability was determined by seven readability indexes, understandability and actionability by the PEMAT-P questionnaire, and safety by the primary surgeons. ResultsOverall, average readability of the original post-op instructions ranged between seventh-eighth grade. Of the sixth-grade ChatGPT instructions, only one was lowered to the sixth-grade level. Of the fourth-grade ChatGPT instructions, all were reduced to sixth-grade or below, but none achieved fourth-grade level. Understandability scores increased as reading level decreased while actionability scores decreased for fourth-grade rewrites. Safety was not compromised in all rewrites. ConclusionsOur findings suggest that ChatGPT can adapt post-op instructions to a more readable sixth-grade level without compromising safety. We suggest prompting ChatGPT to write 1-2 grade levels lower than the desired reading level. While understandability increased for all ChatGPT rewrites, actionability decreased for fourth-grade level instructions. Sixth-grade remains as the optimal reading level for post-op instructions. This study demonstrates that ChatGPT can help improve patient care by improving readability of post-op instructions.

Evaluation of the Quality and Readability of Web-Based Information Regarding Foreign Bodies of the Ear, Nose, and Throat: Qualitative Content Analysis.

Foreign body (FB) inhalation, ingestion, and insertion account for 11% of emergency admissions for ear, nose, and throat conditions. Children are disproportionately affected, and urgent intervention may be needed to maintain airway patency and prevent blood vessel occlusion. High-quality, readable online information could help reduce poor outcomes from FBs. We aim to evaluate the quality and readability of available online health information relating to FBs. In total, 6 search phrases were queried using the Google search engine. For each search term, the first 30 results were captured. Websites in the English language and displaying health information were included. The provider and country of origin were recorded. The modified 36-item Ensuring Quality Information for Patients tool was used to assess information quality. Readability was assessed using a combination of tools: Flesch Reading Ease score, Flesch-Kincaid Grade Level, Gunning-Fog Index, and Simple Measure of Gobbledygook. After the removal of duplicates, 73 websites were assessed, with the majority originating from the United States (n=46, 63%). Overall, the quality of the content was of moderate quality, with a median Ensuring Quality Information for Patients score of 21 (IQR 18-25, maximum 29) out of a maximum possible score of 36. Precautionary measures were not mentioned on 41% (n=30) of websites and 30% (n=22) did not identify disk batteries as a risky FB. Red flags necessitating urgent care were identified on 95% (n=69) of websites, with 89% (n=65) advising patients to seek medical attention and 38% (n=28) advising on safe FB removal. Readability scores (Flesch Reading Ease score=12.4, Flesch-Kincaid Grade Level=6.2, Gunning-Fog Index=6.5, and Simple Measure of Gobbledygook=5.9 years) showed most websites (56%) were below the recommended sixth-grade level. The current quality and readability of information regarding FBs is inadequate. More than half of the websites were above the recommended sixth-grade reading level, and important information regarding high-risk FBs such as disk batteries and magnets was frequently excluded. Strategies should be developed to improve access to high-quality information that informs patients and parents about risks and when to seek medical help. Strategies to promote high-quality websites in search results also have the potential to improve outcomes.

Quality and Readability of Noise-Induced Hearing Loss Prevention-Related Websites.

This cross-sectional website analysis aimed to determine the readability and quality of English and Spanish websites pertaining to the prevention of noise-induced hearing loss. Cross-sectional website analysis. Various online search engines. We queried four popular search engines using the term "noise-induced hearing loss prevention" to reveal the top 50 English and top 50 Spanish websites for data collection. Websites meeting inclusion criteria were stratified based on the presence of a Health on the Net Code certificate (independent assessment of honesty, reliability, and quality). Websites were then independently reviewed by experts using the DISCERN criteria in order to assess information quality. Readability was calculated using the Flesch reading ease score for English and the Fernandez-Huerta formula for Spanish websites. Thirty-six English websites and 32 Spanish websites met the inclusion criteria. English websites had significantly lower readability (average = 56.34, SD = 11.17) compared to Spanish websites (average = 61.88, SD = 5.33) (P < .05). Spanish websites (average = 37, SD = 8.47) were also of significantly higher quality than English websites (average = 25.13, SD = 10.11). This study emphasizes the importance of providing quality and readable materials to patients seeking information about noise-induced hearing loss prevention. All of the English and Spanish websites reviewed were written at a level higher than the American Medical Association-recommended sixth-grade reading level. The study also highlights the need for evidence-based information online provided by experts in our field.

The Use of Artificial Intelligence to Improve Readability of Otolaryngology Patient Education Materials.

The recommended readability of health education materials is at the sixth-grade level. Artificial intelligence (AI) large language models such as the newly released ChatGPT4 might facilitate the conversion of patient-education materials at scale. We sought to ascertain whether online otolaryngology education materials meet recommended reading levels and whether ChatGPT4 could rewrite these materials to the sixth-grade level. We also wished to ensure that converted materials were accurate and retained sufficient content. Seventy-one articles from patient educational materials published online by the American Academy of Otolaryngology-Head and Neck Surgery were selected. Articles were entered into ChatGPT4 with the prompt "translate this text to a sixth-grade reading level." Flesch Reading Ease Score (FRES) and Flesch-Kincaid Grade Level (FKGL) were determined for each article before and after AI conversion. Each article and conversion were reviewed for factual inaccuracies, and each conversion was reviewed for content retention. The 71 articles had an initial average FKGL of 11.03 and FRES of 46.79. After conversion by ChatGPT4, the average FKGL across all articles was 5.80 and FRES was 77.27. Converted materials provided enough detail for patient education with no factual errors. We found that ChatGPT4 improved the reading accessibility of otolaryngology online patient education materials to recommended levels quickly and effectively. Physicians can determine whether their patient education materials exceed current recommended reading levels by using widely available measurement tools, and then apply AI dialogue platforms to modify materials to more accessible levels as needed. Level 5.

Both English- and Spanish-Language Anterior Cruciate Ligament Reconstruction Online Patient Education Materials Are Written at Higher-Than-Recommended Reading Levels

PurposeThe purpose of this study is to examine the overall reading levels of anterior cruciate ligament reconstruction online patient education materials (OPEMs) written in English and Spanish. MethodsInvestigators conducted Google searches for OPEMs utilizing “ACL surgery” and “cirugía LCA” as English and Spanish search terms. Several measures of readability were used to analyze 25 English (Flesch Reading Ease, Flesch Reading Ease Grade Level, Flesch-Kincaid Grade Level, Coleman-Liau Index, Gunning-Fox Index, and the Simple Measure of Gobbledygook) and 25 Spanish (Fernández-Huerta Index, Fernández-Huerta Grade Level, and INFLESZ) OPEMs. English and Spanish OPEMs were compared based on mean overall grade level and number of OPEMs written below a seventh or ninth-grade reading level. ResultsEnglish OPEMs demonstrated a higher mean overall grade level compared to Spanish OPEMs (10.48 ± 1.86 vs 8.64 ± 1.22, p<.001). No significant differences were noted in the number of OPEMs written below a seventh-grade reading level. However, significantly more Spanish OPEMs were written below a ninth-grade reading level, compared to English OPEMs (56% vs 16%, p=.003). ConclusionAlthough Spanish OPEMs were written at a lower reading level, average readability for both English and Spanish OPEMs was significantly higher than the recommended level. Across both languages, only a single English webpage met the AMA-recommended sixth-grade reading level. More Spanish articles were written at or below the average adult reading level in the United States.

The diagnostic and triage accuracy of the GPT-3 artificial intelligence model: an observational study

Artificial intelligence (AI) applications in health care have been effective in many areas of medicine, but they are often trained for a single task using labelled data, making deployment and generalisability challenging. How well a general-purpose AI language model performs diagnosis and triage relative to physicians and laypeople is not well understood. We compared the predictive accuracy of Generative Pre-trained Transformer 3 (GPT-3)'s diagnostic and triage ability for 48 validated synthetic case vignettes (<50 words; sixth-grade reading level or below) of both common (eg, viral illness) and severe (eg, heart attack) conditions to a nationally representative sample of 5000 lay people from the USA who could use the internet to find the correct options and 21 practising physicians at Harvard Medical School. There were 12 vignettes for each of four triage categories: emergent, within one day, within 1 week, and self-care. The correct diagnosis and triage category (ie, ground truth) for each vignette was determined by two general internists at Harvard Medical School. For each vignette, human respondents and GPT-3 were prompted to list diagnoses in order of likelihood, and the vignette was marked as correct if the ground-truth diagnosis was in the top three of the listed diagnoses. For triage accuracy, we examined whether the human respondents' and GPT-3's selected triage was exactly correct according to the four triage categories, or matched a dichotomised triage variable (emergent or within 1 day vs within 1 week or self-care). We estimated GPT-3's diagnostic and triage confidence on a given vignette using a modified bootstrap resampling procedure, and examined how well calibrated GPT-3's confidence was by computing calibration curves and Brier scores. We also performed subgroup analysis by case acuity, and an error analysis for triage advice to characterise how its advice might affect patients using this tool to decide if they should seek medical care immediately. Among all cases, GPT-3 replied with the correct diagnosis in its top three for 88% (42/48, 95% CI 75-94) of cases, compared with 54% (2700/5000, 53-55) for lay individuals (p<0.0001) and 96% (637/666, 94-97) for physicians (p=0·012). GPT-3 triaged 70% correct (34/48, 57-82) versus 74% (3706/5000, 73-75; p=0.60) for lay individuals and 91% (608/666, 89-93%; p<0.0001) for physicians. As measured by the Brier score, GPT-3 confidence in its top prediction was reasonably well calibrated for diagnosis (Brier score=0·18) and triage (Brier score=0·22). We observed an inverse relationship between case acuity and GPT-3 accuracy (p<0·0001) with a fitted trend line of -8·33% decrease in accuracy for every level of increase in case acuity. For triage error analysis, GPT-3 deprioritised truly emergent cases in seven instances. A general-purpose AI language model without any content-specific training could perform diagnosis at levels close to, but below, physicians and better than lay individuals. We found that GPT-3's performance was inferior to physicians for triage, sometimes by a large margin, and its performance was closer to that of lay individuals. Although the diagnostic performance of GPT-3 was comparable to physicians, it was significantly better than a typical person using a search engine. The National Heart, Lung, and Blood Institute.

Readability of online patient education material for foregut surgery.

Health literacy is the ability of individuals to use basic health information and services to make well-informed decisions. Low health literacy among surgical patients has been associated with nonadherence to preoperative and/or discharge instructions as well as poor comprehension of surgery. It likely poses as a barrier to patients considering foregut surgery which requires an understanding of different treatment options and specific diet instructions. The objective of this study was to assess and compare the readability of online patient education materials (PEM) for foregut surgery. Using Google, the terms "anti-reflux surgery, "GERD surgery," and "foregut surgery" were searched and a total of 30 webpages from universities and national organizations were selected. The readability of the text was assessed with seven instruments: Flesch Reading Ease formula (FRE), Gunning Fog (GF), Flesch-Kincaid Grade Level (FKGL), Coleman Liau Index (CL), Simple Measure of Gobbledygook (SMOG), Automated Readability Index (ARI), and Linsear Write Formula (LWF). Mean readability scores were calculated with standard deviations. We performed a qualitative analysis gathering characteristics such as, type of information (preoperative or postoperative), organization, use of multimedia, inclusion of a version in another language. The overall average readability of the top PEM for foregut surgery was 12th grade. There was only one resource at the recommended sixth grade reading level. Nearly half of PEM included some form of multimedia. The American Medical Association and National Institute of Health have recommended that PEMs to be written at the 5th-6th grade level. The majority of online PEM for foregut surgery is above the recommended reading level. This may be a barrier for patients seeking foregut surgery. Surgeons should be aware of the potential gaps in understanding of their patients to help them make informed decisions and improve overall health outcomes.

Readability and Information Quality in Cancer Information From a Free vs Paid Chatbot

The mainstream use of chatbots requires a thorough investigation of their readability and quality of information. To identify readability and quality differences in information between a free and paywalled chatbot cancer-related responses, and to explore if more precise prompting can mitigate any observed differences. This cross-sectional study compared readability and information quality of a chatbot's free vs paywalled responses with Google Trends' top 5 search queries associated with breast, lung, prostate, colorectal, and skin cancers from January 1, 2021, to January 1, 2023. Data were extracted from the search tracker, and responses were produced by free and paywalled ChatGPT. Data were analyzed from December 20, 2023, to January 15, 2024. Free vs paywalled chatbot outputs with and without prompt: "Explain the following at a sixth grade reading level: [nonprompted input]." The primary outcome measured the readability of a chatbot's responses using Flesch Reading Ease scores (0 [graduate reading level] to 100 [easy fifth grade reading level]). Secondary outcomes included assessing consumer health information quality with the validated DISCERN instrument (overall score from 1 [low quality] to 5 [high quality]) for each response. Scores were compared between the 2 chatbot models with and without prompting. This study evaluated 100 chatbot responses. Nonprompted free chatbot responses had lower readability (median [IQR] Flesh Reading ease scores, 52.60 [44.54-61.46]) than nonprompted paywalled chatbot responses (62.48 [54.83-68.40]) (P < .05). However, prompting the free chatbot to reword responses at a sixth grade reading level was associated with increased reading ease scores than the paywalled chatbot nonprompted responses (median [IQR], 71.55 [68.20-78.99]) (P < .001). Prompting was associated with increases in reading ease in both free (median [IQR], 71.55 [68.20-78.99]; P < .001)and paywalled versions (median [IQR], 75.64 [70.53-81.12]; P < .001). There was no significant difference in overall DISCERN scores between the chatbot models, with and without prompting. In this cross-sectional study, paying for the chatbot was found to provide easier-to-read responses, but prompting the free version of the chatbot was associated with increased response readability without changing information quality. Educating the public on how to prompt chatbots may help promote equitable access to health information.

Artificial Intelligence-Generated Patient Education Materials for Helicobacter pylori Infection: A Comparative Analysis.

Patient education contributes to improve public awareness of Helicobacter pylori. Large language models (LLMs) offer opportunities to revolutionize patient education transformatively. This study aimed to assess the quality of patient educational materials (PEMs) generated by LLMs and compared with physician sourced. Unified instruction about composing a PEM about H. pylori at a sixth-grade reading level in both English and Chinese were given to physician and five LLMs (Bing Copilot, Claude 3 Opus, Gemini Pro, ChatGPT-4, and ERNIE Bot 4.0). The assessments of the completeness and comprehensibility of the Chinese PEMs were conducted by five gastroenterologists and 50 patients according to three-point Likert scale. Gastroenterologists were asked to evaluate both English and Chinese PEMs and determine the accuracy and safety. The accuracy was assessed by six-point Likert scale. The minimum acceptable scores were 4, 2, and 2 for accuracy, completeness, and comprehensibility, respectively. The Flesch-Kincaid and Simple Measure of Gobbledygook scoring systems were employed as readability assessment tools. Accuracy and comprehensibility were acceptable for English PEMs of all sources, while completence was not satisfactory. Physician-sourced PEM had the highest accuracy mean score of 5.60 and LLM-generated English PEMs ranged from 4.00 to 5.40. The completeness score was comparable between physician-sourced PEM and LLM-generated PEMs in English. Chinese PEMs from LLMs proned to have lower score in accuracy and completeness assessment than English PEMs. The mean score for completeness of five LLM-generated Chinese PEMs was 1.82-2.70 in patients' perspective, which was higher than gastroenterologists' assessment. Comprehensibility was satisfactory for all PEMs. No PEM met the recommended sixth-grade reading level. LLMs have potential in assisting patient education. The accuracy and comprehensibility of LLM-generated PEMs were acceptable, but further optimization on improving completeness and accounting for a variety of linguistic contexts are essential for enhancing the feasibility.

Readability of English and Spanish online patient education materials for shoulder arthroplasty

BackgroundOnline patient educational materials (OPEMs) are a vital source of information for those undergoing shoulder arthroplasty, though their utility is limited by patient health literacy as the average American reads at an eighth-grade level. To promote optimal reading comprehension, the American Medical Association and National Institutes of Health recommend that OPEMs be written at or below a sixth-grade level. The purpose of this study is to analyze the readability of English and Spanish shoulder arthroplasty OPEMs. MethodsGoogle Search was used to identify the first 25 eligible Shoulder Arthroplasty OPEMs in both English and Spanish using the search terms “shoulder replacement” and “reemplazo de hombro.” Readability of the English OPEMs was calculated using the Flesch-Kincaid Grade Level, Flesch Reading Ease Index (FRE), Flesch Reading Ease Grade Level, Gunning-Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook Index. The readability of Spanish OPEMs was calculated using the Fernandez-Huerta Index (FHI; the Spanish version of FRE), Fernandez-Huerta Index Grade Level, Gutiérrez de Polini's Fórmula de compresibilidad, and Índice Flesch-Szigriszt. ResultsThe mean FHI of Spanish OPEMs was significantly higher than the mean FRE of English OPEMs (60.09 vs. 51.48). As such, Spanish OPEMs were written at a significantly lower grade level than English OPEMs (9.70 vs. 11.53). There was not an English or Spanish OPEM that was written at or below a sixth-grade reading level. 4% of Spanish OPEMs and 4% of English OPEMs were written below an eighth-grade reading level. 52% of Spanish OPEMs and 8% of English OPEMs were written between an eighth- and ninth-grade reading level. The designation of OPEMs as being Very Easy/Easy, Standard/Normal, Fairly/Somewhat Difficult, or Difficult/Very Difficult was significantly dependent on whether the OPEM was written in English or Spanish. Discussion and conclusionEnglish shoulder arthroplasty OPEMs were written at a higher reading difficulty than Spanish OPEMs. However, neither English nor Spanish OPEMs were written at a reading level at or below the recommended sixth-grade reading level. As older adults and those in the United States with limited English proficiency are susceptible to low health literacy, OPEMs must be written at an accessible reading level for all. Addressing these disparities in OPEM readability will improve patient care and understanding of shoulder arthroplasty and other orthopedic procedures.

An Evaluation of the Content Quality, Readability, and Reliability of Publicly Available Web-Based Information on Pneumothorax Surgery in Ireland.

Introduction The internet is increasingly the first port of call for patients introduced to new treatments. Unfortunately, many websites are of poor quality, thereby limiting patients' ability to make informed health decisions.Within thoracic surgery, the treatment options for pneumothoraces may be less intuitive for patients to understand compared to procedures such as lobectomies and wedge resections. Therefore, patients must receive high-quality information to make informed treatment decisions.No study to date has evaluated online information regarding pneumothorax surgery. Knowledge regarding the same may allow physicians to recommend appropriate websites to patients and supplement remaining knowledge gaps. Objective This study aims to evaluate the content, readability, and reliability of online information regarding pneumothorax surgery. Methods A total of 11 search terms including "pneumothorax surgery,""pleurectomy,"and "pleurodesis"were each entered into Google, Bing, and Yahoo. The top 20 websites found through each search were screened, yielding 660 websites. Only free websites designed for patient consumption that provided information on pneumothorax surgery were included. This criterion excluded 581 websites, leaving 79 websites to be evaluated. To evaluate website reliability, the Journal of American Medical Association (JAMA) and DISCERN benchmark criteria were applied.To evaluate the readability, 10 standardized tools were utilized including the Flesch-Kincaid Reading Ease Score. To evaluate website content, a novel,self-designed 10-part questionnaire was utilized to assess whether information deemed essential by the authors was included.It evaluated whether websites comprehensively described the surgery process for patients, including pre- and post-operative care.Website authorship and year of publication were also noted. Results The mean JAMA score was 1.69 ± 1.29out of 4, with only nine websites achieving all four reliability criteria.The median readability score was 13.42 (IQR: 11.48-16.23), which corresponded to a 13th-14thschool grade standard. Only four websites were written at a sixth-grade reading level.In the novel content questionnaire, 31.6% of websites (n = 25) did not mention any side effects of pneumothorax surgery. Similarly, 39.2% (n = 31) did not mention alternative treatment options.There was no correlation between the date of website update and JAMA (r = 0.158, p = 0.123), DISCERN (r = 0.098, p = 0.341), or readability (r = 0.053, p = 0.606) scores. Conclusion Most websites were written above the sixth-grade reading level, as recommended by the US Department of Health and Human Services. Furthermore, the exclusion of essential information regarding pneumothorax surgery from websites highlights the current gaps in online information.These findings emphasize the need to createand disseminatecomprehensive, reliable websites on pneumothorax surgery that enable patients to make informed health decisions.

Readability of Online Patient Education Materials for the 10 Most Common Hand Conditions.

Background Millions of individuals every day turn to the internet for assistance in understanding their hand conditions and potential treatments. While online educational resources appear abundant, there are concerns about whether resources meet the readability recommendations agreed upon by the American Medical Association (AMA) and the National Institutes of Health (NIH). Identifying educational resources that are readable for the majority of patients could improve a patient's understanding of their medical condition, subsequently improving their health outcomes. Methods The readability of the top five websites for the 10 most common hand conditions was examined using the Flesch-Kincaid (FK) analysis, comprising the FK reading ease and FK grade level. The FK reading ease score is an indicator of how difficult a text is to comprehend, while the FK grade level score is the grade level an individual reading a particular text would need to fully understand the text. Results The average FK reading ease was 56.00, which correlates with "fairly difficult (high school)". The average FK corresponded to an eighth-grade reading level, far above the sixth-grade reading level recommendation set by the AMA and NIH. Conclusion Patient education, satisfaction, and the patient-physician relationship can all be improved by providing patients with more readable educational materials. Our study shows there is an opportunity for drastic improvement in the readability of online educational materials. Guiding patients with effective search techniques, advocating for the creation of more readable materials, and having a better understanding of the health literacy barriers patients face will allow hand surgeons to provide more comprehensive care to patients.

Enhancing readability of USFDA patient communications through large language models: a proof-of-concept study

ABSTRACT Background The US Food and Drug Administration (USFDA) communicates new drug safety concerns through drug safety communications (DSCs) and medication guides (MGs), which often challenge patients with average reading abilities due to their complexity. This study assesses whether large language models (LLMs) can enhance the readability of these materials. Methods We analyzed the latest DSCs and MGs, using ChatGPT 4.0© and Gemini© to simplify them to a sixth-grade reading level. Outputs were evaluated for readability, technical accuracy, and content inclusiveness. Results Original materials were difficult to read (DSCs grade level 13, MGs 22). LLMs significantly improved readability, reducing the grade levels to more accessible readings (Single prompt – DSCs: ChatGPT 4.0© 10.1, Gemini© 8; MGs: ChatGPT 4.0© 7.1, Gemini© 6.5. Multiple prompts – DSCs: ChatGPT 4.0© 10.3, Gemini© 7.5; MGs: ChatGPT 4.0© 8, Gemini© 6.8). LLM outputs retained technical accuracy and key messages. Conclusion LLMs can significantly simplify complex health-related information, making it more accessible to patients. Future research should extend these findings to other languages and patient groups in real-world settings.

Evaluating the accuracy and readability of ChatGPT in providing parental guidance for adenoidectomy, tonsillectomy, and ventilation tube insertion surgery

ObjectivesThis study examined the potential of ChatGPT as an accurate and readable source of information for parents seeking guidance on adenoidectomy, tonsillectomy, and ventilation tube insertion surgeries (ATVtis). MethodsChatGPT was tasked with identifying the top 15 most frequently asked questions by parents on internet search engines for each of the three specific surgical procedures. We removed repeated questions from the initial set of 45. Subsequently, we asked ChatGPT to generate answers to the remaining 33 questions. Seven highly experienced otolaryngologists individually assessed the accuracy of the responses using a four-level grading scale, from completely incorrect to comprehensive. The readability of responses was determined using the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) scores. The questions were categorized into four groups: Diagnosis and Preparation Process, Surgical Information, Risks and Complications, and Postoperative Process. Responses were then compared based on accuracy grade, FRE, and FKGL scores. ResultsSeven evaluators each assessed 33 AI-generated responses, providing a total of 231 evaluations. Among the evaluated responses, 167 (72.3 %) were classified as ‘comprehensive.’ Sixty-two responses (26.8 %) were categorized as ‘correct but inadequate,’ and two responses (0.9 %) were assessed as ‘some correct, some incorrect.’ None of the responses were adjudged ‘completely incorrect’ by any assessors. The average FRE and FGKL scores were 57.15(±10.73) and 9.95(±1.91), respectively. Upon analyzing the responses from ChatGPT, 3 (9.1 %) were at or below the sixth-grade reading level recommended by the American Medical Association (AMA). No significant differences were found between the groups regarding readability and accuracy scores (p > 0.05). ConclusionsChatGPT can provide accurate answers to questions on various topics related to ATVtis. However, ChatGPT's answers may be too complex for some readers, as they are generally written at a high school level. This is above the sixth-grade reading level recommended for patient information by the AMA. According to our study, more than three-quarters of the AI-generated responses were at or above the 10th-grade reading level, raising concerns about the ChatGPT text's readability.

How artificial intelligence can provide information about subdural hematoma: Assessment of readability, reliability, and quality of ChatGPT, BARD, and perplexity responses.

Subdural hematoma is defined as blood collection in the subdural space between the dura mater and arachnoid. Subdural hematoma is a condition that neurosurgeons frequently encounter and has acute, subacute and chronic forms. The incidence in adults is reported to be 1.72-20.60/100.000 people annually. Our study aimed to evaluate the quality, reliability and readability of the answers to questions asked to ChatGPT, Bard, and perplexity about "Subdural Hematoma." In this observational and cross-sectional study, we asked ChatGPT, Bard, and perplexity to provide the 100 most frequently asked questions about "Subdural Hematoma" separately. Responses from both chatbots were analyzed separately for readability, quality, reliability and adequacy. When the median readability scores of ChatGPT, Bard, and perplexity answers were compared with the sixth-grade reading level, a statistically significant difference was observed in all formulas (P < .001). All 3 chatbot responses were found to be difficult to read. Bard responses were more readable than ChatGPT's (P < .001) and perplexity's (P < .001) responses for all scores evaluated. Although there were differences between the results of the evaluated calculators, perplexity's answers were determined to be more readable than ChatGPT's answers (P < .05). Bard answers were determined to have the best GQS scores (P < .001). Perplexity responses had the best Journal of American Medical Association and modified DISCERN scores (P < .001). ChatGPT, Bard, and perplexity's current capabilities are inadequate in terms of quality and readability of "Subdural Hematoma" related text content. The readability standard for patient education materials as determined by the American Medical Association, National Institutes of Health, and the United States Department of Health and Human Services is at or below grade 6. The readability levels of the responses of artificial intelligence applications such as ChatGPT, Bard, and perplexity are significantly higher than the recommended 6th grade level.

Comparative Readability Assessment of Four Large Language Models in Answers to Common Contraception Questions [ID 2683638

INTRODUCTION: Large language models (LLMs) have the potential to revolutionize reproductive health education by offering accessible contraception counseling. The CDC, NIH, and AMA recommend presenting patient educational materials at or below a sixth-grade reading level. In this study, we present a comparative analysis of the readability of responses generated by LLMs, including OpenAI's ChatGPT 4.0, OpenAI's ChatGPT 3.5, Google Bard, and Microsoft Bing, to common contraception questions. METHODS: Referencing common contraception questions outlined in a recent review, we presented GPT-4.0, GPT-3.5, Google Bard, and Microsoft Bing with six common contraception questions on June 10, 2023, ensuring no prior chat bias. The readability of LLM outputs was evaluated using Flesch–Kincaid Grade Level (FK), Gunning Fog (GF), Automated Readability Index (ARI), and Coleman–Liau (CL) indices. All index score outputs correspond to a reading grade level (RGL), eg, RGL of 6 represents a sixth-grade level. RESULTS: Of the four LLMs, Google Bard demonstrated the highest readability with an average RGL of 10.6 (GF 10.1, FK 9.6, ARI 11.4, CL 11.4), followed by GPT-3.5 at an average RGL of 13.6 (GF 13.3, FK 12.6, ARI 15.2, CL 16.5). Microsoft Bing and GPT-4.0 yielded responses with higher average RGLs of 14.2 and 15.4, respectively. CONCLUSION: Google Bard and GPT-3.5 offered the most readable responses to contraception questions; however, these models still provided responses significantly above the recommended sixth-grade reading level (RGL of 6). Large language models should undergo evaluation to ensure alignment with established readability standards to optimize patient comprehension in reproductive counseling applications.