Simple Measure Of Gobbledygook Index Research Articles

Artificial intelligence as a modality to enhance the readability of neurosurgical literature for patients.

In this study the authors assessed the ability of Chat Generative Pretrained Transformer (ChatGPT) 3.5 and ChatGPT4 to generate readable and accurate summaries of published neurosurgical literature. Abstracts published in journal issues released between June 2023 and August 2023 (n = 150) were randomly selected from the top 5 ranked neurosurgical journals according to Google Scholar. ChatGPT models were instructed to generate a readable layperson summary of the original abstract from a statistically validated prompt. Readability results and grade-level indicators (RR-GLIs) scores were calculated for GPT3.5- and GPT4-generated summaries and original abstracts. Two physicians independently rated the accuracy of ChatGPT-generated layperson summaries to assess scientific validity. One-way ANOVA followed by pairwise t-test with Bonferroni correction were performed to compare readability scores. Cohen's kappa was used to assess interrater agreement between the two rater physicians. Analysis of 150 original abstracts showed a statistically significant difference for all RR-GLIs between the ChatGPT-generated summaries and original abstracts. The readability scores are formatted as follows (original abstract mean, GPT3.5 summary mean, GPT4 summary mean, p value): Flesch-Kincaid reading grade (12.55, 7.80, 7.70, p < 0.0001); Gunning fog score (15.46, 10.00, 9.00, p < 0.0001); Simple Measure of Gobbledygook (SMOG) index (11.30, 7.13, 6.60, p < 0.0001); Coleman-Liau index (14.67, 11.32, 10.26, p < 0.0001); automated readability index (10.87, 8.50, 7.75, p < 0.0001); and Flesch-Kincaid reading ease (33.29, 68.45, 69.55, p < 0.0001). GPT4-generated summaries demonstrated higher RR-GLIs than GPT3.5-generated summaries in the following categories: Gunning fog score (0.0003); SMOG index (0.027); Coleman-Liau index (< 0.0001); sentences (< 0.0001); complex words (< 0.0001); and % complex words (0.0035). A total of 68.4% and 84.2% of GPT3.5- and GPT4-generated summaries, respectively, maintained moderate scientific accuracy according to the two physician-reviewers. The findings demonstrate promising potential for application of the ChatGPT in patient education. GPT4 is an accessible tool that can be an immediate solution to enhancing the readability of current neurosurgical literature. Layperson summaries generated by GPT4 would be a valuable addition to a neurosurgical journal and would be likely to improve comprehension for patients using internet resources like PubMed.

Assessing the Responses of Large Language Models (ChatGPT-4, Claude 3, Gemini, and Microsoft Copilot) to Frequently Asked Questions in Retinopathy of Prematurity: A Study on Readability and Appropriateness.

To assess the appropriateness and readability of responses provided by four large language models (LLMs) (ChatGPT-4, Claude 3, Gemini, and Microsoft Co-pilot) to parents' queries pertaining to retinopathy of prematurity (ROP). A total of 60 frequently asked questions were collated and categorized into six distinct sections. The responses generated by the LLMs were evaluated by three experienced ROP specialists to determine their appropriateness and comprehensiveness. Additionally, the readability of the responses was assessed using a range of metrics, including the Flesch-Kincaid Grade Level (FKGL), Gunning Fog (GF) Index, Coleman-Liau (CL) Index, Simple Measure of Gobbledygook (SMOG) Index, and Flesch Reading Ease (FRE) score. ChatGPT-4 demonstrated the highest level of appropriateness (100%) and performed exceptionally well in the Likert analysis, scoring 5 points on 96% of questions. The CL Index and FRE scores identified Gemini as the most readable LLM, whereas the GF Index and SMOG Index rated Microsoft Copilot as the most readable. Nevertheless, ChatGPT-4 exhibited the most intricate text structure, with scores of 18.56 on the GF Index, 18.56 on the CL Index, 17.2 on the SMOG Index, and 9.45 on the FRE score. This suggests that the responses demand a college-level comprehension. ChatGPT-4 demonstrated higher performance than other LLMs in responding to questions related to ROP; however, its texts were more complex. In terms of readability, Gemini and Microsoft Copilot were found to be more successful. [J Pediatr Ophthalmol Strabismus. 20XX;XX(X):XXX-XXX.].

Evaluation of the Appropriateness and Readability of ChatGPT-4 Responses to Patient Queries on Uveitis

To compare the utility of ChatGPT-4 as an online uveitis patient education resource with existing patient education websites. Evaluation of technology. Not applicable. The term "uveitis" was entered into the Google search engine, and the first 8 nonsponsored websites were selected to be enrolled in the study. Information regarding uveitis for patients was extracted from Healthline, Mayo Clinic, WebMD, National Eye Institute, Ocular Uveitis and Immunology Foundation, American Academy of Ophthalmology, Cleveland Clinic, and National Health Service websites. ChatGPT-4 was then prompted to generate responses about uveitis in both standard and simplified formats. To generate the simplified response, the following request was added to the prompt: 'Please provide a response suitable for the average American adult, at a sixth-grade comprehension level.' Three dual fellowship-trained specialists, all masked to the sources, graded the appropriateness of the contents (extracted from the existing websites) and responses (generated responses by ChatGPT-4) in terms of personal preference, comprehensiveness, and accuracy. Additionally, 5 readability indices, including Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook index were calculated using an online calculator, Readable.com, to assess the ease of comprehension of each answer. Personal preference, accuracy, comprehensiveness, and readability of contents and responses about uveitis. A total of 497 contents and responses, including 71 contents from existing websites, 213 standard responses, and 213 simplified responses from ChatGPT-4 were recorded and graded. Standard ChatGPT-4 responses were preferred and perceived to be more comprehensive by dually trained (uveitis and retina) specialist ophthalmologists while maintaining similar accuracy level compared with existing websites. Moreover, simplified ChatGPT-4 responses matched almost all existing websites in terms of personal preference, accuracy, and comprehensiveness. Notably, almost all readability indices suggested that standard ChatGPT-4 responses demand a higher educational level for comprehension, whereas simplified responses required lower level of education compared with the existing websites. This study shows that ChatGPT can provide patients with an avenue to access comprehensive and accurate information about uveitis, tailored to their educational level. The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Readability of English and Spanish online patient education materials for shoulder arthroplasty

BackgroundOnline patient educational materials (OPEMs) are a vital source of information for those undergoing shoulder arthroplasty, though their utility is limited by patient health literacy as the average American reads at an eighth-grade level. To promote optimal reading comprehension, the American Medical Association and National Institutes of Health recommend that OPEMs be written at or below a sixth-grade level. The purpose of this study is to analyze the readability of English and Spanish shoulder arthroplasty OPEMs. MethodsGoogle Search was used to identify the first 25 eligible Shoulder Arthroplasty OPEMs in both English and Spanish using the search terms “shoulder replacement” and “reemplazo de hombro.” Readability of the English OPEMs was calculated using the Flesch-Kincaid Grade Level, Flesch Reading Ease Index (FRE), Flesch Reading Ease Grade Level, Gunning-Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook Index. The readability of Spanish OPEMs was calculated using the Fernandez-Huerta Index (FHI; the Spanish version of FRE), Fernandez-Huerta Index Grade Level, Gutiérrez de Polini's Fórmula de compresibilidad, and Índice Flesch-Szigriszt. ResultsThe mean FHI of Spanish OPEMs was significantly higher than the mean FRE of English OPEMs (60.09 vs. 51.48). As such, Spanish OPEMs were written at a significantly lower grade level than English OPEMs (9.70 vs. 11.53). There was not an English or Spanish OPEM that was written at or below a sixth-grade reading level. 4% of Spanish OPEMs and 4% of English OPEMs were written below an eighth-grade reading level. 52% of Spanish OPEMs and 8% of English OPEMs were written between an eighth- and ninth-grade reading level. The designation of OPEMs as being Very Easy/Easy, Standard/Normal, Fairly/Somewhat Difficult, or Difficult/Very Difficult was significantly dependent on whether the OPEM was written in English or Spanish. Discussion and conclusionEnglish shoulder arthroplasty OPEMs were written at a higher reading difficulty than Spanish OPEMs. However, neither English nor Spanish OPEMs were written at a reading level at or below the recommended sixth-grade reading level. As older adults and those in the United States with limited English proficiency are susceptible to low health literacy, OPEMs must be written at an accessible reading level for all. Addressing these disparities in OPEM readability will improve patient care and understanding of shoulder arthroplasty and other orthopedic procedures.

Assessing readability of online patient educational material on concussion and return to play.

Concussions are self-limited forms of mild traumatic brain injury (TBI). Gradual return to play (RTP) is crucial to minimizing the risk of second impact syndrome. Online patient educational materials (OPEM) are often used to guide decision-making. Previous literature has reported that grade-level readability of OPEM is higher than recommended by the American Medical Association and the National Institutes of Health. The authors evaluated the readability of OPEM on concussion and RTP. An online search engine was used to identify websites providing OPEM on concussion and RTP. Text specific to concussion and RTP was extracted from each website and readability was assessed using the following six standardized indices: Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level, Gunning Fog Index, Coleman-Liau Index, Simple Measure of Gobbledygook Index, and Automated Readability Index. One-way ANOVA and Tukey's post hoc test were used to compare readability across sources of information. There were 59 concussion and RTP articles, and readability levels exceeded the recommended 6th grade level, irrespective of the source of information. Academic institutions published OPEM at simpler readability levels (higher FRE scores). Private organizations published OPEM at more complex (higher) grade-level readability levels in comparison with academic and nonprofit institutions (p < 0.05). The readability of OPEM on RTP after concussions exceeds the literacy of the average American. There is a critical need to modify the concussion and RTP OPEM to improve comprehension by a broad audience.

Dyslexia Articles Unboxed: Analyzing Their Readability Level.

Dyslexia is characterized by difficulties with fluent word recognition, decoding, or spelling, and it has been linked to family history. Given the impact of dyslexia on broad academic activities and well-being, ensuring that information about dyslexia is accessible to affected children and their families is vital. This study aims to assess the readability levels of dyslexia-related websites, with the hypothesis that such websites should be written at an appropriate readability level to accommodate those who may also have reading challenges. This study analyzes the readability of 50 web articles on dyslexia using 6 readability formulas: Flesch-Kincaid Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Score, Simple Measure of Gobbledygook (SMOG) Index, Coleman Liau Index, and Automated Readability Index. The search term "What is dyslexia" was used on Google. Each article was analyzed using the online calculation website WebFX. The readability goal for these websites was set at fifth grade, a level recommended for patients with reading challenges. The study found that among the 50 websites, the lowest median readability score was 11.8 (corresponding to a 12th-grade level) on the SMOG Index, while the highest scores were 15.5 on both the Gunning Fog Score and the Coleman Liau Index (indicative of college-level readability). Almost none of the websites had scores below a fifth-grade level. Most websites related to dyslexia are too complex. Tools such as readability metrics and sentence restructuring by AI can help make the information more accessible and understandable to the stakeholders.

Readability and Quality of Online Patient Education Materials Concerning Posterior Cruciate Ligament Reconstruction.

Objective This study aimed to assess the quality of online patient educational materials regarding posterior cruciate ligament (PCL) reconstruction. Methods We performed a search of the top-50 results on Google® (terms: "posterior cruciate ligament reconstruction,""PCL reconstruction,""posterior cruciate ligament surgery," and "PCL surgery") and subsequently filtered to rule out duplicated/inaccessible websites or those containing only videos (67 websites included). Readability was assessed using six formulas: Flesch-Kincaid Reading Ease (FRE), Flesch-Kincaid Grade Level (FKG), Gunning Fog Score (GF), Simple Measure of Gobbledygook (SMOG) Index, Coleman-Liau Index (CLI), Automated Readability Index (ARI); quality was assessed using the JAMA benchmark criteria and recording the presence of the HONcode seal. Results The mean FRE was 49.3 (SD 11.2) and the mean FKG level was 8.09. These results were confirmed by the other readability formulae (average: GF 8.9; SMOG Index 7.3; CLI 14.7; ARI 6.5). A HONcode seal was available for 7.4 % of websites. The average JAMA score was 1.3. Conclusion The reading level of online patient materials concerning PCL reconstruction is too high for the average reader, requiring high comprehension skills. Practice implications Online medical information has been shown to influence patient healthcare decision processes. Patient-oriented educational materials should be clear and easy to understand.

Evaluating the Readability of Online Blood Cancer Education Materials Across Different Readability Measures.

Introduction The National Institutes of Health and the American Medical Association recommend patient education materials (EMs) be at or below the sixth-grade reading level. The American Cancer Society, Leukemia & Lymphoma Society, and National Comprehensive Cancer Network have accurate blood cancer EMs. Methods One hundred one (101) blood cancer EMs from the above organizations were assessed using the following: Flesch Reading Ease Formula (FREF), Flesch-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Simple Measure of Gobbledygook Index (SMOG), and the Coleman-Liau Index (CLI). Results Only 3.96% of patient EMs scored at or below the seventh-grade reading level in all modalities. Healthcare professional education materials (HPEMs) averaged around the college to graduate level. For leukemia and lymphoma patient EMs, there were significant differences for FKGL vs. SMOG, FKGL vs. GFI, FKGL vs. CLI, SMOG vs. CLI, and GFI vs. CLI. For HPEMs, there were significant differences for FKGL vs. GFI and GFI vs. CLI. Conclusion The majority of patient EMs were above the seventh-grade reading level. A lack of easily readable patient EMs could lead to a poor understanding of disease and, thus, adverse health outcomes.Overall, patient EMs should not replace physician counseling. Physicians must close the gaps in patients' understanding throughout their cancer treatment.

Both Patients and Plastic Surgeons Prefer Artificial Intelligence-Generated Microsurgical Information.

With the growing relevance of artificial intelligence (AI)-based patient-facing information, microsurgical-specific online information provided by professional organizations was compared with that of ChatGPT (Chat Generative Pre-Trained Transformer) and assessed for accuracy, comprehensiveness, clarity, and readability. Six plastic and reconstructive surgeons blindly assessed responses to 10 microsurgery-related medical questions written either by the American Society of Reconstructive Microsurgery (ASRM) or ChatGPT based on accuracy, comprehensiveness, and clarity. Surgeons were asked to choose which source provided the overall highest-quality microsurgical patient-facing information. Additionally, 30 individuals with no medical background (ages: 18-81, μ = 49.8) were asked to determine a preference when blindly comparing materials. Readability scores were calculated, and all numerical scores were analyzed using the following six reliability formulas: Flesch-Kincaid Grade Level, Flesch-Kincaid Readability Ease, Gunning Fog Index, Simple Measure of Gobbledygook Index, Coleman-Liau Index, Linsear Write Formula, and Automated Readability Index. Statistical analysis of microsurgical-specific online sources was conducted utilizing paired t-tests. Statistically significant differences in comprehensiveness and clarity were seen in favor of ChatGPT. Surgeons, 70.7% of the time, blindly choose ChatGPT as the source that overall provided the highest-quality microsurgical patient-facing information. Nonmedical individuals 55.9% of the time selected AI-generated microsurgical materials as well. Neither ChatGPT nor ASRM-generated materials were found to contain inaccuracies. Readability scores for both ChatGPT and ASRM materials were found to exceed recommended levels for patient proficiency across six readability formulas, with AI-based material scored as more complex. AI-generated patient-facing materials were preferred by surgeons in terms of comprehensiveness and clarity when blindly compared with online material provided by ASRM. Studied AI-generated material was not found to contain inaccuracies. Additionally, surgeons and nonmedical individuals consistently indicated an overall preference for AI-generated material. A readability analysis suggested that both materials sourced from ChatGPT and ASRM surpassed recommended reading levels across six readability scores.

A Readability Study of Carpal Tunnel Release in 2023.

The National Institutes of Health (NIH) and the American Medical Association (AMA) recommend a sixth-grade reading level for patient-directed content. This study aims to quantitatively evaluate the readability of online information sources related to carpal tunnel surgery using established readability indices. Web searches for "carpal tunnel release" and "carpal tunnel decompression surgery" queries were performed using Google, and the first 20 websites were identified per query. WebFX online software tools were utilized to determine readability. Indices included Flesch Kincaid Reading Ease, Flesch Kincaid Grade Level, Coleman Liau Index, Automated Readability Index, Gunning Fog Score, and the Simple Measure of Gobbledygook Index. Health-specific clickthrough rate (CTR) data were used in order to select the first 20 search engine results page from each query. "Carpal tunnel release" had a mean readability of 8.46, and "carpal tunnel decompression surgery" had a mean readability of 8.70. The range of mean readability scores among the indices used for both search queries was 6.17 to 14.0. The total mean readability for carpal tunnel surgery information was found to be 8.58. This corresponds to approximately a ninth-grade reading level in the United States. The average readability of carpal tunnel surgery online content is three grade levels above the recommended sixth-grade level for patient-directed materials. This discrepancy indicates that existing online materials related to carpal tunnel surgery are more difficult to understand than the standards set by NIH and AMA.

ChatGPT Provides Unsatisfactory Responses to Frequently Asked Questions Regarding Anterior Cruciate Ligament Reconstruction

To determine whether the free online artificial intelligence platform ChatGPT could accurately, adequately, and appropriately answer questions regarding anterior cruciate ligament (ACL) reconstruction surgery. A list of 10 questions about ACL surgery was created based on a review of frequently asked questions that appeared on websites of various orthopaedic institutions. Each question was separately entered into ChatGPT (version 3.5), and responses were recorded, scored, and graded independently by 3 authors. The reading level of the ChatGPT response was calculated using the WordCalc software package, and readability was assessed using the Flesch-Kincaid grade level, Simple Measure of Gobbledygook index, Coleman-Liau index, Gunning fog index, and automated readability index. Of the 10 frequently asked questions entered into ChatGPT, 6 were deemed as unsatisfactory and requiring substantial clarification; 1, as adequate and requiring moderate clarification; 1, as adequate and requiring minor clarification; and 2, as satisfactory and requiring minimal clarification. The mean DISCERN score was 41 (inter-rater reliability, 0.721), indicating the responses to the questions were average. According to the readability assessments, a full understanding of the ChatGPT responses required 13.4 years of education, which corresponds to the reading level of a college sophomore. Most of the ChatGPT-generated responses were outdated and failed to provide an adequate foundation for patients' understanding regarding their injury and treatment options. The reading level required to understand the responses was too advanced for some patients, leading to potential misunderstanding and misinterpretation of information. ChatGPT lacks the ability to differentiate and prioritize information that is presented to patients. Recognizing the shortcomings in artificial intelligence platforms may equip surgeons to better set expectations and provide support for patients considering and preparing for ACL reconstruction.

Can Artificial Intelligence Improve the Readability of Patient Education Materials on Aortic Stenosis? A Pilot Study.

The advent of generative artificial intelligence (AI) dialogue platforms and large language models (LLMs) may help facilitate ongoing efforts to improve health literacy. Additionally, recent studies have highlighted inadequate health literacy among patients with cardiac disease. The aim of the present study was to ascertain whether two freely available generative AI dialogue platforms could rewrite online aortic stenosis (AS) patient education materials (PEMs) to meet recommended reading skill levels for the public. Online PEMs were gathered from a professional cardiothoracic surgical society and academic institutions in the USA. PEMs were then inputted into two AI-powered LLMs, ChatGPT-3.5 and Bard, with the prompt "translate to 5th-grade reading level". Readability of PEMs before and after AI conversion was measured using the validated Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook Index (SMOGI), and Gunning-Fog Index (GFI) scores. Overall, 21 PEMs on AS were gathered. Original readability measures indicated difficult readability at the 10th-12th grade reading level. ChatGPT-3.5 successfully improved readability across all four measures (p < 0.001) to the approximately 6th-7th grade reading level. Bard successfully improved readability across all measures (p < 0.001) except for SMOGI (p = 0.729) to the approximately 8th-9th grade level. Neither platform generated PEMs written below the recommended 6th-grade reading level. ChatGPT-3.5 demonstrated significantly more favorable post-conversion readability scores, percentage change in readability scores, and conversion time compared to Bard (all p < 0.001). AI dialogue platforms can enhance the readability of PEMs for patients with AS but may not fully meet recommended reading skill levels, highlighting potential tools to help strengthen cardiac health literacy in the future.

A comparative analysis of AI-based chatbots: Assessing data quality in orthognathic surgery related patient information

IntroductionThe aim of the current study is to evaluate the quality, reliability, readability, and similarity of data provided by different AI-based chatbots in the field of orthognathic surgery. Materials and methodsGuidelines on orthognathic surgery were reviewed, and a list of questions for patients to ask chatbots was produced by two reasearchers. The questions were categorized into 'General Information and Procedure' and 'Results and Recovery', with 30 questions in each category. Five different scoring criteria were used to evaluate the chatbot responses to 60 questions: Ensuring Quality Information for Patients (EQIP) tool, Reliability Scoring System (adapted from DISCERN), Global Quality Scale (GQS), Simple Measure of Gobbledygook (SMOG) and Similarity Index. ResultsThe highest mean values were observed in OpenEvidence for EQIP tool, SMOG, and Similarity Index, while for Reliability and GQS assessment criteria, MediSearch showed the highest values. When evaluated in terms of reliability and quality, all three AI-based chatbots demonstrated high reliability and good quality; however, they required at least a college-level education for readability based on the SMOG index. Additionally, upon assessment of similarity, ChatGPT-4 displayed high originality, while OpenEvidence exhibited a high degree of similarity. ConclusionAI-based chatbots with a variety of features have usually provided answers with high quality, reliability, and difficult readability to questions. Although the medical information in the field of orthognathic surgery provided through chatbots is of higher quality, it is recommended that individuals consult their healthcare professionals on this issue.

Readability of Spanish-Language Online Patient Educational Materials for Peripheral Artery Disease Do Not Meet Recommended Standards and Represent a Literacy Barrier to Care

Online resources are a valuable source of information for patients and have been reported to improve engagement and adherence to medical care. However, readability of online patient educational materials (OPEMs) is crucial for them to serve their intended purpose. The American Medical Association (AMA) recommends that OPEM be written at or below the sixth grade reading level. To avoid disparities in access to comprehensible health information on peripheral artery disease (PAD), it is imperative that the readability of PAD OPEM is appropriate for both English-speaking and Spanish-speaking patients. The aim of this study is to evaluate the readability of PAD OPEM in Spanish and compare to English-language OPEM. We conducted a Google search in English and Spanish using "peripheral arterial disease" and "enfermedad arterial periferica", respectively, and the top 25 patient-accessible articles were collected for each. Articles were categorized by source type: hospital, professional society, or other. Readability of English-language OPEM was measured using the Flesch Reading Ease Readability Formula, Automated Readability Index, Coleman-Liau Index, Flesch-Kincaid Grade Level, Gunning Fog, Linsear Write Formula, and the Simple Measure of Gobbledygook Index. Readability of Spanish OPEM was measured using the Fernández-Huerta Index and Índice Flesch-Szigriszt Scale. Readability of the articles was compared to the AMA recommendation, between English- and Spanish-language, and across sources using statistical tests appropriate to the data. OPEM from professional societies represented the fewest number of English- (n=7, 28%) and Spanish-language (n=6, 24%) articles. Most English-speaking (n=18, 72%) and Spanish-language (n=20, 80%) OPEM were considered difficult as measured by the Flesch Reading Ease Readability Formula and Fernández-Huerta Index, respectively, but did not significantly differ between languages (P=0.59). There were no significant differences in the average readability of all readability measurements across sources (hospital, professional society, or other). All the average readability grade levels for English-speaking and Spanish-language OPEM was significantly higher than the sixth grade reading level (P<0.01). Only 3 (6%) OPEM met the AMA recommended reading level and there was no significant difference between English-language and Spanish-language OPEM (P=1.0). Nearly all Spanish-language and English-language PAD OPEM assessed were written at a reading grade level higher than recommended by the AMA. There was no significant difference in the readability of materials from hospitals or professional societies. To prevent further widening of health disparities related to literacy, health content creators, particularly hospitals and professional societies, should prioritize, develop, and ensure that English-language and Spanish-language patient education materials are written at a level appropriate for the public.

AI-Generated Information for Vascular Patients: Assessing the Standard of Procedure-Specific Information Provided by the ChatGPT AI-Language Model.

Introduction Ensuring access to high-quality information is paramount to facilitating informed surgical decision-making. The use of the internet to access health-related information is increasing, along with the growing prevalence of AI language models such as ChatGPT. We aim to assess the standard of AI-generated patient-facing information through a qualitative analysis of its readability and quality. Materials and methods We performed a retrospective qualitative analysis of information regarding three common vascular procedures: endovascular aortic repair (EVAR), endovenous laser ablation (EVLA), and femoro-popliteal bypass (FPBP). The ChatGPT responses were compared to patient information leaflets provided by the vascular charity, Circulation Foundation UK. Readability was assessed using four readability scores: the Flesch-Kincaid reading ease (FKRE) score, the Flesch-Kincaid grade level (FKGL), the Gunning fog score (GFS), and the simple measure of gobbledygook (SMOG) index. Quality was assessed using the DISCERN tool by two independent assessors. Results The mean FKRE score was 33.3, compared to 59.1 for the information provided by the Circulation Foundation (SD=14.5, p=0.025) indicating poor readability of AI-generated information. The FFKGL indicated that the expected grade of students likely to read and understand ChatGPT responses was consistently higher than compared to information leaflets at 12.7 vs. 9.4 (SD=1.9, p=0.002). Two metrics measure readability in terms of the number of years of education required to understand a piece of writing: the GFS and SMOG. Both scores indicated that AI-generated answers were less accessible. The GFS for ChatGPT-provided information was 16.7 years versus 12.8 years for the leaflets (SD=2.2, p=0.002) and the SMOG index scores were 12.2 and 9.4 years for ChatGPT and the patient information leaflets, respectively (SD=1.7, p=0.001). The DISCERN scores were consistently higher in human-generated patient information leaflets compared to AI-generated information across all procedures; the mean score for the information provided by ChatGPT was 50.3 vs. 56.0 for the Circulation Foundation information leaflets (SD=3.38, p<0.001). Conclusion We concluded that AI-generated information about vascular surgical procedures is currently poor in both the readability of text and the quality of information. Patients should be directed to reputable, human-generated information sources from trusted professional bodies to supplement direct education from the clinician during the pre-procedure consultation process.

Internet Health Resources on Nocturnal Enuresis: A Readability, Quality, and Accuracy Analysis.

Nocturnal enuresis is a common yet quality-of-life-limiting pediatric condition. There is an increasing trend for parents to obtain information on the disease's nature and treatment options via the internet. However, the quality of health-related information on the internet varies greatly and is largely uncontrolled and unregulated. With this study, a readability, quality, and accuracy evaluation of the health information regarding nocturnal enuresis is carried out. A questionnaire was administered to parents and patients with nocturnal enuresis to determine their use of the internet to research their condition. The most common search terms were determined, and the first 30 websites returned by the most popular search engines were used to assess the quality of information about nocturnal enuresis. Each site was categorized by type and assessed for readability using the Gunning fog score, Simple Measure of Gobbledygook (SMOG) index, and Dale-Chall score; for quality using the DISCERN score; and for accuracy by comparison to the International Children's Continence Society guidelines by three experienced pediatric urologists and nephrologists. A total of 30 websites were assessed and classified into five categories: professional (n = 13), nonprofit (n = 8), commercial (n = 4), government (n = 3), and other (n = 2). The information was considered difficult for the public to comprehend, with mean Gunning fog, SMOG index, and Dale-Chall scores of 12.1 ± 4.3, 14.1 ± 4.3, and 8.1 ± 1.3, respectively. The mean summed DISCERN score was 41 ± 11.6 out of 75. Only seven (23%) websites were considered of good quality (DISCERN score > 50). The mean accuracy score of the websites was 3.2 ± 0.6 out of 5. Commercial websites were of the poorest quality and accuracy. Websites generally scored well in providing their aims and identifying treatment benefits and options, while they lacked references and information regarding treatment risks and mechanisms. Online information about nocturnal enuresis exists for parents; however, most websites are of suboptimal quality, readability, and accuracy. Pediatric surgeons should be aware of parents' health-information-seeking behavior and be proactive in guiding parents to identify high-quality resources.

Readability of Patient-Reported Outcomes in Spine Surgery and Implications for Health Literacy.

Observational study. To evaluate the readability of commonly utilized patient-reported outcome measures (PROMs) in spine surgery. Although studies have evaluated patient education materials, discharge instructions, and informed consent forms in spine surgery, there is a dearth of literature on the readability of PROMs despite widespread health illiteracy. Without knowledge of PROM readability, it is unclear whether these measures are able to be understood by the average spine patient. We analyzed all commonly utilized nonvisual PROMs within the spinal literature and uploaded PROMs into an online readability calculator. The Flesch Reading Ease Score (FRES) and Simple Measure of Gobbledygook (SMOG) Index were collected. A FRES>79 or SMOG<7 was considered readable by the general population per American Medical Association and Centers for Disease Control guidelines. A stricter threshold recommended in health care (SMOG <6 or FRES>89) was then used to further review readability. Seventy-seven PROMs were included. Based on FRES, the mean readability of all PROMs was 69.2 ± 17.2 (range, 10-96.4), indicating an average eighth to ninth-grade reading level. The mean readability score categorized by the SMOG Index was 8.12 ± 2.65 (range, 3.1-25.6), representing an eighth-grade reading level. Compared with the reading level of the general population, 49 (63.6%) PROMs are written above the United States literacy level, according to FRES. Using stricter definitions of readability, 8 PROMs were considered readable, including PROM Information System pain behavior (FRES: 96.4 and SMOG: 5.2), PROM Information System sleep disturbance (SMOG: 5.6), Neck Pain and Disability Scale (SMOG: 4.3), and Zung Depression Scale (SMOG: 3.1). Most PROMs utilized in spine surgery require an average reading competency far above the average patient's comprehension. This may have a meaningful impact on understanding PROM instruments and may affect the accuracy of complete surveys and the rates of incompletion.

Web-based patient educational material on chimeric antigen receptor T-cell therapy: Quantitative assessment of readability.

11040 Background: Chimeric antigen receptor (CAR) T-cell therapy is another paradigm-shifting advancement for hematologic malignancies, creating unprecedented treatment options. However, there is still a lack of knowledge and understanding among patients regarding this novel therapy. Most patients turn to online resources, ranging from social media to federal websites, to gather health information to supplement decision-making. Given that the average American adult reads between the sixth- and eighth-grade levels, national organizations recommend that patient resources be written at the sixth-grade level or below. The purpose of this study was to evaluate web-based patient educational material on CAR -T cell therapy resources using measures of readability and compare it to national guidelines. Methods: Online patient information on CAR-T cell therapy from the top 20 U.S Cancer center as per US news ranking was collected. We analyzed the content by six of the most common readability tests- Flesch Reading Ease score (FRE), Gunning Fog (GF), Flesch-Kincaid Grade Level (FKGL), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG) Index, and the Automated Readability Index (ARI). The text from each article was carefully reviewed and analyzed using the software Readable, and the readability scores with standard deviation (SD) were obtained. Results: The mean score FRE score was 48.5 (SD: 6.7), which corresponds to college level and is difficult to read. The mean GF score was 8.7(SD: 1.5), which represents the writing of an 8th grader. The FKGL score was 8.6 (SD: 1.1), which represents a level of 8th grade or above. The mean CLI score was 11.5(SD: 1.3), which is the text level for high school juniors. The mean SMOG index was 10.9 (SD: 1), which is the level of a 12th grader. The ARI score was 7.8 (SD: 1.3), indicating a 7th-grade level. Conclusions: Overall, web-based CAR- T cell therapy patient educational materials scored poorly and does not meet the national recommendations. Authors of patient resources should incorporate readability criteria and prompt a revision or creation of new educational materials including videos and audio to support general patient understanding. By making CAR-T therapy information easily understandable, internet users can be better informed about their treatment decisions.

Pain management following total shoulder arthroplasty: what are patients reading online?

BackgroundThis study aims to assess the quality and readability of online resources regarding postoperative pain control for patients having undergone total shoulder arthroplasty (TSA), along with identifying gaps among the types of information. MethodsThree search engines were used to conduct internet searches utilizing a combination of TSA and pain management search phrases. The top 20 web pages from each search engine were gathered, reviewed, and scored by five different evaluators. Scoring was performed using 30-point criteria, integrated into an overall quality of resource (QR) score considered significant for patient education about TSA and pain management according to American Pain Society and American Academy of Orthopaedic Surgeons’ published recommendations (Table I). Domain scores (route of administration, type of analgesia, opioid-specific, and general guidance) were calculated by summing the total points and dividing by the total possible score for each domain. The reading level of the Flesch-Kinkaid grade level, the Flesch reading ease, and the simple measure of gobbledygook index was also assessed. ResultsThe overall readability was low, with a mean Flesch-Kincaid grade level score of 10.3 ± 2.0 and Simple Measure of Gobbledygook score of 9.9 ± 1.5. According to this index, only one website (2.3%) was written below the suggested 6th grade reading level. There was a mean QR score of 9.6/30 points, ranging from 2 to 22 (Table II). Most resources (90.9%) did not achieve greater than 50% of the total available points (greater than 15 out of 30). Of the domains, the mean score for route of administration was 31.8%, types of analgesia was 43.6%, opioid-specific was 11.4%, and general guidance was 43.2%. Opioid-specific scores were lower than all three other domain scores (all P < .001). ConclusionAccording to our findings, current online resources for pain management following TSA are written above the average literacy level for adults in the United States and lack educational content, especially regarding opioid-specific discussion and recommendations. Resources must target gaps in online patient education to better inform patients of postoperative pain control, convey expectations, and encourage patient-physician conversations and decision-making.

Readability of Online Patient Education Materials for Congenital Hand Differences.

The aim of the study is to assess the readability of online patient education materials (PEMs) for congenital hand differences. The top 10 online, English-language PEMs for 10 conditions (polydactyly, syndactyly, trigger finger/thumb, clinodactyly, camptodactyly, symbrachydactyly, thumb hypoplasia, radial dysplasia, reduction defect, and amniotic band syndrome) were compiled and categorized by source and country. Readability was assessed using 5 tools: Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), and Simple Measure of Gobbledygook Index (SMOG). To account for the potential effect of each condition's name in the aforementioned formulas, the analysis was repeated after replacing the name with a monosyllabic word/s. The mean readability scores of the 100 PEMs were FRES 56.3, where the target was ≥80, FKGL 8.8, GFI 11.5, CLI 10.9, and SMOG 8.6, and the median grade score was 9.8, where the target grade was ≤6.9. Following adjustment, all readability scores improved significantly (P < .001). Postadjustment scores were FRES 63.8, FKGL 7.8, GFI 10.7, CLI 9.1, and SMOG 8.0, and the median grade score was 8.6. Only 1 webpage met the target level using all tools. Two-sample t test for country of publication (the United States and the United Kingdom) demonstrated that PEMs originating from the United Kingdom were easier to read using the preadjustment CLI (P = .009) and median grade metrics (P = .048). A 1-way analysis of variance revealed no influence of condition or source on readability. Most online PEMs for congenital hand differences are written above the recommended reading level of sixth grade, even when adjusted for the effect of the condition's name.