Fog Index Research Articles

Assessing the Responses of Large Language Models (ChatGPT-4, Claude 3, Gemini, and Microsoft Copilot) to Frequently Asked Questions in Retinopathy of Prematurity: A Study on Readability and Appropriateness.

To assess the appropriateness and readability of responses provided by four large language models (LLMs) (ChatGPT-4, Claude 3, Gemini, and Microsoft Co-pilot) to parents' queries pertaining to retinopathy of prematurity (ROP). A total of 60 frequently asked questions were collated and categorized into six distinct sections. The responses generated by the LLMs were evaluated by three experienced ROP specialists to determine their appropriateness and comprehensiveness. Additionally, the readability of the responses was assessed using a range of metrics, including the Flesch-Kincaid Grade Level (FKGL), Gunning Fog (GF) Index, Coleman-Liau (CL) Index, Simple Measure of Gobbledygook (SMOG) Index, and Flesch Reading Ease (FRE) score. ChatGPT-4 demonstrated the highest level of appropriateness (100%) and performed exceptionally well in the Likert analysis, scoring 5 points on 96% of questions. The CL Index and FRE scores identified Gemini as the most readable LLM, whereas the GF Index and SMOG Index rated Microsoft Copilot as the most readable. Nevertheless, ChatGPT-4 exhibited the most intricate text structure, with scores of 18.56 on the GF Index, 18.56 on the CL Index, 17.2 on the SMOG Index, and 9.45 on the FRE score. This suggests that the responses demand a college-level comprehension. ChatGPT-4 demonstrated higher performance than other LLMs in responding to questions related to ROP; however, its texts were more complex. In terms of readability, Gemini and Microsoft Copilot were found to be more successful. [J Pediatr Ophthalmol Strabismus. 20XX;XX(X):XXX-XXX.].

Assessing the readability and quality of online written information on epistaxis.

The objective of this study was to assess the readability and quality of online written information on epistaxis. The terms 'epistaxis' and 'nosebleed' were entered into Google. The first six webpages generated for each search term were screened. Readability was assessed using the Flesch-Kincaid Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG) Index and Gunning Fog Index (GFOG). Quality was assessed using the DISCERN instrument. Spearman's correlation between quality and readability was calculated. A total of 37 websites met the inclusion criteria. The mean and 95% confidence intervals for FRES, FKGL, SMOG and GFOG were 58.9 (55.3-62.5), 9.65 (8.74-10.6), 9.18 (8.57-9.8) and 12.5 (11.5-13.5), respectively. The DISCERN score was 34.3 (32.0-36.5). Weak negative correlation was noted between DISCERN and FRES (rs = -0.15, p = 0.36). Online information on epistaxis is generally of poor quality and low readability.

Improving Biomedical Science Literacy and Patient-Directed Knowledge of Tuberculosis (TB): A Cross-Sectional Infodemiology Study Examining Readability of Patient-Facing TB Information.

Tuberculosis (TB) continues be the leading cause of death globally due to an infectious agent. There is a paucity of data describing the readability of patient-facing TB information for service users. The aim of this study was to calculate the readability of multiple global TB information sources. Information on tuberculosis (n = 150 sources) included nine categories, Patient-facing information: WHO publications (n = 17), International governments (n = 19), Hospitals (n = 10), Non-government organisations (NGOs)/charities (n = 20), Cochrane Plain Language Summaries (n = 20); LabTestsOnlineUK (n = 4) and Scientific-facing information: Clinical trials (n = 20), Cochrane abstracts (n = 20), Scientific abstracts (n = 20). Readability was calculated using Readable software, defined by (i) Flesch Reading Ease (FRE), (ii) Flesch-Kincaid Grade Level (FKGL), (iii) Gunning Fog Index and (iv) SMOG Index and two text metrics [words/sentence, syllables/word]. Mean readability values for TB information for the FRE and FKGL were 35.6 ± 1.6 (standard error of mean (SEM)) (US Target ≥60; UK Target ≥90) and 12.3 ± 0.3 (US Target ≤8; UK Target ≤6), respectively, with mean words per sentence and syllables per word of 17.2 and 1.8, respectively. Cochrane Plain Language Summaries had similar readability scores to their matching scientific abstract (p = 0.15). LabTestsOnlineUK yielded a mean FRE score of 51.5 ± 1.2, a mean FKGL score of 10.2 ± 0.5 and text metric scores of 16.7 ± 2.3 and 1.6, for words per sentence and syllables per word, respectively. In descending order, TB information from international governments, hospitals and LabTestsOnlineUK were the most readable (FRE = 57.9, 54.1 and 51.5, respectively), whereas scientific abstracts and Cochrane abstracts were the most difficult to read (13.0 and 30.2, respectively). Patient-facing TB information analysed had poor readability. Effective communication of biomedical science concepts and information relating to TB is vital for service users to enhance their health literacy of tuberculosis, thereby promoting better clinical outcomes. Biomedical scientists are important custodians of scientific information for their service user populations, including other healthcare professionals within the TB multidisciplinary (MDT) team and patient service users. When preparing TB information, this should be checked and modified in real time employing readability calculators, to align with health readability targets.

Development and Validation of an Artificial Intelligence-Assisted Patient Education Material for Ostomy Patients: A Methodological Study.

To develop and test the validity of an artificial intelligence-assisted patient education material for ostomy patients. A methodological study. The study was carried out in two main stages and five steps: (1) determining the information needs of ostomy patients, (2) creating educational content, (3) converting the educational content into patient education material, (4) validation of patient education material based on expert review and (5) measuring the readability of the patient education material. We used ChatGPT 4.0 to determine the information needs and create patient education material content, and Publuu Online Flipbook Maker was used to convert the educational content into patient education material. Understandability and applicability scores were assessed using the Patient Education Materials Assessment Tool submitted to 10 expert reviews. The tool inter-rater reliability was determined via the intraclass correlation coefficient. Readability was analysed using the Flesch-Kincaid Grade Level, Gunning Fog Index and Simple Measure of Gobbledygook formula. The mean Patient Education Materials Assessment Tool understandability score of the patient education material was 81.91%, and the mean Patient Education Materials Assessment Tool actionability score was 85.33%. The scores for the readability indicators were calculated to be Flesch-Kincaid Grade Level: 8.53, Gunning Fog: 10.9 and Simple Measure of Gobbledygook: 7.99. The AI-assisted patient education material for ostomy patients provided accurate information with understandable and actionable responses to patients, but is at a high reading level for patients. Artificial intelligence-assisted patient education materials can significantly increase patient information rates in the health system regarding ease of practice. Artificial intelligence is currently not an option for creating patient education material, and their impact on the patient is not fully known. The study followed the STROBE checklist guidelines. No patient or public contributions.

Appropriateness and readability of Google Bard and ChatGPT-3.5 generated responses for surgical treatment of glaucoma.

To evaluate the appropriateness and readability of the medical knowledge provided by ChatGPT-3.5 and Google Bard, artificial-intelligence-powered conversational search engines, regarding surgical treatment for glaucoma. In this retrospective, cross-sectional study, 25 common questions related to the surgical management of glaucoma were asked on ChatGPT-3.5 and Google Bard. Glaucoma specialists graded the responses' appropriateness, and different scores assessed readability. Appropriate answers to the posed questions were obtained in 68% of the responses with Google Bard and 96% with ChatGPT-3.5. On average, the responses generated by Google Bard had a significantly lower proportion of sentences, having more than 30 and 20 syllables (23% and 52% respectively) compared to ChatGPT-3.5 (66% and 82% respectively), as noted by readability. Google Bard had significantly (p<0.0001) lower readability grade scores and significantly higher "Flesch Reading ease score", implying greater ease of readability amongst the answers generated by Google Bard. Many patients and their families turn to LLM chatbots for information, necessitating clear and accurate content. Assessments of online glaucoma information have shown variability in quality and readability, with institutional websites generally performing better than private ones. We found that ChatGPT-3.5, while precise, has lower readability than Google Bard, which is more accessible but less precise. For example, the Flesch Reading Ease Score was 57.6 for Google Bard and 22.6 for ChatGPT, indicating Google Bard's content is easier to read. Moreover, the Gunning Fog Index scores suggested that Google Bard's text is more suitable for a broader audience. ChatGPT's knowledge is limited to data up to 2021, whereas Google Bard, trained with real-time data, offers more current information. Further research is needed to evaluate these tools across various medical topics. The answers generated by ChatGPT-3.5™ AI are more accurate than the ones given by Google Bard. However, comprehension of ChatGPT-3.5™ answers may be difficult for the public with glaucoma. This study emphasized the importance of verifying the accuracy and clarity of online information that glaucoma patients rely on to make informed decisions about their ocular health. This is an exciting new area for patient education and health literacy.

Evaluating AI-Generated informed consent documents in oral surgery: A comparative study of ChatGPT-4, Bard gemini advanced, and human-written consents

This study evaluates the quality and readability of informed consent documents generated by AI platforms ChatGPT-4 and Bard Gemini Advanced compared to those written by a first-year oral surgery resident for common oral surgery procedures. The evaluation, conducted by 18 experienced oral and maxillofacial surgeons, assessed consents for accuracy, completeness, readability, and overall quality.ChatGPT-4 consistently outperformed both Bard and human-written consents. ChatGPT-4 consents had a median accuracy score of 4 [IQR 4-4], compared to Bard's 3 [IQR 3–4] and human's 4 [IQR 3–4]. Completeness scores were higher for ChatGPT-4 (4 [IQR 4–5]) than Bard (3 [IQR 3–4]) and human (4 [IQR 3–4]). Readability was also superior for ChatGPT-4, with a median score of 4 [IQR 4–5] compared to Bard and human consents, both at 4 [IQR 4-4] and 4 [IQR 3–4], respectively. The Gunning Fog Index for ChatGPT-4 was 17.2 [IQR 16.5–18.2], better than Bard's 23.1 [IQR 20.5–24.7] and the human consents' 20 [IQR 19.2–20.9].Overall, ChatGPT-4's consents received the highest quality ratings, underscoring AI's potential in enhancing patient communication and the informed consent process. The study suggests AI can reduce misinformation risks and improve patient understanding, but continuous evaluation, oversight, and patient feedback integration are crucial to ensure the effectiveness and appropriateness of AI-generated content in clinical practice.

Abstract B105: Quality and Accessibility of Online Information Regarding CAR-T Therapy in Hematologic Malignancies

Abstract Background Chimeric antigen receptor T-cell (CAR-T) therapies have changed the treatment landscape for various hematologic malignancies and are currently under development and being tested in clinical trials for multiple solid tumors. Given that patients commonly use the internet as a source of health-related information, we sought to investigate the quality, understandability, and readability of online information regarding CAR-T therapy. Methods We conducted a search in a commonly used search engine (Google) of five pre-determined search terms to identify 100 unique web pages regarding CAR-T therapy. Scientific publications were excluded. Validated instruments were used to ascertain health information quality (DISCERN), understandability (Patient Education Materials Assessment Tool; PEMAT), and readability (Flesch-Kincaid, Gunning Fog, and SMOG indices). Results were summarized using descriptive statistics. Results Among the 100 web pages reviewed, 68% originated from academic medical centers, and 64% included links to clinical trials. Only 11% of the web pages included references to primary literature. 17% of web pages were available in a language other than English, with Spanish (15%) and Arabic (7%) being the most common. Moderate-to-high quality information (DISCERN ≥ 3) was presented by 47% of web pages, and only 31% of web pages were considered understandable (PEMAT ≥ 70%). Average readability scores were 10.6 (SD = 2.09), 12.6 (SD = 2.3), and 13.1 (SD = 1.7) using the Flesch-Kincaid, Gunning Fog, and SMOG indices, respectively. Conclusion Our study highlights that online information for CAR-T therapies commonly features low-quality health information that is typically not understandable to the general population. Further, contrary to recommendations from federal agencies and medical societies, available online information requires, on average, high school reading comprehension. Healthcare teams and stakeholders should strive to improve existing content and ensure patients have access to trusted, high-quality information given the unique complexities, processes, and toxicities associated with receiving CAR-T therapy. Citation Format: Luis C. Garcia, Ana I. Velázquez. Quality and Accessibility of Online Information Regarding CAR-T Therapy in Hematologic Malignancies [abstract]. In: Proceedings of the 17th AACR Conference on the Science of Cancer Health Disparities in Racial/Ethnic Minorities and the Medically Underserved; 2024 Sep 21-24; Los Angeles, CA. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2024;33(9 Suppl):Abstract nr B105.

Pillar 3: The impact of language complexity on the preferences of commercial bank website users

This research explores the impact of document complexity and readability on user preferences for disclosure information on commercial bank web portals, with a focus on Pillar 3 disclosures. The study investigates the usage patterns of disclosed information by stakeholders. Through an analysis of web portal access variables and document complexity and readability measures, the study identifies correlations between text complexity/readability and user preferences. The experiment used traditional readability measures like the Gunning Fog Index and Flesch Reading Ease but also focused on less-used metrics such as LIX, various text characteristics, and part-of-speech metrics. The findings reveal that while stakeholders show interest in financial documents despite their complexity, preferences in using these documents vary based on document type and readability. Notably, documents with higher complexity tend to attract more attention, suggesting potential challenges in accessibility and comprehension. The paper highlights the importance of presenting disclosure information in a manner that enhances readability and accessibility to better engage stakeholders. Additionally, the research introduces user preference indicators aligned with web portal access metrics, providing insights into stakeholders’ behavior. The research followed up on previous findings of extremely low interest in Pillar 3 information from commercial bank stakeholders. The results showed that stakeholders are more interested in less demanding and more readable texts such as annual reports, and as Pillar 3 documents are more complex and harder to read, they are less interested in them. The research concludes that stakeholders’ increasing interest in this information requires finding approaches to presenting it in less complicated and more readable ways.

Evaluación de la fiabilidad y legibilidad de las respuestas de los chatbots como recurso de información al paciente para las exploraciones PET-TC más communes

PurposeThis study aimed to evaluate the reliability and readability of responses generated by two popular AI-chatbots, ‘ChatGPT-4.0’ and ‘Google Gemini’, to potential patient questions about PET/CT scans. Materials and methodsThirty potential questions for each of [18F]FDG and [68Ga]Ga-DOTA-SSTR PET/CT, and twenty-nine potential questions for [68Ga]Ga-PSMA PET/CT were asked separately to ChatGPT-4 and Gemini in May 2024. The responses were evaluated for reliability and readability using the modified DISCERN (mDISCERN) scale, Flesch Reading Ease (FRE), Gunning Fog Index (GFI), and Flesch-Kincaid Reading Grade Level (FKRGL). The inter-rater reliability of mDISCERN scores provided by three raters (ChatGPT-4, Gemini, and a nuclear medicine physician) for the responses was assessed. ResultsThe median [min-max] mDISCERN scores reviewed by the physician for responses about FDG, PSMA and DOTA PET/CT scans were 3.5 [2–4], 3 [3–4], 3 [3–4] for ChatPT-4 and 4 [2–5], 4 [2–5], 3.5 [3–5] for Gemini, respectively. The mDISCERN scores assessed using ChatGPT-4 for answers about FDG, PSMA, and DOTA-SSTR PET/CT scans were 3.5 [3–5], 3 [3–4], 3 [2–3] for ChatGPT-4, and 4 [3–5], 4 [3–5], 4 [3–5] for Gemini, respectively. The mDISCERN scores evaluated using Gemini for responses FDG, PSMA, and DOTA-SSTR PET/CTs were 3 [2–4], 2 [2–4], 3 [2–4] for ChatGPT-4, and 3 [2–5], 3 [1–5], 3 [2–5] for Gemini, respectively. The inter-rater reliability correlation coefficient of mDISCERN scores for ChatGPT-4 responses about FDG, PSMA, and DOTA-SSTR PET/CT scans were 0.629 (95% CI = 0,32−0,812), 0.707 (95% CI = 0.458−0.853) and 0.738 (95% CI = 0.519−0.866), respectively (p < 0.001). The correlation coefficient of mDISCERN scores for Gemini responses about FDG, PSMA, and DOTA-SSTR PET/CT scans were 0.824 (95% CI = 0.677−0.910), 0.881 (95% CI = 0.78−0.94) and 0.847 (95% CI = 0.719−0.922), respectively (p < 0.001). The mDISCERN scores assessed by ChatGPT-4, Gemini, and the physician showed that the chatbots' responses about all PET/CT scans had moderate to good statistical agreement according to the inter-rater reliability correlation coefficient (p < 0,001). There was a statistically significant difference in all readability scores (FKRGL, GFI, and FRE) of ChatGPT-4 and Gemini responses about PET/CT scans (p < 0,001). Gemini responses were shorter and had better readability scores than ChatGPT-4 responses. ConclusionThere was an acceptable level of agreement between raters for the mDISCERN score, indicating agreement with the overall reliability of the responses. However, the information provided by AI-chatbots cannot be easily read by the public.

Evaluation of the Appropriateness and Readability of ChatGPT-4 Responses to Patient Queries on Uveitis

To compare the utility of ChatGPT-4 as an online uveitis patient education resource with existing patient education websites. Evaluation of technology. Not applicable. The term "uveitis" was entered into the Google search engine, and the first 8 nonsponsored websites were selected to be enrolled in the study. Information regarding uveitis for patients was extracted from Healthline, Mayo Clinic, WebMD, National Eye Institute, Ocular Uveitis and Immunology Foundation, American Academy of Ophthalmology, Cleveland Clinic, and National Health Service websites. ChatGPT-4 was then prompted to generate responses about uveitis in both standard and simplified formats. To generate the simplified response, the following request was added to the prompt: 'Please provide a response suitable for the average American adult, at a sixth-grade comprehension level.' Three dual fellowship-trained specialists, all masked to the sources, graded the appropriateness of the contents (extracted from the existing websites) and responses (generated responses by ChatGPT-4) in terms of personal preference, comprehensiveness, and accuracy. Additionally, 5 readability indices, including Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook index were calculated using an online calculator, Readable.com, to assess the ease of comprehension of each answer. Personal preference, accuracy, comprehensiveness, and readability of contents and responses about uveitis. A total of 497 contents and responses, including 71 contents from existing websites, 213 standard responses, and 213 simplified responses from ChatGPT-4 were recorded and graded. Standard ChatGPT-4 responses were preferred and perceived to be more comprehensive by dually trained (uveitis and retina) specialist ophthalmologists while maintaining similar accuracy level compared with existing websites. Moreover, simplified ChatGPT-4 responses matched almost all existing websites in terms of personal preference, accuracy, and comprehensiveness. Notably, almost all readability indices suggested that standard ChatGPT-4 responses demand a higher educational level for comprehension, whereas simplified responses required lower level of education compared with the existing websites. This study shows that ChatGPT can provide patients with an avenue to access comprehensive and accurate information about uveitis, tailored to their educational level. The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Readability and Accessibility of Patient-Education Materials for Heart Failure in the United States

Online education materials are widely used by patients and caregivers to understand the management of complex chronic diseases such as heart failure (HF). Organizations such as the American Medical Association and National Institutes of Health recommend that materials be written at a 6th grade reading level. The current study examined the readability and accessibility of online education materials for patients with HF. Whole page text from each included website was entered into an online readability calculator. Five validated readability indices (Flesch-Kincaid Grade Level, Flesch Reading Ease Scale, Gunning Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook (SMOG Index)) were used to evaluate each source. Websites were categorized by source (government, public, and private). The availability of audiovisual accessibility features and content in non-English languages were assessed for each website. Of the 36 online resources analyzed, the median readability level was 9-10th grade by the Flesch-Kincaid Grade Level and college level using the Flesch Reading Ease Scale. The Gunning Fog Index and Coleman-Liau Index both showed median readability scores corresponding to a 12th grade reading level, while the SMOG Index showed a median score corresponding to that of the 9th grade. Only 10 websites (28%) offered information in languages other than English, and none provided comprehensive accessibility features for users with disabilities. Common online educational materials for patients with HF are characterized by a higher readability level than that recommended by the National Institutes of Health and American Medical Association with limited multilingual and accessibility options, potentially limiting the accessibility of resources to patients and caregivers.

Readability of Patient-Facing Information of Antibiotics Used in the WHO Short 6-Month and 9-Month All Oral Treatment for Drug-Resistant Tuberculosis

ObjectivesReadability of patient-facing information of oral antibiotics detailed in the WHO all oral short (6 months, 9 months) has not been described to date. The aim of this study was therefore to examine (i) how readable patient-facing TB antibiotic information is compared to readability reference standards and (ii) if there are differences in readability between high-incidence countries versus low-incidence countries.MethodsTen antibiotics, including bedaquiline, clofazimine, ethambutol, ethionamide, isoniazid, levofloxacin, linezolid, moxifloxacin, pretomanid, pyrazinamide, were investigated. TB antibiotic information sources were examined, consisting of 85 Patient Information Leaflets (PILs) and 40 antibiotic web resouces. Of these 85 PILs, 72 were taken from the National Medicines Regulator from six countries (3 TB high-incidence [Rwanda, Malaysia, South Africa] + 3 TB low-incidence [UK, Ireland, Malta] countries). Readability data was grouped into three categories, including (i) high TB-incidence countries (n = 33 information sources), (ii) low TB-incidence countries (n = 39 information sources) and (iii) web information (n = 53). Readability was calculated using Readable software, to obtain four readability scores [(i) Flesch Reading Ease (FRE), (ii) Flesch-Kincaid Grade Level (FKGL), (iii) Gunning Fog Index and (iv) SMOG Index], as well as two text metrics [words/sentence, syllables/word].ResultsMean readability scores of patient-facing TB antibiotic information for FRE and FKGL, were 47.4 ± 12.6 (sd) (target ≥ 60) and 9.2 ± 2.0 (target ≤ 8.0), respectively. There was no significant difference in readability between low incidence countries and web resources, but there was significantly poorer readability associated with PILs from high incidence countries versus low incidence countries (FRE; p = 0.0056: FKGL; p = 0.0095).ConclusionsReadability of TB antibiotic PILs is poor. Improving readability of PILs should be an important objective when preparing patient-facing written materials, thereby improving patient health/treatment literacy.

Lessons to be learned when designing comprehensible patient-oriented online information about temporomandibular disorders.

Temporomandibular disorders (TMD) are a prevalent ailment with a global impact, affecting a substantial number of individuals. While some individuals are receiving treatment from orthodontists for TMD, a significant proportion of individuals obtain knowledge through websites. Our purpose had been to evaluate, from a patient-oriented perspective, the readability of home pages of websites scored in the 10 most prominent devoted to TMD. We also determined what level of education would have been needed to get an overview of the information on the websites under scrutiny. This approach ensures that our findings are centred on the patient experience, providing insights into how accessible and understandable websites about TMD. We determined the top 10 patient-focused English language websites by searching for 'temporomandibular disorders' in the 'no country redirect' plugin of the Google Chrome browser (www.google.com/ncr). The readability of the texts was assessed using the Gunning fog index (GFI), Coleman Liau index (CLI), Automated readability index (ARI) Simple Measure of Gobbledygook (SMOG), Flesch Kincald grade level (FKGL), Flesh reasing ease (FRE) (https://readabilityformulas.com). The mean Flesch reading ease index score was determined to be 48.67, accompanied by a standard deviation of 15.04 and these websites require an average of 13.49 years of formal education (GFI), with a standard deviation of 2.62, for ease of understanding. Our research indicates that a significant proportion of websites related to TMD can be defined as a level of complexity that exceeds the ability to read comprehension of the general population.

Assessing readability of online patient educational material on concussion and return to play.

Concussions are self-limited forms of mild traumatic brain injury (TBI). Gradual return to play (RTP) is crucial to minimizing the risk of second impact syndrome. Online patient educational materials (OPEM) are often used to guide decision-making. Previous literature has reported that grade-level readability of OPEM is higher than recommended by the American Medical Association and the National Institutes of Health. The authors evaluated the readability of OPEM on concussion and RTP. An online search engine was used to identify websites providing OPEM on concussion and RTP. Text specific to concussion and RTP was extracted from each website and readability was assessed using the following six standardized indices: Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level, Gunning Fog Index, Coleman-Liau Index, Simple Measure of Gobbledygook Index, and Automated Readability Index. One-way ANOVA and Tukey's post hoc test were used to compare readability across sources of information. There were 59 concussion and RTP articles, and readability levels exceeded the recommended 6th grade level, irrespective of the source of information. Academic institutions published OPEM at simpler readability levels (higher FRE scores). Private organizations published OPEM at more complex (higher) grade-level readability levels in comparison with academic and nonprofit institutions (p < 0.05). The readability of OPEM on RTP after concussions exceeds the literacy of the average American. There is a critical need to modify the concussion and RTP OPEM to improve comprehension by a broad audience.

Assessing the Readability of Patient Education Materials on Cardiac Catheterization From Artificial Intelligence Chatbots: An Observational Cross-Sectional Study.

Artificial intelligence (AI) is a burgeoning new field that has increased in popularity over the past couple of years, coinciding with the public release of large language model (LLM)-driven chatbots. These chatbots, such as ChatGPT, can be engaged directly in conversation, allowing users to ask them questions or issue other commands. Since LLMs are trained on large amounts of text data, they can also answer questions reliably and factually, an ability that has allowed them to serve as a source for medical inquiries. This study seeks to assess the readability of patient education materials on cardiac catheterization across four of the most common chatbots: ChatGPT, Microsoft Copilot, Google Gemini, and Meta AI. A set of 10 questions regarding cardiac catheterization was developed using website-based patient education materials on the topic. We then asked these questions in consecutive order tofour of the most common chatbots: ChatGPT, Microsoft Copilot, Google Gemini, and Meta AI. The Flesch Reading Ease Score (FRES) was used to assess the readability score. Readability grade levels were assessed using six tools: Flesch-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG) Index, Automated Readability Index (ARI), and FORCAST Grade Level. The mean FRES across all four chatbots was 40.2, while overall mean grade levels for the four chatbots were 11.2, 13.7, 13.7, 13.3, 11.2, and 11.6 across the FKGL, GFI, CLI, SMOG, ARI, and FORCAST indices, respectively. Mean reading grade levels across the six tools were 14.8 for ChatGPT, 12.3 for Microsoft Copilot, 13.1 for Google Gemini, and 9.6 for Meta AI. Further, FRES values for the four chatbots were 31, 35.8, 36.4, and 57.7, respectively. This study shows that AI chatbots are capable of providing answers to medical questions regarding cardiac catheterization. However, the responses across the four chatbots had overall mean reading grade levels at the 11th-13th-grade level, depending on the tool used. This means that the materials were at the high school and even college reading level, which far exceeds the recommended sixth-grade level for patient education materials. Further, there is significant variability in the readability levels provided by different chatbots as, across all six grade-level assessments, Meta AI had the lowest scores andChatGPT generally had the highest.

Does Political Uncertainty Obfuscate Narrative Disclosure?

ABSTRACT We examine the relation between political uncertainty and narrative disclosure complexity in conference calls. Using firm-level political uncertainty, we find that political uncertainty is positively associated with firms’ disclosures complexity as measured by the Fog index. Decomposing complexity into two latent components—information and obfuscation—we show that political uncertainty significantly increases the obfuscation but has no impact on the information. Further analysis reveals that complex disclosure is motivated by expected poor future performance amid political uncertainty. We also show that, during periods of heightened political uncertainty, obfuscated disclosure is associated with reduced earnings informativeness, increased dispersion in analyst forecasts, and higher volatility in forecast revisions. These findings are robust to including and excluding sentences containing complex political bigrams when calculating Fog. Further evidence shows that, during periods of political uncertainty, managers tend to use a more ambiguous tone and provide scripted and shorter (longer) responses to analysts’ questions (presentations).

Comparing patient education tools for chronic pain medications: Artificial intelligence chatbot versus traditional patient information leaflets.

Artificial intelligence (AI) chatbots like Conversational Generative Pre-trained Transformer (ChatGPT) have recently created much buzz, especially regarding patient education. Such informed patients understand and adhere to the management and get involved in shared decision making. The accuracy and understandability of the generated educational material are prime concerns. Thus, we compared ChatGPT with traditional patient information leaflets (PILs) about chronic pain medications. Patients' frequently asked questions were generated from PILs available on the official websites of the British Pain Society (BPS) and the Faculty of Pain Medicine. Eight blinded annexures were prepared for evaluation, consisting of traditional PILs from the BPS and AI-generated patient information materials structured similar to PILs by ChatGPT. The authors performed a comparative analysis to assess materials' readability, emotional tone, accuracy, actionability, and understandability. Readability was measured using Flesch Reading Ease (FRE), Gunning Fog Index (GFI), and Flesch-Kincaid Grade Level (FKGL). Sentiment analysis determined emotional tone. An expert panel evaluated accuracy and completeness. Actionability and understandability were assessed with the Patient Education Materials Assessment Tool. Traditional PILs generally exhibited higher readability (P values < 0.05), with [mean (standard deviation)] FRE [62.25 (1.6) versus 48 (3.7)], GFI [11.85 (0.9) versus 13.65 (0.7)], and FKGL [8.33 (0.5) versus 10.23 (0.5)] but varied emotional tones, often negative, compared to more positive sentiments in ChatGPT-generated texts. Accuracy and completeness did not significantly differ between the two. Actionability and understandability scores were comparable. While AI chatbots offer efficient information delivery, ensuring accuracy and readability, patient-centeredness remains crucial. It is imperative to balance innovation with evidence-based practice.

Enhancing Readability of Online Patient-Facing Content: The Role of AI Chatbots in Improving Cancer Information Accessibility.

Internet-based health education is increasingly vital in patient care. However, the readability of online information often exceeds the average reading level of the US population, limiting accessibility and comprehension. This study investigates the use of chatbot artificial intelligence to improve the readability of cancer-related patient-facing content. We used ChatGPT 4.0 to rewrite content about breast, colon, lung, prostate, and pancreas cancer across 34 websites associated with NCCN Member Institutions. Readability was analyzed using Fry Readability Score, Flesch-Kincaid Grade Level, Gunning Fog Index, and Simple Measure of Gobbledygook. The primary outcome was the mean readability score for the original and artificial intelligence (AI)-generated content. As secondary outcomes, we assessed the accuracy, similarity, and quality using F1 scores, cosine similarity scores, and section 2 of the DISCERN instrument, respectively. The mean readability level across the 34 websites was equivalent to a university freshman level (grade 13±1.5). However, after ChatGPT's intervention, the AI-generated outputs had a mean readability score equivalent to a high school freshman education level (grade 9±0.8). The overall F1 score for the rewritten content was 0.87, the precision score was 0.934, and the recall score was 0.814. Compared with their original counterparts, the AI-rewritten content had a cosine similarity score of 0.915 (95% CI, 0.908-0.922). The improved readability was attributed to simpler words and shorter sentences. The mean DISCERN score of the random sample of AI-generated content was equivalent to "good" (28.5±5), with no significant differences compared with their original counterparts. Our study demonstrates the potential of AI chatbots to improve the readability of patient-facing content while maintaining content quality. The decrease in requisite literacy after AI revision emphasizes the potential of this technology to reduce health care disparities caused by a mismatch between educational resources available to a patient and their health literacy.

Quality, Reliability, Readability, and Accountability of Online Information on Leukocoria.

To evaluate the quality, reliability, and readability of online patient educational materials on leukocoria. In this cross-sectional study, the Google search engine was searched for the terms "leukocoria" and "white pupil." The first 50 search outcomes were evaluated for each search term based on predefined inclusion criteria, excluding duplicates, peer-reviewed papers, forum posts, paywalled content, and multimedia links. Sources were categorized as "institutional" or "private." Three independent raters assessed each website for quality and reliability using DISCERN, Health on the Net Code of Conduct (HONcode), and JAMA criteria. Readability was evaluated using seven formulas: Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG) Index, Automated Readability Index (ARI), Linsear Write (LW), Gunning Fog Index (GFI), and Coleman-Liau Index (CLI). A total of 51 websites were included. Quality, assessed by the DISCERN tool, showed a median score of 4, denoting moderate to high quality, with no significant differences between institutional and private sites or search terms. HONcode scores indicated variable reliability and trustworthiness (median: 10, range: 3 to 16), with institutional sites excelling in financial disclosure and ad differentiation. Additionally, institutional and private sites performed well in reliability and accountability, as measured by the JAMA Benchmark criteria (median: 3; range: 1 to 4). Readability, averaging an 11.3 ± 3.7 grade level, did not differ significantly between site types or search terms, consistently falling short of the recommended sixth-grade level for patient educational materials. The patient educational materials on leukocoria demonstrated moderate to high quality, commendable reliability, and accountability. However, the readability scores were above the recommended level for the layperson. [J Pediatr Ophthalmol Strabismus. 20XX;X(X):XX-XX.].

A Cross-Sectional Analysis of the Readability of Online Information Regarding Hip Osteoarthritis.

Introduction Osteoarthritis (OA) is an age-related degenerative joint disease. There is a 25% risk of symptomatic hip OA in patients who live up to 85 years of age. It can impair a person's daily activities and increase their reliance on healthcare services. It is primarily managed with education, weight loss and exercise, supplemented with pharmacological interventions. Poor health literacy is associated with negative treatment outcomes and patient dissatisfaction. A literature search found there are no previously published studies examining the readability of online information about hip OA. Objectives To assess the readability of healthcare websites regarding hip OA. Methods The terms "hip pain", "hip osteoarthritis", "hip arthritis", and "hip OA" were searched on Google and Bing. Of 240 websites initially considered, 74 unique websites underwent evaluation using the WebFX online readability software (WebFX®, Harrisburg, USA). Readability was determined using the Flesch Reading Ease Score (FRES), Flesch-Kincaid Reading Grade Level (FKGL), Gunning Fog Index (GFI), Simple Measure of Gobbledygook (SMOG), Coleman-Liau Index (CLI), and Automated Readability Index (ARI). In line with recommended guidelines and previous studies, FRES >65 or a grade level score of sixth grade and under was considered acceptable. Results The average FRES was 56.74±8.18 (range 29.5-79.4). Only nine (12.16%) websites had a FRES score >65. The average FKGL score was 7.62±1.69 (range 4.2-12.9). Only seven (9.46%) websites were written at or below a sixth-grade level according to the FKGL score. The average GFI score was 9.20±2.09 (range 5.6-16.5). Only one (1.35%) website was written at or below a sixth-grade level according to the GFI score.The average SMOG score was 7.29±1.41 (range 5.4-12.0). Only eight (10.81%) websites were written at or below a sixth-grade level according to the SMOG score. The average CLI score was 13.86±1.75 (range 9.6-19.7). All 36 websites were written above a sixth-grade level according to the CLI score. The average ARI score was 6.91±2.06 (range 3.1-14.0). Twenty-eight(37.84%) websites were written at or below a sixth-grade level according to the ARI score. One-sample t-tests showed that FRES (p<0.001, CI -10.2 to -6.37), FKGL (p<0.001, CI 1.23 to 2.01), GFI (p<0.001, CI 2.72 to 3.69), SMOG (p<0.001, CI 0.97 to 1.62), CLI (p<0.001, CI 7.46 to 8.27), and ARI (p<0.001, CI 0.43 to 1.39) scores were significantly different from the accepted standard. One-way analysis of variance (ANOVA) testing of FRES scores (p=0.009) and CLI scores (p=0.009) showed a significant difference between categories. Post hoc testing showed a significant difference between academic and non-profit categories for FRES scores (p=0.010, CI -15.17 to -1.47) and CLI scores (p=0.008, CI 0.35 to 3.29). Conclusions Most websites regarding hip OA are written above recommended reading levels, hence exceeding the comprehension levels of the average patient. Readability of these resources must be improved to improve patient access to online healthcare information which can lead to improved patient understanding of their own condition and treatment outcomes.