Flesch-Kincaid Research Articles

Readability of Patient-Facing Information of Antibiotics Used in the WHO Short 6-Month and 9-Month All Oral Treatment for Drug-Resistant Tuberculosis

ObjectivesReadability of patient-facing information of oral antibiotics detailed in the WHO all oral short (6 months, 9 months) has not been described to date. The aim of this study was therefore to examine (i) how readable patient-facing TB antibiotic information is compared to readability reference standards and (ii) if there are differences in readability between high-incidence countries versus low-incidence countries.MethodsTen antibiotics, including bedaquiline, clofazimine, ethambutol, ethionamide, isoniazid, levofloxacin, linezolid, moxifloxacin, pretomanid, pyrazinamide, were investigated. TB antibiotic information sources were examined, consisting of 85 Patient Information Leaflets (PILs) and 40 antibiotic web resouces. Of these 85 PILs, 72 were taken from the National Medicines Regulator from six countries (3 TB high-incidence [Rwanda, Malaysia, South Africa] + 3 TB low-incidence [UK, Ireland, Malta] countries). Readability data was grouped into three categories, including (i) high TB-incidence countries (n = 33 information sources), (ii) low TB-incidence countries (n = 39 information sources) and (iii) web information (n = 53). Readability was calculated using Readable software, to obtain four readability scores [(i) Flesch Reading Ease (FRE), (ii) Flesch-Kincaid Grade Level (FKGL), (iii) Gunning Fog Index and (iv) SMOG Index], as well as two text metrics [words/sentence, syllables/word].ResultsMean readability scores of patient-facing TB antibiotic information for FRE and FKGL, were 47.4 ± 12.6 (sd) (target ≥ 60) and 9.2 ± 2.0 (target ≤ 8.0), respectively. There was no significant difference in readability between low incidence countries and web resources, but there was significantly poorer readability associated with PILs from high incidence countries versus low incidence countries (FRE; p = 0.0056: FKGL; p = 0.0095).ConclusionsReadability of TB antibiotic PILs is poor. Improving readability of PILs should be an important objective when preparing patient-facing written materials, thereby improving patient health/treatment literacy.

Both English- and Spanish-Language Anterior Cruciate Ligament Reconstruction Online Patient Education Materials Are Written at Higher-Than-Recommended Reading Levels

PurposeThe purpose of this study is to examine the overall reading levels of anterior cruciate ligament reconstruction online patient education materials (OPEMs) written in English and Spanish. MethodsInvestigators conducted Google searches for OPEMs utilizing “ACL surgery” and “cirugía LCA” as English and Spanish search terms. Several measures of readability were used to analyze 25 English (Flesch Reading Ease, Flesch Reading Ease Grade Level, Flesch-Kincaid Grade Level, Coleman-Liau Index, Gunning-Fox Index, and the Simple Measure of Gobbledygook) and 25 Spanish (Fernández-Huerta Index, Fernández-Huerta Grade Level, and INFLESZ) OPEMs. English and Spanish OPEMs were compared based on mean overall grade level and number of OPEMs written below a seventh or ninth-grade reading level. ResultsEnglish OPEMs demonstrated a higher mean overall grade level compared to Spanish OPEMs (10.48 ± 1.86 vs 8.64 ± 1.22, p<.001). No significant differences were noted in the number of OPEMs written below a seventh-grade reading level. However, significantly more Spanish OPEMs were written below a ninth-grade reading level, compared to English OPEMs (56% vs 16%, p=.003). ConclusionAlthough Spanish OPEMs were written at a lower reading level, average readability for both English and Spanish OPEMs was significantly higher than the recommended level. Across both languages, only a single English webpage met the AMA-recommended sixth-grade reading level. More Spanish articles were written at or below the average adult reading level in the United States.

Understanding the cancer health literacy gap: NCCN member institutions’ website readability across the high mortality cancers in the US

BackgroundThe readability of cancer educational content often exceeds the average reading level of the U.S. population. This study evaluated the readability of online educational material for the five deadliest cancers in the US. MethodsWe assessed the readability of diagnosis and treatment information across 34 NCCN member institution websites. Readability was analyzed using four validated formulas: Fry Readability Score, Flesch-Kincaid Grade Level, Gunning-Fog Index, and Simple Measure of Gobbledygook. The primary outcome was the mean organizational readability level for each resource, and secondary outcomes included differences in readability between diagnosis and treatment materials and an analysis of sentence and word complexity. ResultsThe mean readability level across 34 websites was 13 ± 1.96, equivalent to a university freshman level. Treatment materials were significantly harder to read than diagnosis materials (14 ± 1.8 vs. 12 ± 1.8, p < 0.01) across all cancers. Readability varied significantly depending on the type of cancer, with material relating to the diagnosis of pancreatic cancer requiring the highest educational level (13 ± 1.9) and material relating to treating breast cancer requiring the lowest reading proficiency (12 ± 1.7). Furthermore, the analysis demonstrated that cancer diagnosis materials contained a lower median percentage of complex and long words than treatment materials but had a higher percentage of long sentences. ConclusionsThese findings underscore the need for organizations to prioritize matching the nation’s literacy levels. Despite the growing reliance on online resources for patient education, the current readability levels hinder patient comprehension

Readability analysis of ChatGPT's responses on lung cancer

For common diseases such as lung cancer, patients often use the internet to obtain medical information. As a result of advances in artificial intelligence and large language models such as ChatGPT, patients and health professionals use these tools to obtain medical information. The aim of this study was to evaluate the readability of ChatGPT-generated responses with different readability scales in the context of lung cancer. The most common questions in the lung cancer section of Medscape® were reviewed, and questions on the definition, etiology, risk factors, diagnosis, treatment, and prognosis of lung cancer (both NSCLC and SCLC) were selected. A set of 80 questions were asked 10 times to ChatGPT via the OpenAI API. ChatGPT's responses were tested using various readability formulas. The mean Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning FOG Scale, SMOG Index, Automated Readability Index, Coleman-Liau Index, Linsear Write Formula, Dale-Chall Readability Score, and Spache Readability Formula scores are at a moderate level (mean and standard deviation: 40.52 ± 9.81, 12.56 ± 1.66, 13.63 ± 1.54, 14.61 ± 1.45, 15.04 ± 1.97, 14.24 ± 1.90, 11.96 ± 2.55, 10.03 ± 0.63 and 5.93 ± 0.50, respectively). The readability levels of the answers generated by ChatGPT are "collage" and above and are difficult to read. Perhaps in the near future, the ChatGPT can be programmed to produce responses that are appropriate for people of different educational and age groups.

Caution Regarding ChatGPT's Appropriateness and Reliability Regarding Surgery for Wrist Arthritis.

Chat Generative Pre-Trained Transformer (ChatGPT), an artificial intelligence (AI) program, is widely used for information compilation. This study sought to analyze the quality and consistency of the information generated by ChatGPT regarding common procedures for wrist arthritis. 32 standardized questions regarding wrist osteoarthritis and related procedures (4-corner-fusion [4CF], proximal row carpectomy [PRC], resurfacing capitate pyrocarbon implant, wrist denervation, and total wrist arthrodesis and arthroplasty) were presented to the ChatGPT-3.5 interface 3 separate times, without feedback. ChatGPT's answers were evaluated for medical accuracy by 3 reviewers and rated as "appropriate," "appropriate but incomplete," or "inappropriate." Ratings were then converted to numerical values to calculate an intraclass correlation coefficient (ICC). A DISCERN score was used to assess quality, and Flesch-Kincade Grade Level and Flesch Reading Ease Score for readability. 75% of the responses were deemed "appropriate," with 23 questions receiving unanimous appropriate ratings across all responses. The ICC was 0.97 (95% CI [0.46, 0.98]), indicating excellent reliability. DISCERN score was 60 (good). The Flesch-Kincaid Grade Level was 14.6 ± 1.9, and the Flesch Reading Ease Score was 25.3 ± 6.7, implying a college reading level. The information that ChatGPT provided for PRC and total wrist arthrodesis and arthroplasty, appeared to be more reliable than for 4CF and denervation. ChatGPT's reliability and accuracy of information varied across procedures, possibly due to unknown and diverse sources. Furthermore, while some answers were factually correct, many provided generic information across differing questions, limiting usefulness. ChatGPT must be used cautiously and the limitations understood.

Analysis of internet educational websites on tobacco cessation: A content analysis

Background There have been significant changes in the lifestyles of individuals in the past few decades, which has led to increased morbidity and mortality worldwide. Both smoking and chewing forms of tobacco are highly prevalent, especially in India, and are implicated as causes of diseases, including oropharyngeal carcinomas. Effective tobacco cessation techniques and sources can help overcome addiction and reduce the disease burden in society. The aim was to evaluate the quality and readability of contents of various sources on an internet website about tobacco cessation. Objectives i) To evaluate the readability of internet content (Google) regarding tobacco cessation using Flesch–Kincaid readability tests and the quality of internet content (Google) by using the JAMA benchmark, HONcode and DISCERN questionnaire. Methods A content review was employed to screen the content of the Google search engine for educational tobacco cessation websites, and the top 50 websites were selected according to criteria and reviewed by two reviewers. The readability of the internet content (Google) regarding tobacco cessation was evaluated using Flesch–Kincaid readability tests. The quality of the screened sites was evaluated by using the JAMA benchmark, HONcode and DISCERN questionnaire, and the readability and quality of the screened websites were correlated using the above instruments. Results FK readability ease was found to be 49% standard and 30% easy. The FK grade test found that 33% of the content could be easily understood by &lt; 5th grade. All 4 JAMA benchmarks were met by 23% of websites, and authorship was the least fulfilled criterion. Correlation analysis revealed a significant association between FK ease score and FK grade score. Conclusions The Read-ability Ease and Read-ability Grade Levels of the websites related to tobacco cessation were not standard, and few websites fulfilled the JAMA benchmarks and had HONcode certification.

Online information about oral health in autism spectrum disorder: Is it good enough?

The use of the internet has surged significantly over the years. Patients and caregivers of patients with autism spectrum disorder (ASD) might consult the internet for oral health-related information. Hence, this study aimed to assess the quality and readability of online information available in the English language regarding oral health in ASD. Online search using Google.com was conducted using the terms "Autism and dental care," "Autism and oral health," and "Autism and dentistry". The first 100 websites for each term were screened. Quality of information was assessed using the Patient Education Materials Assessment Tool for printed material (PEMAT-P) and the Journal of American Medical Association (JAMA) benchmarks. A PEMAT score higher than 70% is considered acceptable for readability and actionability. The JAMA benchmarks are authorship, attribution, disclosure, and currency. Readability was evaluated using the Flesch reading ease score and Simple Measure of Gobbledygook (SMOG) readability formula. Out of the 300 screened websites, 66 were eventually included. The mean PEMAT understandability and actionability scores were 77.13%, and 42.12%, respectively. Only 12.1% of the websites displayed all four JAMA benchmarks. The mean Flesch score was 10th-12th grade level, and the mean SMOG score was 10th grade level. While the understandability of the information was acceptable, the readability and actionability were too challenging for lay people. Health care professionals and organizations involved in patient education should place more efforts in promoting the quality of online information targeting patients with ASD.

Lessons to be learned when designing comprehensible patient-oriented online information about temporomandibular disorders.

Temporomandibular disorders (TMD) are a prevalent ailment with a global impact, affecting a substantial number of individuals. While some individuals are receiving treatment from orthodontists for TMD, a significant proportion of individuals obtain knowledge through websites. Our purpose had been to evaluate, from a patient-oriented perspective, the readability of home pages of websites scored in the 10 most prominent devoted to TMD. We also determined what level of education would have been needed to get an overview of the information on the websites under scrutiny. This approach ensures that our findings are centred on the patient experience, providing insights into how accessible and understandable websites about TMD. We determined the top 10 patient-focused English language websites by searching for 'temporomandibular disorders' in the 'no country redirect' plugin of the Google Chrome browser (www.google.com/ncr). The readability of the texts was assessed using the Gunning fog index (GFI), Coleman Liau index (CLI), Automated readability index (ARI) Simple Measure of Gobbledygook (SMOG), Flesch Kincald grade level (FKGL), Flesh reasing ease (FRE) (https://readabilityformulas.com). The mean Flesch reading ease index score was determined to be 48.67, accompanied by a standard deviation of 15.04 and these websites require an average of 13.49 years of formal education (GFI), with a standard deviation of 2.62, for ease of understanding. Our research indicates that a significant proportion of websites related to TMD can be defined as a level of complexity that exceeds the ability to read comprehension of the general population.

Assessing the response quality and readability of chatbots in cardiovascular health, oncology, and psoriasis: A comparative study

BackgroundChatbots using the Large Language Model (LLM) generate human responses to questions from all categories. Due to staff shortages in healthcare systems, patients waiting for an appointment increasingly use chatbots to get information about their condition. Given the number of chatbots currently available, assessing the responses they generate is essential. MethodsFive chatbots with free access were selected (Gemini, Microsoft Copilot, PiAI, ChatGPT, ChatSpot) and blinded using letters (A, B, C, D, E). Each chatbot was asked questions about cardiology, oncology, and psoriasis. Responses were compared to guidelines from the European Society of Cardiology, American Academy of Dermatology and American Society of Clinical Oncology. All answers were assessed using readability scales (Flesch Reading Scale, Gunning Fog Scale Level, Flesch-Kincaid Grade Level and Dale-Chall Score). Using a 3-point Likert scale, two independent medical professionals assessed the compliance of the responses with the guidelines. ResultsA total of 45 questions were asked of all chatbots. Chatbot C gave the shortest answers, 7.0 (6.0 – 8.0), and Chatbot A the longest 17.5 (13.0 – 24.5). The Flesch Reading Ease Scale ranged from 16.3 (12.2 – 21.9) (Chatbot D) to 39.8 (29.0 – 50.4) (Chatbot A). Flesch-Kincaid Grade Level ranged from 12.5 (10.6 – 14.6) (Chatbot A) to 15.9 (15.1 – 17.1) (Chatbot D). Gunning Fog Scale Level ranged from 15.77 (Chatbot A) to 19.73 (Chatbot D). Dale-Chall Score ranged from 10.3 (9.3 – 11.3) (Chatbot A) to 11.9 (11.5 – 12.4) (Chatbot D). ConclusionThis study indicates that chatbots vary in length, quality, and readability. They answer each question in their own way, based on the data they have pulled from the web. Reliability of the responses generated by chatbots is high. This suggests that people who want information from a chatbot need to be careful and verify the answers they receive, particularly when they ask about medical and health aspects.

TrachGPT: Appraisal of tracheostomy care recommendations from an artificial intelligent Chatbot.

Safe home tracheostomy care requires engagement and troubleshooting by patients, who may turn to online, AI-generated information sources. This study assessed the quality of ChatGPT responses to such queries. In this cross-sectional study, ChatGPT was prompted with 10 hypothetical tracheostomy care questions in three domains (complication management, self-care advice, and lifestyle adjustment). Responses were graded by four otolaryngologists for appropriateness, accuracy, and overall score. The readability of responses was evaluated using the Flesch Reading Ease (FRE) and Flesch-Kincaid Reading Grade Level (FKRGL). Descriptive statistics and ANOVA testing were performed with statistical significance set to p < .05. On a scale of 1-5, with 5 representing the greatest appropriateness or overall score and a 4-point scale with 4 representing the highest accuracy, the responses exhibited moderately high appropriateness (mean = 4.10, SD = 0.90), high accuracy (mean = 3.55, SD = 0.50), and moderately high overall scores (mean = 4.02, SD = 0.86). Scoring between response categories (self-care recommendations, complication recommendations, lifestyle adjustments, and special device considerations) revealed no significant scoring differences. Suboptimal responses lacked nuance and contained incorrect information and recommendations. Readability indicated college and advanced levels for FRE (Mean = 39.5, SD = 7.17) and FKRGL (Mean = 13.1, SD = 1.47), higher than the sixth-grade level recommended for patient-targeted resources by the NIH. While ChatGPT-generated tracheostomy care responses may exhibit acceptable appropriateness, incomplete or misleading information may have dire clinical consequences. Further, inappropriately high reading levels may limit patient comprehension and accessibility. At this point in its technological infancy, AI-generated information should not be solely relied upon as a direct patient care resource.

Readability of online patient education material for foregut surgery.

Health literacy is the ability of individuals to use basic health information and services to make well-informed decisions. Low health literacy among surgical patients has been associated with nonadherence to preoperative and/or discharge instructions as well as poor comprehension of surgery. It likely poses as a barrier to patients considering foregut surgery which requires an understanding of different treatment options and specific diet instructions. The objective of this study was to assess and compare the readability of online patient education materials (PEM) for foregut surgery. Using Google, the terms "anti-reflux surgery, "GERD surgery," and "foregut surgery" were searched and a total of 30 webpages from universities and national organizations were selected. The readability of the text was assessed with seven instruments: Flesch Reading Ease formula (FRE), Gunning Fog (GF), Flesch-Kincaid Grade Level (FKGL), Coleman Liau Index (CL), Simple Measure of Gobbledygook (SMOG), Automated Readability Index (ARI), and Linsear Write Formula (LWF). Mean readability scores were calculated with standard deviations. We performed a qualitative analysis gathering characteristics such as, type of information (preoperative or postoperative), organization, use of multimedia, inclusion of a version in another language. The overall average readability of the top PEM for foregut surgery was 12th grade. There was only one resource at the recommended sixth grade reading level. Nearly half of PEM included some form of multimedia. The American Medical Association and National Institute of Health have recommended that PEMs to be written at the 5th-6th grade level. The majority of online PEM for foregut surgery is above the recommended reading level. This may be a barrier for patients seeking foregut surgery. Surgeons should be aware of the potential gaps in understanding of their patients to help them make informed decisions and improve overall health outcomes.

Using ChatGPT to Improve Readability of Interventional Radiology Procedure Descriptions.

This project examines ChatGPT's potential to enhance the readability of patient educational materials about interventional radiology (IR) procedures. The descriptions of IR procedures from the Cardiovascular and Interventional Radiological Society of Europe (CIRSE) were used as the original text. Readability scores were calculated using three metrics: Flesch Reading Ease (FRE), Gunning Fog (GF), and the Automated Readability Index (ARI) using an online calculator ( https://readabilityformulas.com ). FRE is scored on a scale of 0-100, where 100 indicates easy-to-read texts, and GF and ARI represent the grade level required to comprehend the text. The DISCERN instrument measured credibility and reliability. ChatGPT was prompted to simplify the texts to a fifth-grade reading level, with subsequent recalculation of readability and DISCERN scores for comparison. Statistical significance was determined using a Wilcoxon Signed-Rank Test. Articles were subsequently organized by subgroups and analyzed. 73 interventional radiology procedures from CIRSE were analyzed. The original FRE score was 47.2 (Difficult), improved to 78.4 (Fairly Easy) by ChatGPT. GF and ARI scores dropped from 14.4 and 11.2 to 7.8 and 5.8, respectively, after simplification, showing significant improvement (p < 0.001). However, the average DISCERN score decreased from 3.73 to 2.99 (p < 0.001) post-ChatGPT simplification. This study shows ChatGPT's ability to make interventional radiology descriptions more readable but highlights its struggle to maintain the original's reliability, suggesting the need for human review and prompt engineering to enhance outcomes. Level 6.

Is ChatGPT a Reliable Source of Patient Information on Asthma?

ChatGPT (OpenAI, San Francisco, CA, USA) is a novel artificial intelligence (AI) application that is used by millions of people, and the numbers are growing by the day. Because it has the potential to be a source of patient information, the study aimed to evaluate the ability of ChatGPT to answer frequently asked questions (FAQs) about asthma with consistent reliability, acceptability, and easy readability. We collected 30 FAQs about asthma from the Global Initiative for Asthma website. ChatGPT was asked each question twice, by two different users, to assess for consistency. The responses were evaluated by five board-certified internal medicine physicians for reliability and acceptability. The consistency of responses was determined by the differences in evaluation between the two answers to the same question. The readability of all responses was measured using the Flesch Reading Ease Scale (FRES), the Flesch-Kincaid Grade Level (FKGL), and the Simple Measure of Gobbledygook (SMOG). Sixty responses were collected for evaluation. Fifty-six (93.33%) of the responses were of good reliability. The average rating of the responses was 3.65 out of 4 total points. 78.3% (n=47) of the responses were found acceptable by the evaluators to be the only answer for an asthmatic patient. Only two (6.67%) of the 30 questions had inconsistent answers. The average readability of all responses was determined to be 33.50±14.37 on the FRES, 12.79±2.89 on the FKGL, and 13.47±2.38 on the SMOG. Compared to online websites, we found that ChatGPT can be a reliable and acceptable source of information for asthma patients in terms of information quality. However, all responses were of difficult readability, and none followed the recommended readability levels. Therefore, the readability of this AI application requires improvement to be more suitable for patients.

Written information about retinopathy of prematurity in Aotearoa New Zealand: identification, review and opportunities for improvement.

The aims of this research include adapting a patient information tool for whānau (extended family) Māori needs, identifying and reviewing written information provided for the retinopathy of prematurity eye examination (ROPEE) and identifying improvements to ROPEE written information. ROPEE patient information (printed leaflets, website, app) was obtained from all tertiary neonatal intensive care units in Aotearoa New Zealand (Aotearoa). Information was reviewed using an adapted "20 good-design principles" guide and given a star rating and Flesch-Kincaid readability score to identify acceptability and usability for patients. Seven ROPEE information materials were reviewed and varied in alignment with the adapted good-design principles tool. Based on the adapted good-design principles, opportunities were identified in many aspects of the written information for improvement, including words and language, tone and meaning, content and design. The Flesch-Kincaid grade level reading scores ranged from 12-22 years reading age. Written information also did not use te reo Māori (Aotearoa Indigenous language) or extensively use Māori imagery. Opportunities exist to improve ROPEE whānau information, including making content more readable, understandable and visually appealing. Optimising the clinical information on ROPEE nationally for Aotearoa will support whānau decision making, and aligning written information with Māori (Indigenous peoples of Aotearoa) is a priority.

20 An Evaluation of the Quality and Readability of Online Information Regarding TMJ Dysfunction

Abstract Aim The accessibility and availability of the internet has allowed people to increasingly seek health information online, and therefore it plays an essential role in the decision-making process of the patient. The study aimed to assess the quality and readability of online health information on TMJ dysfunction. Method The search was conducted on Google using keywords: jaw pain, jaw locking, jaw discomfort, jaw clicking, jaw popping, temporomandibular joint, TMJ dysfunction, TMD and temporomandibular disorder. Ensuring Quality Information for Patient (EQIP) tool was used to assess quality of information and the application of readability score reflected through Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level(FKGL), Gunning-Fog Index(GFI) and Simple Measure of Gobbledygook(SMOG). Results 134 websites were eligible for assessment. The overall median EQIP score was 23, with an IQR of 7 and maximum score of 30. The most common source of information was healthcare industry (n=60; 45%), followed by news services (n=19; 14%), with a median EQIP score of 20 and 25 respectively. Statistically significant differences in overall EQIP scores were found between different sources of information (p=0.004). Only 31(23%) websites achieved the recommended 6th grade or below readability level, and only 14 (10%) discussed complication rates with none mentioning mortality rates at all. Conclusions Most online information on TMJ dysfunction is of suboptimal quality and readability. This study identified areas for improvements when producing patient material regarding TMJ dysfunction. To improve patient education and awareness, content creators should strive to be informative, accurate, and patient-centric.

Is ChatGPT an Accurate and Readable Patient Aid for Third Molar Extractions?

Artificial intelligence (AI) platforms such as Chat Generative Pre-Trained Transformer (ChatGPT) (Open AI, San Francisco, California, USA) have the capacity to answer health-related questions. It remains unknown whether AI can be a patient-friendly and accurate resource regarding third molar extraction. The purpose was to determine the accuracy and readability of AI responses to common patient questions regarding third molar extraction. This is a cross sectional in-silico assessment of readability and soundness of a computer-generated report. Not applicable. Accuracy, or the ability to provide clinically correct and relevant information, was determined subjectively by 2 reviewers using a 5-point Likert scale, and objectively by comparing responses to American Association of Oral and Maxillofacial Surgeons (AAOMS) clinical consensus papers. Readability, or how easy a piece of text is to read, was assessed using the Flesch Kincaid Reading Ease (FKRE) and Flesch Kincaid Grade Level (FKGL). Both assess readability based on mean number of syllables per word, and words per sentence. To be deemed "readable", FKRE should be >60 and FKGL should be <8. Not applicable. Descriptive statistics were used to analyze the findings of this study. AI-generated responses above the recommended level for the average patient (FKRE: 52; FKGL: 10). The average Likert score was 4.36, suggesting that most responses were accurate with minor inaccuracies or missing information. AI correctly deferred to the provider in instances where no definitive answer exists. Of the responses that addressed content in AAOMS consensus papers, 18/19 responses closely aligned with them. All prompts did not provide citations or references. AI was able to provide mostly accurate responses, and content was closely aligned with AAOMS guidelines. However, responses were too complex for the average third molar extraction patient, and were deficient in citations and references. It is important for providers to educate patients on the utility of AI, and to decide whether to recommend using it for information. Ultimately, the best resource for answers is from the practitioners themselves because the AI platform lacks clinical experience.

Online content on eating disorders: a natural language processing study

ABSTRACT Background Online content can inform the personal risk of developing an eating disorder, and it can influence the time and motivation to seek treatment. Patients routinely seek information online, and access to information is crucial for both prevention and treatment. The primary aim of the current study was to quantify the readability scores of online content on eating disorders using natural language processing algorithms, across two languages: English and Italian. Methods Unique terms related to single diagnoses were searched using Google®. The content available on Wikipedia was also assessed. Readability was defined according to the Flesch Readability Ease (FRE) and the Rate Readability Index (RIX). The scientific support of retrieved content and the authoritativeness of sources were measured through standardized variables. Results In Italian, online content was more likely published by private psychotherapy institutes or by websites that promote diet-advice or weight-loss. In both languages, the most readable content was on Anorexia Nervosa (RIX 4.18, FRE-en 59.6, FRE-it 41.69), Bulimia Nervosa (RIX 3.99, FRE-en 66.27, FRE-it 39.66) or Binge Eating (RIX 4.01, FRE-en 68.10, FRE-it 38.62). English sources consistently had more references than Italian pages (range 35–182, vs 1–163, respectively). and had a higher percentage of citations available in the target language. The content of these references was mainly reflective of peer-reviewed or clinical manuals. Conclusion Attention should be given to developing online content for Muscle Dysmorphia and Orthorexia Nervosa, as well as improving the overall readability of online content on eating disorders, especially for languages other than English.

Capturing the ghost in the machine: a process for development and validation of measures of phenomenal consciousness in delirium.

Delirium has conventionally been considered a disorder of consciousness. Alertness and arousal are used as surrogates in clinical practice but are insufficient for the purposes of a more dimensional assessment of consciousness. We present a process of development and validation of candidate measures of phenomenal consciousness that could be applied to the diagnosis of delirium. First, a narrative review of available instruments in the fields of phenomenal consciousness, including prereflective consciousness, the phenomenal-sensed experience and reflective thought, was undertaken. Eligibility of tools in the context of applicability to delirium was based upon objectivity in test interpretation and the requirement for tester administration. Second, where there was a gap in suitable cognitive tools, new items were derived using the silent generation technique. A process of face and construct validity using a diverse panel of experts was performed, and readability was evaluated. 814 articles were screened from the literature review. Fourteen candidate tools were reported from the three domains of phenomenal consciousness. One of these met the eligibility criteria for a delirium assessment. Fifty-seven new tests of phenomenal consciousness were identified. After a process of item reduction, a total of 26 individual tests were identified. After content validity, 22 of the 26 items were retained. The scale average content validity index was 0.89. The agreement between raters was between 80% and 97%. 100% of responses for face validity were rated as positive. Flesch Reading Ease Score was 91.6 (very easy to read). Candidate measures of phenomenal consciousness are described, and early validity studies are promising.

The Promise and the Challenge: Large Language Models for Patient Education - Are We There Yet?

This study aimed to evaluate the potential of large language models (LLMs) for delivering patient education materials. Two LLMs, Gemini and ChatGPT 3.5, were analysed for their ability to provide clear and understandable information on the topic of blepharitis, a common eye condition. The understandability and actionability of the information provided by the LLMs in response to a set of questions were evaluated using PEMAT, a standardised tool for assessing educational materials. The responses included the important aspects of blepharitis, yet the Flesch-Kincaid readability scores were below the suggested range of 60-70 for patient education materials. Gemini received a score of 38.75, whereas ChatGPT 3.5 earned 26.35, suggesting that the content might be too intricate for the target audience.. These findings suggest that while LLMs have the potential to be informative resources, their current readability levels may limit their effectiveness in providing accessible health information to patients. Further research is needed to explore methods for adapting LLM outputs to ensure clear and concise communication suitable for patient education.

Content Analysis of Arabic Websites As Patient Resources for Osteoporosis.

Osteoporosis is a prevalent metabolic bone disease in the Middle East. Middle Easterners rely on the Internet as a source of information about osteoporosis and its treatment. Adequate awareness can help to prevent osteoporosis and its complications. Websites covering osteoporosis in Arabic must be of good quality and readability to be beneficial for people in the Middle East. Two Arabic terms for osteoporosis were searched on Google.com (Google Inc., Mountainview, CA), and the first 100 results for each term were examined for eligibility. Two independent raters evaluated the websites using DISCERNand the Journal of the American Medical Association (JAMA) criteria for quality and reliability. The Flesch Kincaid grade level (FKGL), Simple Measure of Gobbledygook (SMOG), and Flesch Reading Ease (FRE) scale were used to evaluate the readability of each website's content. Twenty-five websites were included and evaluated in our study. The average DISCERNscore was 28.36±12.18 out of 80 possible scores. The average JAMA score was 1.05±1.15 out of four total scores. The readability scores of all websites were, on average, 50.71±21.96 on the FRE scale, 9.25±4.89 on the FKGL, and 9.74±2.94 on the SMOG. There was a significant difference (p = 0.026 and 0.044) in the DISCERNand JAMA scores, respectively, between the websites on the first Google page and the websites seen on later pages. The study found Arabic websites covering osteoporosis to be of low quality and difficult readability. Because these websites are a major source for patient education, improving their quality and readability is a must. The use of simpler language is needed, as is covering more aspects of the diseases, such as prevention.