Reading Level Research Articles

Understanding the cancer health literacy gap: NCCN member institutions’ website readability across the high mortality cancers in the US

BackgroundThe readability of cancer educational content often exceeds the average reading level of the U.S. population. This study evaluated the readability of online educational material for the five deadliest cancers in the US. MethodsWe assessed the readability of diagnosis and treatment information across 34 NCCN member institution websites. Readability was analyzed using four validated formulas: Fry Readability Score, Flesch-Kincaid Grade Level, Gunning-Fog Index, and Simple Measure of Gobbledygook. The primary outcome was the mean organizational readability level for each resource, and secondary outcomes included differences in readability between diagnosis and treatment materials and an analysis of sentence and word complexity. ResultsThe mean readability level across 34 websites was 13 ± 1.96, equivalent to a university freshman level. Treatment materials were significantly harder to read than diagnosis materials (14 ± 1.8 vs. 12 ± 1.8, p < 0.01) across all cancers. Readability varied significantly depending on the type of cancer, with material relating to the diagnosis of pancreatic cancer requiring the highest educational level (13 ± 1.9) and material relating to treating breast cancer requiring the lowest reading proficiency (12 ± 1.7). Furthermore, the analysis demonstrated that cancer diagnosis materials contained a lower median percentage of complex and long words than treatment materials but had a higher percentage of long sentences. ConclusionsThese findings underscore the need for organizations to prioritize matching the nation’s literacy levels. Despite the growing reliance on online resources for patient education, the current readability levels hinder patient comprehension

Readability analysis of ChatGPT's responses on lung cancer

For common diseases such as lung cancer, patients often use the internet to obtain medical information. As a result of advances in artificial intelligence and large language models such as ChatGPT, patients and health professionals use these tools to obtain medical information. The aim of this study was to evaluate the readability of ChatGPT-generated responses with different readability scales in the context of lung cancer. The most common questions in the lung cancer section of Medscape® were reviewed, and questions on the definition, etiology, risk factors, diagnosis, treatment, and prognosis of lung cancer (both NSCLC and SCLC) were selected. A set of 80 questions were asked 10 times to ChatGPT via the OpenAI API. ChatGPT's responses were tested using various readability formulas. The mean Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning FOG Scale, SMOG Index, Automated Readability Index, Coleman-Liau Index, Linsear Write Formula, Dale-Chall Readability Score, and Spache Readability Formula scores are at a moderate level (mean and standard deviation: 40.52 ± 9.81, 12.56 ± 1.66, 13.63 ± 1.54, 14.61 ± 1.45, 15.04 ± 1.97, 14.24 ± 1.90, 11.96 ± 2.55, 10.03 ± 0.63 and 5.93 ± 0.50, respectively). The readability levels of the answers generated by ChatGPT are "collage" and above and are difficult to read. Perhaps in the near future, the ChatGPT can be programmed to produce responses that are appropriate for people of different educational and age groups.

The Readability of Non-Financial Information. The Role of Stakeholders' Pressure in the European Setting

This study explores the factors that may explain the readability of non-financial information (NFI) disclosure through the interpretive lens of legitimacy theory. In particular, it investigates whether the readability of the NFI disclosure, prepared under Directive 2014/95/EU, is affected by the pressure of certain stakeholders and by the presence of a corporate social responsibility committee (CSRc). The sample is based on 9,218 European listed firms, covering the period 2017-2020. The results show that pressure from environmentally sensitive industries, consumers, and some investors plays a crucial role in improving the readability level of NFI, while employees and the State owner do not exercise an influence on the level of readability. In addition, the presence of a CSRc is not only relevant per se, but also acts as a moderator in enhancing the association with these stakeholders on the readability of NFI disclosure.

Caution Regarding ChatGPT's Appropriateness and Reliability Regarding Surgery for Wrist Arthritis.

Chat Generative Pre-Trained Transformer (ChatGPT), an artificial intelligence (AI) program, is widely used for information compilation. This study sought to analyze the quality and consistency of the information generated by ChatGPT regarding common procedures for wrist arthritis. 32 standardized questions regarding wrist osteoarthritis and related procedures (4-corner-fusion [4CF], proximal row carpectomy [PRC], resurfacing capitate pyrocarbon implant, wrist denervation, and total wrist arthrodesis and arthroplasty) were presented to the ChatGPT-3.5 interface 3 separate times, without feedback. ChatGPT's answers were evaluated for medical accuracy by 3 reviewers and rated as "appropriate," "appropriate but incomplete," or "inappropriate." Ratings were then converted to numerical values to calculate an intraclass correlation coefficient (ICC). A DISCERN score was used to assess quality, and Flesch-Kincade Grade Level and Flesch Reading Ease Score for readability. 75% of the responses were deemed "appropriate," with 23 questions receiving unanimous appropriate ratings across all responses. The ICC was 0.97 (95% CI [0.46, 0.98]), indicating excellent reliability. DISCERN score was 60 (good). The Flesch-Kincaid Grade Level was 14.6 ± 1.9, and the Flesch Reading Ease Score was 25.3 ± 6.7, implying a college reading level. The information that ChatGPT provided for PRC and total wrist arthrodesis and arthroplasty, appeared to be more reliable than for 4CF and denervation. ChatGPT's reliability and accuracy of information varied across procedures, possibly due to unknown and diverse sources. Furthermore, while some answers were factually correct, many provided generic information across differing questions, limiting usefulness. ChatGPT must be used cautiously and the limitations understood.

The diagnostic and triage accuracy of the GPT-3 artificial intelligence model: an observational study

Artificial intelligence (AI) applications in health care have been effective in many areas of medicine, but they are often trained for a single task using labelled data, making deployment and generalisability challenging. How well a general-purpose AI language model performs diagnosis and triage relative to physicians and laypeople is not well understood. We compared the predictive accuracy of Generative Pre-trained Transformer 3 (GPT-3)'s diagnostic and triage ability for 48 validated synthetic case vignettes (<50 words; sixth-grade reading level or below) of both common (eg, viral illness) and severe (eg, heart attack) conditions to a nationally representative sample of 5000 lay people from the USA who could use the internet to find the correct options and 21 practising physicians at Harvard Medical School. There were 12 vignettes for each of four triage categories: emergent, within one day, within 1 week, and self-care. The correct diagnosis and triage category (ie, ground truth) for each vignette was determined by two general internists at Harvard Medical School. For each vignette, human respondents and GPT-3 were prompted to list diagnoses in order of likelihood, and the vignette was marked as correct if the ground-truth diagnosis was in the top three of the listed diagnoses. For triage accuracy, we examined whether the human respondents' and GPT-3's selected triage was exactly correct according to the four triage categories, or matched a dichotomised triage variable (emergent or within 1 day vs within 1 week or self-care). We estimated GPT-3's diagnostic and triage confidence on a given vignette using a modified bootstrap resampling procedure, and examined how well calibrated GPT-3's confidence was by computing calibration curves and Brier scores. We also performed subgroup analysis by case acuity, and an error analysis for triage advice to characterise how its advice might affect patients using this tool to decide if they should seek medical care immediately. Among all cases, GPT-3 replied with the correct diagnosis in its top three for 88% (42/48, 95% CI 75-94) of cases, compared with 54% (2700/5000, 53-55) for lay individuals (p<0.0001) and 96% (637/666, 94-97) for physicians (p=0·012). GPT-3 triaged 70% correct (34/48, 57-82) versus 74% (3706/5000, 73-75; p=0.60) for lay individuals and 91% (608/666, 89-93%; p<0.0001) for physicians. As measured by the Brier score, GPT-3 confidence in its top prediction was reasonably well calibrated for diagnosis (Brier score=0·18) and triage (Brier score=0·22). We observed an inverse relationship between case acuity and GPT-3 accuracy (p<0·0001) with a fitted trend line of -8·33% decrease in accuracy for every level of increase in case acuity. For triage error analysis, GPT-3 deprioritised truly emergent cases in seven instances. A general-purpose AI language model without any content-specific training could perform diagnosis at levels close to, but below, physicians and better than lay individuals. We found that GPT-3's performance was inferior to physicians for triage, sometimes by a large margin, and its performance was closer to that of lay individuals. Although the diagnostic performance of GPT-3 was comparable to physicians, it was significantly better than a typical person using a search engine. The National Heart, Lung, and Blood Institute.

Evaluating Health Literacy Resources for Secondary Hyperparathyroidism in End-Stage Kidney Disease

IntroductionParathyroidectomy is recommended for severe secondary hyperparathyroidism (SHPT) due to end-stage kidney disease (ESKD), but surgery is underutilized. High quality and accessible online health information, recommended to be at a 6th-grade reading level, is vital to improve patient health literacy. This study evaluated available online resources for SHPT from ESKD based on information quality and readability. MethodsThree search engines were queried using the terms “parathyroidectomy for secondary hyperparathyroidism,” “parathyroidectomy kidney/renal failure,” “parathyroidectomy dialysis patients,” “should I have surgery for hyperparathyroidism due to kidney failure?,” and “do I need surgery for hyperparathyroidism due to kidney failure if I do not have symptoms?” Websites were categorized by source and origin. Two independent reviewers determined information quality using JAMA (0-4) and DISCERN (1-5) frameworks, and scores were averaged. Cohen's kappa evaluated inter-rater reliability. Readability was determined using the Flesch Kincaid Reading Ease, Flesch Kincaid Grade Level, and Simple Measure of Gobbledygook tools. Median readability scores were calculated, and corresponding grade level determined. Websites with reading difficulties >6th grade level were calculated. ResultsThirty one (86.1%) websites originated from the U.S., with most from hospital-associated (63.9%) and foundation/advocacy sources (30.6%). The mean JAMA and DISCERN scores for all websites were 1.3 ± 1.4 and 2.6 ± 0.7, respectively. Readability scores ranged from grade level 5–college level, and most websites scored above the recommended 6th grade level. ConclusionsPatient-oriented websites tailoring SHPT from ESKD are at a reading level higher than recommended, and the quality of information is low. Efforts must be made to improve the accessibility and quality of information for all patients.

Word recognition thresholds in novice readers: exploring when reading and listening comprehension are comparable

AbstractThis study proposes and identifies two levels of word reading at which good comprehenders can obtain a similar understanding of written and spoken narrative texts: The oral-written matching functional and efficient thresholds. These thresholds are compared with the level of word reading over which a relationship between listening comprehension and reading comprehension is evident (a basic word recognition threshold) and with the level after which better word recognition does not result in improvements in reading comprehension (an upper word recognition threshold). The three first thresholds were calculated in a sample of 344 first to third-grade Spanish students who read and listened to two narrative texts of different difficulty and who were assessed on measures of comprehension and word recognition. Listening and reading comprehension were related when participants accurately read more than 73% of pseudowords in a list (basic word recognition threshold). For participants with good listening comprehension, reading comprehension performance matched listening comprehension once a moderate reading speed of ~ 66 decontextualized words per minute was achieved (oral-written matching functional threshold) or when the rate of reading the text was ~ 140 words per minute (oral-written matching efficient threshold). The value of the oral-written matching efficient threshold was, in this study, similar to the upper word recognition threshold identified by prior research. The thresholds calculated in this study delineate a sequence of increasingly challenges to the developing reader, allowing a nuanced description of the initial stages of reading development.

Handwriting Legibility is Related to Spelling Level, but Not to Reading and Spelling Progress

ABSTRACT In three studies the relationship between handwriting difficulties and dyslexia was investigated. Study 1 examined differences in handwriting difficulties between children with (n = 331) and without dyslexia (n = 305). Children with dyslexia experienced more difficulties with legible handwriting than those without. Study 2 explored the relationship between handwriting difficulties and reading/spelling proficiency among children with dyslexia (n = 311). Reading level and legibility were not related, whereas spelling level and legibility were. Children whose handwriting was extremely poor were more likely to have spelling problems. Study 3 investigated whether handwriting difficulties affected progress in reading and spelling during remediation (n = 249). While children with limited handwriting legibility generally exhibited poorer spelling skills, it did not impede their progress in reading and spelling during the intervention. These findings suggest that handwriting proficiency does not necessarily affect remediation efforts for reading and spelling.

Assessing the readability and quality of online patient information for laser tattoo removal

Just as tattoos continue to increase in popularity, many people with tattoos also seek removal, often due to career concerns. Prospective clients interested in laser tattoo removal may do research about the procedure online, as the internet increasingly becomes a resource to get preliminary health information. However, it is important that the online health information on the topic be of high quality and be accessible to all patients. We analyzed 77 websites from a Google search query using the terms “Laser tattoo removal patient Information” and “Laser tattoo removal patient Instructions” to assess this. The websites were evaluated for their readability using multiple validated indices and comprehensiveness. We found that websites had a broad readability range, from elementary to college, though most were above the recommended eighth-grade reading level. Less than half of the websites adequately discussed the increased risk of pigmentary complications in the skin of color clients or emphasized the importance of consulting with a board-certified dermatologist/plastic surgeon before the procedure. Over 90% of the websites noted that multiple laser treatments are likely needed for complete clearance of tattoos. The findings from our study underscore a significant gap in the accessibility and quality of online information for patients considering laser tattoo removal, particularly in addressing specific risks for patients with darker skin tones and emphasizing the need for consulting a board-certified physician before undergoing the procedure. It is important that online resources for laser tattoo removal be appropriately written to allow better decision-making, expectations, and future satisfaction for potential clients interested in the procedure.

Application of English Multi-Modal Reading Teaching Mode in the Context of “Internet Plus”

Under the background of “Internet plus”, the application of English multi-modal reading teaching mode is of great significance. With the continuous development of Internet technology, the field of education has also been deeply influenced, especially in English reading teaching. Therefore, this study aims to explore how to integrate multi-modal resources into English reading teaching under the background of “Internet plus”. Through literature review and case analysis, it is found that multi-modal reading teaching mode can improve students' reading interest and effectiveness. The results show that the use of images, audio, video and other forms of media resources in the teaching process can help students better understand and absorb English reading materials. Therefore, this paper proposes that in the era of “Internet plus”, teachers should actively design multi-modal English reading teaching activities with the help of various digital media resources, so as to improve students' reading level and ability and promote the development and innovation of English teaching.

TrachGPT: Appraisal of tracheostomy care recommendations from an artificial intelligent Chatbot.

Safe home tracheostomy care requires engagement and troubleshooting by patients, who may turn to online, AI-generated information sources. This study assessed the quality of ChatGPT responses to such queries. In this cross-sectional study, ChatGPT was prompted with 10 hypothetical tracheostomy care questions in three domains (complication management, self-care advice, and lifestyle adjustment). Responses were graded by four otolaryngologists for appropriateness, accuracy, and overall score. The readability of responses was evaluated using the Flesch Reading Ease (FRE) and Flesch-Kincaid Reading Grade Level (FKRGL). Descriptive statistics and ANOVA testing were performed with statistical significance set to p < .05. On a scale of 1-5, with 5 representing the greatest appropriateness or overall score and a 4-point scale with 4 representing the highest accuracy, the responses exhibited moderately high appropriateness (mean = 4.10, SD = 0.90), high accuracy (mean = 3.55, SD = 0.50), and moderately high overall scores (mean = 4.02, SD = 0.86). Scoring between response categories (self-care recommendations, complication recommendations, lifestyle adjustments, and special device considerations) revealed no significant scoring differences. Suboptimal responses lacked nuance and contained incorrect information and recommendations. Readability indicated college and advanced levels for FRE (Mean = 39.5, SD = 7.17) and FKRGL (Mean = 13.1, SD = 1.47), higher than the sixth-grade level recommended for patient-targeted resources by the NIH. While ChatGPT-generated tracheostomy care responses may exhibit acceptable appropriateness, incomplete or misleading information may have dire clinical consequences. Further, inappropriately high reading levels may limit patient comprehension and accessibility. At this point in its technological infancy, AI-generated information should not be solely relied upon as a direct patient care resource.

Evaluating the Accuracy, Quality, and Readability of Online Breast Cancer Information.

To assess the accuracy, quality, and readability of patient-focused breast cancer websites using expert evaluation and validated tools. Ensuring access to accurate, high-quality, and readable online health information supports informed decision-making and health equity but has not been recently evaluated. A qualitative analysis on 50 websites was conducted; the first 10 eligible websites for the following search terms were included: "breast cancer," "breast surgery," "breast reconstructive surgery," "breast chemotherapy," and "breast radiation therapy." Websites were required to be in English and not intended for healthcare professionals. Accuracy was evaluated by 5 breast cancer specialists. Quality was evaluated through the DISCERN questionnaire. Readability was measured using 9 standardized tests. Mean readability was compared with the American Medical Association and National Institutes of Health 6th grade recommendation. Nonprofit hospital websites had the highest accuracy (mean = 4.06, SD = 0.42); however, no statistical differences were observed in accuracy by website affiliation (P = 0.08). The overall mean quality score was 50.8 ("fair"/"good" quality) with no significant differences among website affiliations (P = 0.10). Mean readability was at the 10th grade reading level, the lowest being for commercial websites with a mean 9th grade reading level (SD = 2.38). All websites exceeded the American Medical Association- and National Institutes of Health-recommended reading level by 4.4 levels (P < 0.001). Websites with higher accuracy tended to have lower readability levels, whereas those with lower accuracy had higher readability levels. As breast cancer treatment has become increasingly complex, improving online quality and readability while maintaining high accuracy is essential to promote health equity and empower patients to make informed decisions about their care.

Readability of online patient education material for foregut surgery.

Health literacy is the ability of individuals to use basic health information and services to make well-informed decisions. Low health literacy among surgical patients has been associated with nonadherence to preoperative and/or discharge instructions as well as poor comprehension of surgery. It likely poses as a barrier to patients considering foregut surgery which requires an understanding of different treatment options and specific diet instructions. The objective of this study was to assess and compare the readability of online patient education materials (PEM) for foregut surgery. Using Google, the terms "anti-reflux surgery, "GERD surgery," and "foregut surgery" were searched and a total of 30 webpages from universities and national organizations were selected. The readability of the text was assessed with seven instruments: Flesch Reading Ease formula (FRE), Gunning Fog (GF), Flesch-Kincaid Grade Level (FKGL), Coleman Liau Index (CL), Simple Measure of Gobbledygook (SMOG), Automated Readability Index (ARI), and Linsear Write Formula (LWF). Mean readability scores were calculated with standard deviations. We performed a qualitative analysis gathering characteristics such as, type of information (preoperative or postoperative), organization, use of multimedia, inclusion of a version in another language. The overall average readability of the top PEM for foregut surgery was 12th grade. There was only one resource at the recommended sixth grade reading level. Nearly half of PEM included some form of multimedia. The American Medical Association and National Institute of Health have recommended that PEMs to be written at the 5th-6th grade level. The majority of online PEM for foregut surgery is above the recommended reading level. This may be a barrier for patients seeking foregut surgery. Surgeons should be aware of the potential gaps in understanding of their patients to help them make informed decisions and improve overall health outcomes.

Readability of Online Spanish Materials for Breast Reconstruction Using Deep Inferior Epigastric Perforator Flaps.

Background The internet has become an increasingly popular tool for patients to find information pertaining to medical procedures. Although the information is easily accessible, data shows that many online educational materials pertaining to surgical subspecialties are far above the average reading level in the United States. The aim of this study was to evaluate the English and Spanish online materials for the deep inferior epigastric perforator (DIEP) flap reconstruction procedure. Methods The first eight institutional or organizational websites that provided information on the DIEP procedure in English and Spanish were included. Each website was evaluated using the Patient Education and Materials Assessment Tool (PEMAT), Cultural Sensitivity Assessment Tool (CSAT), and either Simplified Measure of Gobbledygook (SMOG) for English websites or Spanish Orthographic Length (SOL) for Spanish websites. Results The English websites had a statistically lower CSAT score compared to the Spanish websites (p=0.006). However, Spanish websites had a statistically higher percentage of complex words compared to English sources (p<0.001). An analysis of reading grade levels through SMOG and SOL scores revealed that Spanish websites had statistically lower scores (p<0.001). There were no statistically significant differences in the understandability or actionability scores between the English and Spanish websites. Conclusions Online educational materials on the DIEP flap reconstruction procedure should be readable, understandable, actionable, and culturally sensitive. Our analysis revealed that improvements can be made in understandability and actionability on these websites.Plastic surgeons should be aware of what constitutes a great online educational resource and what online educational materials their patients will have access to.

Identifying the Careless Responders: A Cross-Country Comparison on PISA 2018 Dataset

Careless responding (CR) is one of the major concerns in self-report measures since it affects the quality of collected data. In this research, it was aimed to examine the dataset in terms of CR, to investigate the effects of CR on the psychometric properties of data, and to examine the effectiveness of CR detection methods in a cross-cultural context. Specifically; response time, outlier and long-string indices were used to flag CR and efficiency of using these were compared on PISA 2018 reading related attitude scales for Singapore and Turkey. As a result, it was indicated that the amount of CR was higher for Singapore than Turkey at long-string and response time methods. Furthermore, after removal of CR from dataset, enjoyment level of reading, perception of competence and perception of difficulty of PISA tests scores increased. Another result was that long-string and response time analyses provided significant improvement in model fit and long-string had the highest improvement level. Moreover, removing respondents according to outlier analyses didn’t provide any improvement on model-data fit. In general, results illustrated that careless responding behavior may have important effects on psychometric properties of self-report and screening the data for CR is strongly suggested in future studies.

LOOKING THROUGH THE LENS OF READING COMPREHENSION IN ENGLISH EDUCATION: CATCH-UP FRIDAYS IN FOCUS

The main purpose of the study was to recognize the individuality of each student’s learning approach and classified students based on their reading levels. This thoughtful categorization enabled the school to tailor activities, ensuring that every student could participate in meaningful learning experiences on Catch-Up Friday. Schools paused regular classes to immerse the entire school in a reading-focused day. This study provided an opportunity for each kid, regardless of reading proficiency, to have a personalized plan for the day centered on the joy of reading. Activities were planned to meet the children’ reading levels. For novices, comfortable reading areas with bright picture books were built. Intermediate readers engaged in group reading sessions, while advanced readers were assigned more difficult topics. Teachers gave support suited to each student’s reading level throughout the day. In addition to academic catch-up, Catch-Up Friday aims to promote a love of reading. The primary focus of this study was the elementary program teachers who participated in catch-up Friday activities. Each member teaches in a particular public school in Davao City. These participants have been teaching reading for more than five years, and they are presently teaching in the 2023–2024 school year. Ten people in all are included in this study and will participate in a concentrated group discussion and in-depth interview. The information obtained from the IDI and FGD is meticulously documented and converted into an organized summary of the participants' answers. Three themes emerged from the data analysis: the significance of reading comprehension, the need to adapt one's reading techniques, and the imposition of strong incentive. KEYWORDS: Learning, strategies, comprehension, elementary, teachers, reading, education, program, intervention

Using ChatGPT to Improve Readability of Interventional Radiology Procedure Descriptions.

This project examines ChatGPT's potential to enhance the readability of patient educational materials about interventional radiology (IR) procedures. The descriptions of IR procedures from the Cardiovascular and Interventional Radiological Society of Europe (CIRSE) were used as the original text. Readability scores were calculated using three metrics: Flesch Reading Ease (FRE), Gunning Fog (GF), and the Automated Readability Index (ARI) using an online calculator ( https://readabilityformulas.com ). FRE is scored on a scale of 0-100, where 100 indicates easy-to-read texts, and GF and ARI represent the grade level required to comprehend the text. The DISCERN instrument measured credibility and reliability. ChatGPT was prompted to simplify the texts to a fifth-grade reading level, with subsequent recalculation of readability and DISCERN scores for comparison. Statistical significance was determined using a Wilcoxon Signed-Rank Test. Articles were subsequently organized by subgroups and analyzed. 73 interventional radiology procedures from CIRSE were analyzed. The original FRE score was 47.2 (Difficult), improved to 78.4 (Fairly Easy) by ChatGPT. GF and ARI scores dropped from 14.4 and 11.2 to 7.8 and 5.8, respectively, after simplification, showing significant improvement (p < 0.001). However, the average DISCERN score decreased from 3.73 to 2.99 (p < 0.001) post-ChatGPT simplification. This study shows ChatGPT's ability to make interventional radiology descriptions more readable but highlights its struggle to maintain the original's reliability, suggesting the need for human review and prompt engineering to enhance outcomes. Level 6.

Developing Lexical Competence in Second-Year Middle School Learners

This article aims to highlight the usefulness of storytelling in vocabulary development for middle secondary school. The aim of this study is to improve the vocabulary skills of learner’s .In order to respond to the problem, an experiment is conducted in the classroom proposing a variety of stories .The results suggest that exposure to stories tailored to students’ reading levels can be beneficial in improving their lexical skills.

Is ChatGPT a Reliable Source of Patient Information on Asthma?

ChatGPT (OpenAI, San Francisco, CA, USA) is a novel artificial intelligence (AI) application that is used by millions of people, and the numbers are growing by the day. Because it has the potential to be a source of patient information, the study aimed to evaluate the ability of ChatGPT to answer frequently asked questions (FAQs) about asthma with consistent reliability, acceptability, and easy readability. We collected 30 FAQs about asthma from the Global Initiative for Asthma website. ChatGPT was asked each question twice, by two different users, to assess for consistency. The responses were evaluated by five board-certified internal medicine physicians for reliability and acceptability. The consistency of responses was determined by the differences in evaluation between the two answers to the same question. The readability of all responses was measured using the Flesch Reading Ease Scale (FRES), the Flesch-Kincaid Grade Level (FKGL), and the Simple Measure of Gobbledygook (SMOG). Sixty responses were collected for evaluation. Fifty-six (93.33%) of the responses were of good reliability. The average rating of the responses was 3.65 out of 4 total points. 78.3% (n=47) of the responses were found acceptable by the evaluators to be the only answer for an asthmatic patient. Only two (6.67%) of the 30 questions had inconsistent answers. The average readability of all responses was determined to be 33.50±14.37 on the FRES, 12.79±2.89 on the FKGL, and 13.47±2.38 on the SMOG. Compared to online websites, we found that ChatGPT can be a reliable and acceptable source of information for asthma patients in terms of information quality. However, all responses were of difficult readability, and none followed the recommended readability levels. Therefore, the readability of this AI application requires improvement to be more suitable for patients.

20 An Evaluation of the Quality and Readability of Online Information Regarding TMJ Dysfunction

Abstract Aim The accessibility and availability of the internet has allowed people to increasingly seek health information online, and therefore it plays an essential role in the decision-making process of the patient. The study aimed to assess the quality and readability of online health information on TMJ dysfunction. Method The search was conducted on Google using keywords: jaw pain, jaw locking, jaw discomfort, jaw clicking, jaw popping, temporomandibular joint, TMJ dysfunction, TMD and temporomandibular disorder. Ensuring Quality Information for Patient (EQIP) tool was used to assess quality of information and the application of readability score reflected through Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level(FKGL), Gunning-Fog Index(GFI) and Simple Measure of Gobbledygook(SMOG). Results 134 websites were eligible for assessment. The overall median EQIP score was 23, with an IQR of 7 and maximum score of 30. The most common source of information was healthcare industry (n=60; 45%), followed by news services (n=19; 14%), with a median EQIP score of 20 and 25 respectively. Statistically significant differences in overall EQIP scores were found between different sources of information (p=0.004). Only 31(23%) websites achieved the recommended 6th grade or below readability level, and only 14 (10%) discussed complication rates with none mentioning mortality rates at all. Conclusions Most online information on TMJ dysfunction is of suboptimal quality and readability. This study identified areas for improvements when producing patient material regarding TMJ dysfunction. To improve patient education and awareness, content creators should strive to be informative, accurate, and patient-centric.