6th Grade Level Research Articles

Enhancing Spanish Patient Education Materials: Comparing the Readability of Artificial Intelligence‐Generated Spanish Patient Education Materials to the Society of Pediatric Dermatology Spanish Patient Brochures

ABSTRACTPatient education materials (PEMs) are crucial for improving patient adherence and outcomes; however, they may not be accessible due to high reading levels. Our study used seven readability measures to compare the readability of Spanish PEMs from the Society of Pediatric Dermatology (SPD) with those generated by Open‐AI's ChatGPT 4.0 and Google Gemini. Our results showed that when prompted to produce material at a 6th grade level, both AI ChatBots generated significantly improved readability scores when compared to the SPD handouts. These findings suggest that AI‐generated PEMs could better meet readability standards and potentially improve patient outcomes, although further studies are needed to confirm this.

Assessing the Readability of Online English and Spanish Resources for Polydactyly and Syndactyly.

Online patient education materials (PEMs) that are difficult to read disproportionately affect patients with low health literacy and educational attainment. Patients may not be fully informed or empowered to engage meaningfully with providers and advocate for their goals. We aim to assess the readability of online PEMs regarding polydactyly and syndactyly. Google was used to query "polydactyly" and "syndactyly" in English and Spanish. The first 50 results were categorized into institutional (government, medical school, teaching hospital), noninstitutional (private practice, blog), and academic (journal articles, book chapters). Readability scores were generated using the Simple Measure of Gobbledygook and Spanish Simple Measure of Gobbledygook scales. All polydactyly PEMs and >95% of syndactyly PEMs exceeded the National Institutes of Health recommended 6th-grade reading level. Altogether, English PEMs had an average reading level of a university freshman and Spanish PEMs had an average reading level of nearly a high school sophomore. For both diagnoses, English PEMs were harder to read than Spanish PEMs overall and when compared across the 3 categories between the 2 languages. Generally, noninstitutional PEMs were more difficult to read than their institutional counterparts. To improve patient education, health literacy, and language equity, online resources for polydactyly and syndactyly should be written at the 6th-grade level. Currently, these PEMs are too advanced, which can make accessing, understanding, and pursuing healthcare decisions more challenging. Understanding health conditions and information is crucial to empower patients, regardless of literacy.

Evaluation of Hafıza-i Saltanat theater play and activities in terms of history teaching

The aim of this study is to evaluate the Hafıza-i Saltanat theatre play and activities, which combines evidence-based learning and history-themed theatre, written based on archival documents, in terms of history teaching. In this study, firstly, Hafıza-i Saltanat theatre play and its activities were introduced. Then, the play was examined in the context of historical reality, 10th grade history course curriculum achievements and textbooks, and the activities of the play were examined with student opinions in the context of age, level, knowledge, skills and 10th grade history course curriculum achievements. Document analysis and holistic single case study design, which are qualitative research methods, were used in the study. While the text of the theatre play, video recording and activities were evaluated through document analysis, watching the play and implementing the activities were handled as a holistic single case. The study group consisted of a total of 19 volunteer students, 6 boys and 13 girls, studying at the 10th grade level of secondary education. The data collection tools of the study were the text of the Hafıza-i Saltanat theatre play, the video recording of the play, the activities of the play, the video recordings of the application and the researcher's observation notes. The data obtained from the data collection tools were analyzed by descriptive analysis method. As a result of the study, it was determined that the archival documents used in the play were taken from the Mühimme Books, where the records of the Divan-ı Hümayun were kept, and the issues that were not included in the documents were included in the fiction by considering the historical facts, and for this reason, the play was written and performed largely in parallel with the historical reality. At the same time, it has been determined that the play and its activities meet and gain the achievements of units 4, 5, 6 and 7 of the 10th Grade History Course Teaching Program. It has been concluded that the use of Hafıza-i Sultanate play and activities in history lessons is appropriate for the specified achievements and that it supports the evidence-based learning approach and provides students with the necessary knowledge and skills.

Utilization of Artificial Intelligence to Improve Equitable Healthcare Access for Breast Implant Patients

Abstract The recently mandated US Food and Drug Administration (FDA) patient decision checklists were developed with the goal of improving the informed decision-making process for patients considering breast implants. Yet, prior research has demonstrated that these checklists are written at reading levels far in excess of that recommended by the National Institutes of Health and the American Medical Association. Patients considering breast implants are required to sign these forms prior to surgery, even if they have little to no comprehension of their contents. This study aims to improve the accessibility, and therefore the utility, of the mandated FDA patient literature using the assistance of artificial intelligence (AI). Patient decision checklists were obtained from the three most utilized breast implant manufacturers in the United States– Allergan, Mentor, and Sientra. AI was then asked to synthesize a novel patient decision checklist, written at the 6th grade reading level, using these checklists as source material. The AI-assisted checklist was edited by plastic surgeons for both formatting and content. Readability analysis of all documents was performed using the Flesch Reading-Ease Score, Flesch-Kincaid Grade Level, Gunning-Fog Index, Coleman-Liau Index, Simplified Measure of Gobbledygook, and Automated Readability Index. Textual analysis included sentence count, word count, number and percentage of complex words, average number of words per sentence, and average number of syllables per word. The overall readability of Allergan, Mentor, and Sientra patient checklists correlated with the college reading level. These documents were of a statistically significantly higher reading level the AI-assisted checklist, which was written at the recommended 6th grade level. Text composition analysis similarly demonstrated substantial differences between the AI-assisted and FDA mandated literature. The currently mandated breast implant patient checklists are written at a college reading level and are inaccessible to the average patient. We propose a new patient decision checklist, generated with the assistance of artificial intelligence, in order to improve healthcare access within plastic surgery. This simplified material can be used as an adjunct to the current checklists in order to improve shared decision making.

Investigating the capabilities of advanced large language models in generating patient instructions and patient educational material

ObjectivesLarge language models (LLMs) with advanced language generation capabilities have the potential to enhance patient interactions. This study evaluates the effectiveness of ChatGPT 4.0 and Gemini 1.0 Pro in providing...

Bridging Gaps in Health Literacy for Cleft Lip and Palate: The Role of Artificial Intelligence and Interactive Educational Materials.

Our goal is to address the health literacy gap concerning cleft lip and palate using artificial intelligence (AI), specifically OpenAI's ChatGPT-4, to enhance the readability of patient educational materials (PEMs). Additionally, we aim to explore the integration of interactive educational materials into these resources. A cross-sectional study. Websites of craniofacial teams approved by the American Cleft Palate-Craniofacial Association (ACPA). PEMs collected from these websites. First, we examined the PEMs for interactive educational content such as videos, photos, case examples, and audio clips. Subsequently, we used the WebFX online readability calculator and Grammarly to evaluate the texts' readability and clarity. After that, we used ChatGPT-4 to simplify the PEM texts to a 6th-grade reading level and re-evaluated their readability and clarity. Readability levels (grade level) and clarity scores of PEMs before and after AI modification, and the presence of interactive educational content. Initial evaluations using the Flesch-Kincaid Grade Level formula showed that the texts were written at a median 9th-grade reading level. Modifications using ChatGPT-4 significantly improved readability, achieving a median 6th-grade level and enhanced clarity. Only 27% of the websites featured interactive educational content, indicating a gap in utilizing educational enhancements. The use of AI improved the readability and clarity of patient educational materials, aligning with recommended standards. However, the underutilization of interactive tools suggests a need for broader integration of these resources to optimize patient education and engagement.

Evaluating the readability of online patient information for cholangiocarcinoma.

264 Background: The digital age has facilitated the dissemination of free comprehensive information regarding the diagnosis and treatment of various cancers. Especially for rarer cancers, such as cholangiocarcinoma, patients heavily rely on online patient information (OPI) as a primary source of information. However, the usefulness of these websites is dependent on the readability of the information. The National Institute of Health (NIH) recommends OPI have a readability level of grade 8 or lower. We sought to assess the readability of OPI for cholangiocarcinoma. Methods: We utilized Google to identify the top ten websites utilizing the search query "cholangiocarcinoma". The readability score of each website was defined as the median of ten popular standardized readability scales - Flesch–Kincaid, Gunning Fog, Coleman–Liau, Simple Measure of Gobbledygook, Automated Readability Index, FORCAST, Raygor, Fry Graph, and New Dale Chall. These readability scales were obtained using www.readable.com. Group comparisons of readability score were performed for affiliation (i.e., academic, non-profit, or government) and website content (i.e., treatment modalities, complications, clinical trials, supportive care, and survivorship). Statistical analysis was done in RStudio. Pairwise comparisons were performed using the Wilcoxon rank sum test. Results: A total of 10 websites were analyzed with 4/10 (40%) affiliated with academic institutions, 3/10 (30%) affiliated with non-profit organization, and 3/10 (30%) affiliated with government websites. The median readability score for all 10 websites was 9.95 (IQR 9.15-10.75). The majority of the websites (9/10, 90%) websites had readability scores that did not satisfy the NIH recommendation for readability. Readability scores did not differ significantly between academic, government, and nonprofit websites (p=0.68). Readability also did not differ by website content. The correlation between word count and readability score was not statistically significant (r = 0.03, p = 0.92). Conclusions: We performed a first of its kind analysis to assess the readability of OPI for cholangiocarcinoma. The median readability of OPI for cholangiocarcinoma was at a 10th grade level and the majority of the websites did not satisfy the NIH guideline for readability. Surprisingly, government websites were not simpler to read than their academic and nonprofit counterparts. Our study underscores the need to develop OPI for cholangiocarcinoma that is easily readable for patients and their caregivers. Organizations publishing OPI may use readability scores to ensure their content is accessible for the general population.

Accuracy and Readability of Artificial Intelligence Chatbot Responses to Vasectomy-Related Questions: Public Beware.

Purpose Artificial intelligence (AI) has rapidly gained popularity with the growth of ChatGPT (OpenAI, San Francisco, USA) and other large-language model chatbots, and these programs have tremendous potential to impact medicine. One important area of consequence in medicine and public health is that patients may use these programs in search of answers to medical questions. Despite the increased utilization of AI chatbots by the public, there is little research to assess the reliability of ChatGPT and alternative programs when queried for medical information. This study seeks to elucidate the accuracy and readability of AI chatbots in answering patient questions regarding urology. As vasectomy is one of the most common urologic procedures, this study investigates AI-generated responses to frequently asked vasectomy-related questions. For this study, five popular and free-to-access AI platforms were utilized to undertake this investigation. Methods Fifteen vasectomy-related questions were individually queried to five AI chatbots from November-December 2023: ChatGPT (OpenAI, San Francisco, USA), Bard (Google Inc., Mountainview, USA) Bing (Microsoft, Redmond, USA) Perplexity (Perplexity AI Inc., San Francisco, USA), and Claude (Anthropic, San Francisco, USA). Responses from each platform were graded by two attending urologists, two urology research faculty, and one urological resident physician using a Likert (1-6) scale: (1-completely inaccurate, 6-completely accurate) based on comparison to existing American Urological Association guidelines. Flesch-Kincaid Grade levels (FKGL) and Flesch Reading Ease scores (FRES) (1-100) were calculated for each response. To assess differences in Likert, FRES, and FKGL, Kruskal-Wallis tests were performed using GraphPad Prism V10.1.0 (GraphPad,San Diego,USA) with Alpha set at 0.05. Results Analysis shows that ChatGPT provided the most accurate responses across the five AI chatbots with an average score of 5.04 on the Likert scale. Subsequently, Microsoft Bing (4.91), Anthropic Claude (4.65), Google Bard (4.43), and Perplexity (4.41) followed. All five chatbots were found to score, on average, higher than 4.41 corresponding to a score of at least "somewhat accurate." Google Bard received the highest Flesch Reading Ease score (49.67) and lowest Grade level (10.1) when compared to the other chatbots. Anthropic Claude scored 46.7 on the FRES and 10.55 on the FKGL. Microsoft Bing scored 45.57 on the FRES and 11.56 on the FKGL. Perplexity scored 36.4 on the FRES and 13.29 on the FKGL. ChatGPT had the lowest FRES of 30.4 and highest FKGL of 14.2. Conclusion This study investigates the use of AI in medicine, specifically urology, and it helps to determine whether large-language model chatbots can be reliable sources of freely available medical information. All five AI chatbots on average were able to achieve at least "somewhat accurate" on a 6-point Likert scale. In terms of readability, all five AI chatbots on average had Flesch Reading Ease scores of less than 50 and were higher than a 10th-grade level. In this small-scale study, there were several significant differences identified between the readability scores of each AI chatbot. However, there were no significant differences found among their accuracies. Thus, our study suggests that major AI chatbots may perform similarly in their ability to be correct but differ in their ease of being comprehended by the general public.

Improving readability and comprehension levels of otolaryngology patient education materials using ChatGPT

ObjectiveA publicly available large language learning model platform may help determine current readability levels of otolaryngology patient education materials, as well as translate these materials to the recommended 6th-grade and 8th-grade reading levels. Study designCross-sectional analysis. SettingOnline using large language learning model, ChatGPT. MethodsThe Patient Education pages of the American Laryngological Association (ALA) and American Academy of Otolaryngology-Head and Neck Surgery (AAO-HNS) websites were accessed. Materials were input into ChatGPT (OpenAI, San Francisco, CA; version 3.5) and Microsoft Word (Microsoft, Redmond, WA; version 16.74). Programs calculated Flesch Reading Ease (FRE) scores, with higher scores indicating easier readability, and Flesch-Kincaid (FK) grade levels, estimating U.S. grade level required to understand text. ChatGPT was prompted to “translate to a 5th-grade reading level” and provide new scores. Scores were compared for statistical differences, as well as differences between ChatGPT and Word gradings. ResultsPatient education materials were reviewed and 37 ALA and 72 AAO-HNS topics were translated. Overall FRE scores and FK grades demonstrated significant improvements following translation of materials, as scored by ChatGPT (p < 0.001). Word also scored significant improvements in FRE and FK following translation by ChatGPT for AAO-HNS materials overall (p < 0.001) but not for individual topics or for subspecialty-specific categories. Compared with Word, ChatGPT significantly exaggerated the change in FRE grades and FK scores (p < 0.001). ConclusionOtolaryngology patient education materials were found to be written at higher reading levels than recommended. Artificial intelligence may prove to be a useful resource to simplify content to make it more accessible to patients.

Assessing the Readability, Reliability, and Quality of AI-Modified and Generated Patient Education Materials for Endoscopic Skull Base Surgery.

Despite National Institutes of Health and American Medical Association recommendations to publish online patient education materials at or below sixth-grade literacy, those pertaining to endoscopic skull base surgery (ESBS) have lacked readability and quality. ChatGPT is an artificial intelligence (AI) system capable of synthesizing vast internet data to generate responses to user queries but its utility in improving patient education materials has not been explored. To examine the current state of readability and quality of online patient education materials and determined the utility of ChatGPT for improving articles and generating patient education materials. An article search was performed utilizing 10 different search terms related to ESBS. The ten least readable existing patient-facing articles were modified with ChatGPT and iterative queries were used to generate an article de novo. The Flesch Reading Ease (FRE) and related metrics measured overall readability and content literacy level, while DISCERN assessed article reliability and quality. Sixty-six articles were located. ChatGPT improved FRE readability of the 10 least readable online articles (19.7 ± 4.4 vs. 56.9 ± 5.9, p < 0.001), from university to 10th grade level. The generated article was more readable than 48.5% of articles (38.9 vs. 39.4 ± 12.4) and higher quality than 94% (51.0 vs. 37.6 ± 6.1). 56.7% of the online articles had "poor" quality. ChatGPT improves the readability of articles, though most still remain above the recommended literacy level for patient education materials. With iterative queries, ChatGPT can generate more reliable and higher quality patient education materials compared to most existing online articles and can be tailored to match readability of average online articles.

Designing Patient-Reported Measures of Fertility: Cognitive Interview Findings from Adolescents and Young Adults with Cancer.

Purpose: Fertility concerns (FC) are central to the well-being of many adolescent and young adult (AYA) cancer survivors. Clinical conversations about FC and fertility preservation are suboptimal, increasing patient distress. The goal of this project was to establish content validity and comprehensibility of self-report questions on FCs for AYAs with cancer. Methods: Following best practices, we conducted: (1) item identification, refinement, and generation; (2) translatability and reading level review; and (3) cognitive interviews. Items were reviewed by five AYAs in each round of cognitive interviews. Results: A systematic search yielded 63 measures and 873 items. Fifty items were subsequently modified to enhance clarity and relevance, representing subdomains of psychological and social/relational FC. Flesch-Kincaid analysis found 31 items written above the 6th grade level, which were subsequently revised. Translatability review resulted in the modification of 3 items. During cognitive interviews, 76% of AYAs found items easy to answer with 52% describing them as "very easy" and 24% as "somewhat easy." Sixty percent of participants indicated the items captured their experiences. The majority of those who reported items only somewhat reflected or did not reflect their experiences suggested items were simply not applicable for their particular case. Conclusion: This study is a critical step toward the foundation for an FC measurement system that is reliable, flexible, developmentally appropriate, comprehensible, translatable, and interpretable. Subsequent steps include psychometric testing to examine the construct validity and reliability of the FC items and calibration to enable the application of computer-adaptive testing and short form development. The evaluation will include potential item response bias by age range, gender identity, and race/ethnicity.

Assessment of readability, reliability, and quality of ChatGPT®, BARD®, Gemini®, Copilot®, Perplexity® responses on palliative care.

There is no study that comprehensively evaluates data on the readability and quality of "palliative care" information provided by artificial intelligence (AI) chatbots ChatGPT®, Bard®, Gemini®, Copilot®, Perplexity®. Our study is an observational and cross-sectional original research study. In our study, AI chatbots ChatGPT®, Bard®, Gemini®, Copilot®, and Perplexity® were asked to present the answers of the 100 questions most frequently asked by patients about palliative care. Responses from each 5 AI chatbots were analyzed separately. This study did not involve any human participants. Study results revealed significant differences between the readability assessments of responses from all 5 AI chatbots (P < .05). According to the results of our study, when different readability indexes were evaluated holistically, the readability of AI chatbot responses was evaluated as Bard®, Copilot®, Perplexity®, ChatGPT®, Gemini®, from easy to difficult (P < .05). In our study, the median readability indexes of each of the 5 AI chatbots Bard®, Copilot®, Perplexity®, ChatGPT®, Gemini® responses were compared to the "recommended" 6th grade reading level. According to the results of our study answers of all 5 AI chatbots were compared with the 6th grade reading level, statistically significant differences were observed in the all formulas (P < .001). The answers of all 5 artificial intelligence robots were determined to be at an educational level well above the 6th grade level. The modified DISCERN and Journal of American Medical Association scores was found to be the highest in Perplexity® (P < .001). Gemini® responses were found to have the highest Global Quality Scale score (P < .001). It is emphasized that patient education materials should have a readability level of 6th grade level. Of the 5 AI chatbots whose answers about palliative care were evaluated, Bard®, Copilot®, Perplexity®, ChatGPT®, Gemini®, their current answers were found to be well above the recommended levels in terms of readability of text content. Text content quality assessment scores are also low. Both the quality and readability of texts should be brought to appropriate recommended limits.

The utility of artificial intelligence platforms for patient-generated questions in Mohs micrographic surgery: amulti-national, blinded expert panel evaluation.

Artificial intelligence (AI) and large language models (LLMs) transform how patients inform themselves. LLMs offer potential as educational tools, but their quality depends upon the information generated. Current literature examining AI as an informational tool in dermatology has been limited in evaluating AI's multifaceted roles and diversity of opinions. Here, we evaluate LLMs as a patient-educational tool for Mohs micrographic surgery (MMS) in and out of the clinic utilizing an international expert panel. The most common patient MMS questions were extracted from Google and transposed into two LLMs and Google's search engine. 15 MMS surgeons evaluated the generated responses, examining their appropriateness as a patient-facing informational platform, sufficiency of response in a clinical environment, and accuracy of content generated. Validated scales were employed to assess the comprehensibility of each response. The majority of reviewers deemed all LLM responses appropriate. 75% of responses were rated as mostly accurate or higher. ChatGPT had the highest mean accuracy. The majority of the panel deemed 33% of responses sufficient for clinical practice. The mean comprehensibility scores for all platforms indicated a required 10th-grade reading level. LLM-generated responses were rated as appropriate patient informational sources and mostly accurate in their content. However, these platforms may not provide sufficient information to function in a clinical environment, and complex comprehensibility may represent a barrier to utilization. As the popularity of these platforms increases, it is important for dermatologists to be aware of these limitations.

Evaluating Health Literacy Resources for Secondary Hyperparathyroidism in End-Stage Kidney Disease

IntroductionParathyroidectomy is recommended for severe secondary hyperparathyroidism (SHPT) due to end-stage kidney disease (ESKD), but surgery is underutilized. High quality and accessible online health information, recommended to be at a 6th-grade reading level, is vital to improve patient health literacy. This study evaluated available online resources for SHPT from ESKD based on information quality and readability. MethodsThree search engines were queried using the terms “parathyroidectomy for secondary hyperparathyroidism,” “parathyroidectomy kidney/renal failure,” “parathyroidectomy dialysis patients,” “should I have surgery for hyperparathyroidism due to kidney failure?,” and “do I need surgery for hyperparathyroidism due to kidney failure if I do not have symptoms?” Websites were categorized by source and origin. Two independent reviewers determined information quality using JAMA (0-4) and DISCERN (1-5) frameworks, and scores were averaged. Cohen's kappa evaluated inter-rater reliability. Readability was determined using the Flesch Kincaid Reading Ease, Flesch Kincaid Grade Level, and Simple Measure of Gobbledygook tools. Median readability scores were calculated, and corresponding grade level determined. Websites with reading difficulties >6th grade level were calculated. ResultsThirty one (86.1%) websites originated from the U.S., with most from hospital-associated (63.9%) and foundation/advocacy sources (30.6%). The mean JAMA and DISCERN scores for all websites were 1.3 ± 1.4 and 2.6 ± 0.7, respectively. Readability scores ranged from grade level 5–college level, and most websites scored above the recommended 6th grade level. ConclusionsPatient-oriented websites tailoring SHPT from ESKD are at a reading level higher than recommended, and the quality of information is low. Efforts must be made to improve the accessibility and quality of information for all patients.

Online information about oral health in autism spectrum disorder: Is it good enough?

The use of the internet has surged significantly over the years. Patients and caregivers of patients with autism spectrum disorder (ASD) might consult the internet for oral health-related information. Hence, this study aimed to assess the quality and readability of online information available in the English language regarding oral health in ASD. Online search using Google.com was conducted using the terms "Autism and dental care," "Autism and oral health," and "Autism and dentistry". The first 100 websites for each term were screened. Quality of information was assessed using the Patient Education Materials Assessment Tool for printed material (PEMAT-P) and the Journal of American Medical Association (JAMA) benchmarks. A PEMAT score higher than 70% is considered acceptable for readability and actionability. The JAMA benchmarks are authorship, attribution, disclosure, and currency. Readability was evaluated using the Flesch reading ease score and Simple Measure of Gobbledygook (SMOG) readability formula. Out of the 300 screened websites, 66 were eventually included. The mean PEMAT understandability and actionability scores were 77.13%, and 42.12%, respectively. Only 12.1% of the websites displayed all four JAMA benchmarks. The mean Flesch score was 10th-12th grade level, and the mean SMOG score was 10th grade level. While the understandability of the information was acceptable, the readability and actionability were too challenging for lay people. Health care professionals and organizations involved in patient education should place more efforts in promoting the quality of online information targeting patients with ASD.

Assessing readability of online patient educational material on concussion and return to play.

Concussions are self-limited forms of mild traumatic brain injury (TBI). Gradual return to play (RTP) is crucial to minimizing the risk of second impact syndrome. Online patient educational materials (OPEM) are often used to guide decision-making. Previous literature has reported that grade-level readability of OPEM is higher than recommended by the American Medical Association and the National Institutes of Health. The authors evaluated the readability of OPEM on concussion and RTP. An online search engine was used to identify websites providing OPEM on concussion and RTP. Text specific to concussion and RTP was extracted from each website and readability was assessed using the following six standardized indices: Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level, Gunning Fog Index, Coleman-Liau Index, Simple Measure of Gobbledygook Index, and Automated Readability Index. One-way ANOVA and Tukey's post hoc test were used to compare readability across sources of information. There were 59 concussion and RTP articles, and readability levels exceeded the recommended 6th grade level, irrespective of the source of information. Academic institutions published OPEM at simpler readability levels (higher FRE scores). Private organizations published OPEM at more complex (higher) grade-level readability levels in comparison with academic and nonprofit institutions (p < 0.05). The readability of OPEM on RTP after concussions exceeds the literacy of the average American. There is a critical need to modify the concussion and RTP OPEM to improve comprehension by a broad audience.

Assessing the readability, reliability, and quality of artificial intelligence chatbot responses to the 100 most searched queries about cardiopulmonary resuscitation: An observational study.

This study aimed to evaluate the readability, reliability, and quality of responses by 4 selected artificial intelligence (AI)-based large language model (LLM) chatbots to questions related to cardiopulmonary resuscitation (CPR). This was a cross-sectional study. Responses to the 100 most frequently asked questions about CPR by 4 selected chatbots (ChatGPT-3.5 [Open AI], Google Bard [Google AI], Google Gemini [Google AI], and Perplexity [Perplexity AI]) were analyzed for readability, reliability, and quality. The chatbots were asked the following question: "What are the 100 most frequently asked questions about cardio pulmonary resuscitation?" in English. Each of the 100 queries derived from the responses was individually posed to the 4 chatbots. The 400 responses or patient education materials (PEM) from the chatbots were assessed for quality and reliability using the modified DISCERN Questionnaire, Journal of the American Medical Association and Global Quality Score. Readability assessment utilized 2 different calculators, which computed readability scores independently using metrics such as Flesch Reading Ease Score, Flesch-Kincaid Grade Level, Simple Measure of Gobbledygook, Gunning Fog Readability and Automated Readability Index. Analyzed 100 responses from each of the 4 chatbots. When the readability values of the median results obtained from Calculators 1 and 2 were compared with the 6th-grade reading level, there was a highly significant difference between the groups (P < .001). Compared to all formulas, the readability level of the responses was above 6th grade. It can be seen that the order of readability from easy to difficult is Bard, Perplexity, Gemini, and ChatGPT-3.5. The readability of the text content provided by all 4 chatbots was found to be above the 6th-grade level. We believe that enhancing the quality, reliability, and readability of PEMs will lead to easier understanding by readers and more accurate performance of CPR. So, patients who receive bystander CPR may experience an increased likelihood of survival.

Evaluating the accuracy and readability of ChatGPT in providing parental guidance for adenoidectomy, tonsillectomy, and ventilation tube insertion surgery

ObjectivesThis study examined the potential of ChatGPT as an accurate and readable source of information for parents seeking guidance on adenoidectomy, tonsillectomy, and ventilation tube insertion surgeries (ATVtis). MethodsChatGPT was tasked with identifying the top 15 most frequently asked questions by parents on internet search engines for each of the three specific surgical procedures. We removed repeated questions from the initial set of 45. Subsequently, we asked ChatGPT to generate answers to the remaining 33 questions. Seven highly experienced otolaryngologists individually assessed the accuracy of the responses using a four-level grading scale, from completely incorrect to comprehensive. The readability of responses was determined using the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) scores. The questions were categorized into four groups: Diagnosis and Preparation Process, Surgical Information, Risks and Complications, and Postoperative Process. Responses were then compared based on accuracy grade, FRE, and FKGL scores. ResultsSeven evaluators each assessed 33 AI-generated responses, providing a total of 231 evaluations. Among the evaluated responses, 167 (72.3 %) were classified as ‘comprehensive.’ Sixty-two responses (26.8 %) were categorized as ‘correct but inadequate,’ and two responses (0.9 %) were assessed as ‘some correct, some incorrect.’ None of the responses were adjudged ‘completely incorrect’ by any assessors. The average FRE and FGKL scores were 57.15(±10.73) and 9.95(±1.91), respectively. Upon analyzing the responses from ChatGPT, 3 (9.1 %) were at or below the sixth-grade reading level recommended by the American Medical Association (AMA). No significant differences were found between the groups regarding readability and accuracy scores (p > 0.05). ConclusionsChatGPT can provide accurate answers to questions on various topics related to ATVtis. However, ChatGPT's answers may be too complex for some readers, as they are generally written at a high school level. This is above the sixth-grade reading level recommended for patient information by the AMA. According to our study, more than three-quarters of the AI-generated responses were at or above the 10th-grade reading level, raising concerns about the ChatGPT text's readability.

Readability Analysis of Spanish-Language Patient-Reported Outcome Measures in Orthopaedic Surgery.

Patient-reported outcome measures (PROMs) are increasingly used to quantify patient symptomatology when assessing intervention effectiveness as well as to guide patient-centered health-care management and research. Previous studies have analyzed the readability of PROMs used in orthopaedics in the English language; however, the readability of PROMs used in orthopaedics in the Spanish language has not been evaluated. Given the increasing number of Spanish-speaking individuals seeking orthopaedic care in the United States, it is imperative that PROMs are at an adequate reading level for patients to answer appropriately in their native language. This study aimed to assess the readability of publicly available Spanish-language PROMs used in orthopaedics. Searches of the PubMed, Google Scholar, and Embase databases were conducted to identify publicly available Spanish-language versions of PROMs used in orthopaedics. Additionally, the PROMIS Health Organization was contacted to obtain Spanish-language versions of the PROMIS (Patient-Reported Outcomes Measurement Information System) questionnaires. A total of 42 Spanish-language PROMs were identified and included in this study. The text of each PROM was inserted into multilingual readability software (legible.es), which analyzed the readability of each PROM using the Fernández Huerta and Índice de Legibilidad de Flesch-Szigriszt (INFLESZ) readability formulas. The mean and standard deviation (SD) of the raw readability scores were calculated for the PROMs. The percentage of PROMs at or below the 6th-grade reading level was also calculated. The mean readability of PROMs using the INFLESZ formula was at a 7th to 10th-grade reading level (mean = 63, SD = 16), and the mean readability using the Fernández Huerta formula was 68 (SD = 15). On average, 57% of the PROMs used in orthopaedics were at or below the INFLESZ 6th-grade readability level. Approximately half of Spanish-language PROMs used in orthopaedics are written at a reading level that does not meet the National Institutes of Health and the American Medical Association recommendations at or below the 6th-grade level. PROM developers and translators should consider the importance of readability when translating PROMs, to maximize their use and efficacy among orthopaedic patients speaking that language.

How artificial intelligence can provide information about subdural hematoma: Assessment of readability, reliability, and quality of ChatGPT, BARD, and perplexity responses.

Subdural hematoma is defined as blood collection in the subdural space between the dura mater and arachnoid. Subdural hematoma is a condition that neurosurgeons frequently encounter and has acute, subacute and chronic forms. The incidence in adults is reported to be 1.72-20.60/100.000 people annually. Our study aimed to evaluate the quality, reliability and readability of the answers to questions asked to ChatGPT, Bard, and perplexity about "Subdural Hematoma." In this observational and cross-sectional study, we asked ChatGPT, Bard, and perplexity to provide the 100 most frequently asked questions about "Subdural Hematoma" separately. Responses from both chatbots were analyzed separately for readability, quality, reliability and adequacy. When the median readability scores of ChatGPT, Bard, and perplexity answers were compared with the sixth-grade reading level, a statistically significant difference was observed in all formulas (P < .001). All 3 chatbot responses were found to be difficult to read. Bard responses were more readable than ChatGPT's (P < .001) and perplexity's (P < .001) responses for all scores evaluated. Although there were differences between the results of the evaluated calculators, perplexity's answers were determined to be more readable than ChatGPT's answers (P < .05). Bard answers were determined to have the best GQS scores (P < .001). Perplexity responses had the best Journal of American Medical Association and modified DISCERN scores (P < .001). ChatGPT, Bard, and perplexity's current capabilities are inadequate in terms of quality and readability of "Subdural Hematoma" related text content. The readability standard for patient education materials as determined by the American Medical Association, National Institutes of Health, and the United States Department of Health and Human Services is at or below grade 6. The readability levels of the responses of artificial intelligence applications such as ChatGPT, Bard, and perplexity are significantly higher than the recommended 6th grade level.