Grade Reading Research Articles

Can Generative Artificial Intelligence Enhance Health Literacy About Lateral Epicondylitis?

Health literacy is a critical determinant of a patient's overall health status, and studies have demonstrated a consistent link between poor health literacy and negative health outcomes. The Centers for Disease Control and Prevention (CDC) and the National Institutes of Health (NIH) advise that patient educational materials (PEMs) should be written at an eighth-grade reading level or lower, matching the average reading level of adult Americans. The purpose of this study was to investigate the ability of generative artificial intelligence (AI) to edit PEMs from orthopaedic institutions to meet the CDC and NIH guidelines. PEMs about lateral epicondylitis (LE) from the top 25 ranked orthopaedic institutions from the 2022 U.S. News & World Report Best Hospitals Specialty Ranking were gathered. ChatGPT Plus (version 4.0) was then instructed to rewrite PEMs on LE from these institutions to comply with CDC and NIH-recommended guidelines. Readability scores were calculated for the original and rewritten PEMs, and paired t-tests were used to determine statistical significance. Analysis of the original and edited PEMs about LE revealed significant reductions in reading grade level and word count of 3.70 ± 1.84 (p<0.001) and 346.72 ± 364.63 (p<0.001), respectively. Our study demonstrated generative AI's ability to rewrite PEM about LE at a reading comprehension level that conforms to the CDC and NIH guidelines. Hospital administrators and orthopaedic surgeons should consider the findings of this studyand the potential utility of artificial intelligence when crafting PEMs of their own.

Comparative Readability Assessment of Four Large Language Models in Answers to Common Contraception Questions [ID 2683638

INTRODUCTION: Large language models (LLMs) have the potential to revolutionize reproductive health education by offering accessible contraception counseling. The CDC, NIH, and AMA recommend presenting patient educational materials at or below a sixth-grade reading level. In this study, we present a comparative analysis of the readability of responses generated by LLMs, including OpenAI's ChatGPT 4.0, OpenAI's ChatGPT 3.5, Google Bard, and Microsoft Bing, to common contraception questions. METHODS: Referencing common contraception questions outlined in a recent review, we presented GPT-4.0, GPT-3.5, Google Bard, and Microsoft Bing with six common contraception questions on June 10, 2023, ensuring no prior chat bias. The readability of LLM outputs was evaluated using Flesch–Kincaid Grade Level (FK), Gunning Fog (GF), Automated Readability Index (ARI), and Coleman–Liau (CL) indices. All index score outputs correspond to a reading grade level (RGL), eg, RGL of 6 represents a sixth-grade level. RESULTS: Of the four LLMs, Google Bard demonstrated the highest readability with an average RGL of 10.6 (GF 10.1, FK 9.6, ARI 11.4, CL 11.4), followed by GPT-3.5 at an average RGL of 13.6 (GF 13.3, FK 12.6, ARI 15.2, CL 16.5). Microsoft Bing and GPT-4.0 yielded responses with higher average RGLs of 14.2 and 15.4, respectively. CONCLUSION: Google Bard and GPT-3.5 offered the most readable responses to contraception questions; however, these models still provided responses significantly above the recommended sixth-grade reading level (RGL of 6). Large language models should undergo evaluation to ensure alignment with established readability standards to optimize patient comprehension in reproductive counseling applications.

Representing Rural: A Critical Content Analysis of Contemporary Middle Grade Novels Set in Rural Places

Children’s literature contains shared meanings that not only reflect societal norms, but also reinstate and reconstitute societal norms. This study used critical content analysis methods grounded in place theory to analyze the textual constructions of rurality in 52 contemporary, middle grade, realistic fiction novels set in US rural places. Findings revealed five salient themes, three of which are discussed in this article: systems work to keep rural people in poverty; rural people have deep connections to place; and rural people have diverse, intersectional identities. While some middle grade books in the sample move toward challenging stereotypes of rural places as monolithic (e.g., White-majority, socially conservative) by including nuanced portrayals of some characters of color, LGBTQ+ characters, and characters with disabilities, others rely on simplistic and otherwise problematic representations, using familiar tropes about rural people that suggest racial and cultural homogeneity privileging Whiteness and making invisible BIPOC in rural communities. Given the powerful impact of stories on identity formation and sensemaking, this study analyzes textual representations of rural people and places in books for middle grade readers.

Examining the effects of prekindergarten to second grade classroom quality on early elementary achievement

Previous research has investigated the influence of preschool interventions in early elementary environments with some results suggesting that high quality early preschool experiences may have meaningful effects on student academic outcomes not only immediately following the preschool years but also in the long term. This study aimed to investigate this topic further by using a novel cross-classified model to account for the complex hierarchical structure of a large-scale longitudinal and multilevel dataset. First, we examined the carryover classroom effects from prekindergarten (PK), kindergarten (K), and first grade on later grades’ reading and mathematics scores. Second, we explored the effect of PK to second grade classroom quality on students’ K, first, and second grade reading and mathematics scores. Our analyses yielded significant PK carryover classroom effects for reading; these results provide evidence that a student’s PK classroom continues to impact that student for at least three years after. Additionally, one domain of PK classroom quality —classroom organization—was found to be positively related to students’ K, first, and second grade reading outcomes.

PENGGUNAAN MEDIA KARTU SUKU KATA BERGAMBAR UNTUK MENINGKATKAN KEMAMPUAN MEMBACA PERMULAAN SISWA KELAS II SDK WOLOMELI

This research was motivated by the low reading ability of class II students at SDK Wolomeli. Learning to read in lower grades is the initial foundation for the development of science. Learning with syllable card media is considered capable of facilitating and strengthening students' memories, increasing insight and skills, attracting students' interest in recognizing several syllables so that students can respond to the meaning of images as a support for students' imagination which provides a connection between the content of learning material and the world. virtual reality through picture imagery, so that students' abilities can develop. This research aims to improve beginning reading skills using illustrated syllable card media for class II students at SDK Wolomeli. The research carried out was classroom action research which cycled through four stages, namely planning, implementation, observation and reflection with the research subjects, namely 13 class II students at SDK Wolomeli, consisting of 5 womens and 8 mens. The results obtained are based on the results of the cycle I and cycle II tests. Students' ability in initial reading has increased. The first cycle of class II students obtained a result of 38% and the second cycle test obtained a result of 71% and there was an increase of 33% so that the level of students' reading ability was included in the quite good category.

Design and Validation of Structural Causal Model: A focus on SENSE-EGRA Datasets

Designing and validation of causal model correctness from a dataset whose background knowledge is gotten from a research process is not a common phenomenon. In fact, studies have shown that in many critical areas such as healthcare and education, researchers develop models from direct acyclic graphs without testing them. This phenomenon is worrisome and is bound to cast a dark shadow on the inference estimates that many arise from such models. In this study, we have design a novel application-based SCM for the first time using the background knowledge gotten from the American university of Nigeria (AUN), Yola, on the letter identification subtask of Early Grade reading Assessment (EGRA) program on Strengthen Education in Northeast Nigeria (SENSE-EGRA) project dataset, which was sponsored by the USAID. We employed the conditional independence test (CIT) criteria for the model’s correctness validation testing, and the results shows a near perfect SCM.

The Effectiveness of Early Grade Reading Assessment (EGRA) Method on Elementary Students’ Reading Fluency

Along with the evolution of time, individuals were required to learn quickly, especially in mastering new things. Therefore, it became the main reason reading is considered a fundamental skill that should be mastered from a young age. Several methods have been created to enhance reading fluency, including the Early Grade Reading Assessment (EGRA). This research aims to determine the EGRA method's effectiveness on students' reading fluency. A quantitative approach with a quasi-experimental method and a nonrandomized pretest-posttest control group design was employed. Thirty-seven fourth-grade elementary school students at SDN X Cikarang Regency participated. Eight (8) elements of EGRA were used as treatment materials, namely: (a) letter identification; (b) distinguishing initial sound; (c) nonword reading; (d) read aloud/oral reading; (e) reading comprehension; (f) listening comprehension; (g) Indonesian vocabulary; and (h) dictation. As for analyzing the data, the researcher employed Mann-Whitney U to compare the pretest and posttest scores of the control and experimental groups. The results showed that The EGRA method significantly improved reading fluency in fourth-grade elementary school students (p &lt; 0.05). Thus, the EGRA method can help educators teach elementary school students to read more fluently.

Online resources as a source of information for exercise and physical activity in solid organ transplant recipients.

Exercise training post-transplant has been shown to improve physical function and quality of life in solid organ transplant (SOT) recipients. Online resources in the form of websites and videos are commonly used to provide education and instruction on exercise and physical activity in SOT; however, the content and quality of these online resources has not been evaluated. The first 200 websites and videos identified on Google and YouTube using the English search term "exercise and physical activity in solid organ transplantation" were analyzed. Website and video content was evaluated based on 25 key components of exercise and physical activity in SOT as described in established exercise program recommendations. Website and video quality was determined using DISCERN, Global Quality Scale (GQS), and Patient Education Materials and Assessment Tool (PEMAT; threshold for which material is deemed understandable or actionable is >70%). Parametric and non-parametric tests were used to assess website and video characteristics, content, and quality metrics. Forty-nine unique SOT websites (n = 15) and videos (n = 34) were identified, with the two most common categories being foundation/advocacy organizations and scientific resources. The average reading grade level of websites was 13 ± 3. Website and video content scores varied significantly (websites 11.3 ± 6.4; videos 8.4 ± 5.3). DISCERN total score and GQS score were low (median range for DISCERN 2.5-3.0; median for GQS 2.0 for both websites and videos, out of 5). PEMAT understandability and actionability scores were also low across websites and videos (mean range 57%-67% and 47%-65%, respectively). Foundation/advocacy websites had higher content and quality scores compared to scientific organizations and news/media articles. To our knowledge, this is the first comprehensive assessment of online content and quality of website and video resources on physical activity and exercise in adult SOT recipients. There were a limited number of online English patient-directed resources related to physical activity in SOT, most of which only partly captured items outlined in consensus exercise program recommendations and were of low quality and understandability and actionability. This work provides important insight to the English-speaking transplant community on the current state of online exercise health information and provides future direction for resource development.

OSSEOINTEGRATION IN LIMB REPLACEMENT SURGERY: AN ANALYSIS OF THE QUALITY AND READABILITY OF AVAILABLE INFORMATION ON THE INTERNET

IntroductionThe use of osseointegrated limb replacement prostheses for amputees are rapidly growing in popularity due to the many benefits over conventional sockets. Many institutions worldwide are beginning to provide this surgical option to their patients and are advertising the service online. The aim of this study was to assess the quality and reading level of related information available online.Materials &amp; MethodsA search was performed using the 3 most popular English language search engines reviewing the top 40 websites from each, using the term osseointegration on. Duplicates, videos, medical journal articles and dentistry websites were excluded. All included websites were assessed for reading using the FKGL, G-Fog and FRE. Quality was assessed using the JAMA benchmark criteria, HONC erificate and a novel 16-point scoring system developed specifically for limb replacement osseointegration.ResultsThe search resulted in 9,985,000 websites, 120 were assessed. 23 websites were included after removal of duplicates, medical journals and dentistry websites. None with reading grades of 6 or lower. Mean FKGL grade was 11.3/18. Mean FRE was 38.83/100. Mean G-FOG was 12.57/20. No websites had HONC. Mean JAMA score was 1.76/4. Mean HSSOIS was 5.70/16. Most provided information on indications(16/23) and explanation of osseointegration 17/23) but few discussed costs(3/23) or restrictions(9/23).ConclusionsThere is already a large amount of information available online regarding osseointegrated prostheses. However, the most accessible information may exceed most patients’ reading levels, limiting is usefulness. Further, the information available is generally of low quality, The quality of information and the readability of online content should be improved to ensure more effective patient education.

Search engine optimization and its association with readability and accessibility of diabetic retinopathy websites

PurposeThis study investigated whether websites regarding diabetic retinopathy are readable for patients, and adequately designed to be found by search engines.MethodsThe term “diabetic retinopathy” was queried in the Google search engine. Patient-oriented websites from the first 10 pages were categorized by search result page number and website organization type. Metrics of search engine optimization (SEO) and readability were then calculated.ResultsAmong the 71 sites meeting inclusion criteria, informational and organizational sites were best optimized for search engines, and informational sites were the most visited. Better optimization as measured by authority score was correlated with lower Flesch Kincaid Grade Level (r = 0.267, P = 0.024). There was a significant increase in Flesch Kincaid Grade Level with successive search result pages (r = 0.275, P = 0.020). Only 2 sites met the 6th grade reading level AMA recommendation by Flesch Kincaid Grade Level; the average reading level was 10.5. There was no significant difference in readability between website categories.ConclusionWhile the readability of diabetic retinopathy patient information was poor, better readability was correlated to better SEO metrics. While we cannot assess causality, we recommend websites improve their readability, which may increase uptake of their resources.

No-fee school consistently outperforms Progress in International Reading and Literacy benchmarks: Presenting early grade reading data from a case in Makhanda, Eastern Cape

No-fee school consistently outperforms Progress in International Reading and Literacy benchmarks: Presenting early grade reading data from a case in Makhanda, Eastern Cape

Readability of Neurosurgical Patient Education Resources by the American Association of Neurological Surgeons

The importance of patient education has become increasingly apparent in recent years. A prominent patient education tool in neurosurgery is the Neurosurgical Conditions and Treatments page provided by the American Association of Neurological Surgeons (AANS). This study aimed to investigate the readability of this resource page as many new articles have been incorporated in the past decade. Flesch-Kincaid Grade Level and Flesch-Kincaid Reading Ease scores were calculated for each article. One-way analysis of variance and Scheffé, Tukey-Kramer, Bonferroni, Fisher least significant difference, and Dunnett test post hoc analyses were conducted to determine differences between each category with respect to their mean grade level and reading ease scores. Overall mean (SD) Flesch-Kincaid Reading Ease score was 40.2 (12.24), and overall mean (SD) Flesch-Kincaid Grade Level score was 7.48 (1.26). Significant differences were found between mean reading ease scores between each categorization by the AANS (P= 0.014). No significant differences were found between mean grade level score for each categorization (analysis of variance, P= 0.154). As compared to a single previous investigation conducted 10 years ago, the readability of articles has changed modestly, and the reading grade level remains well above the recommendations by the American Medical Association and National Institutes of Health. The 6 new articles introduced in the past decade have demonstrated similar readability, presenting a persistent challenge in the realm of patient education in neurosurgery.

Early cognitive predictors of spelling and reading in German-speaking children

Theoretical backgroundWhile reading and spelling skills often are interconnected in models of literacy development, recent research suggests that the two skills can dissociate and that reading and spelling are associated with at least partly different cognitive predictors. However, previous research on dissociations between reading and spelling skills focused on children who have already mastered the first phases of literacy development. These findings suggest that dissociations are due to distinct deficits in orthographic processing (i.e., unprecise orthographic representations vs. inefficient serial processing). It is therefore unclear whether dissociations already become apparent during the initial stages, or rather emerge later in development. This study aims to enhance the understanding of the predictors of early spelling and reading skills, investigating potential variations, by considering various cognitive factors beyond well-established ones.MethodsData were collected at two time points: cognitive predictors and early reading and spelling skills were assessed at the end of kindergarten (T1) before formal literacy instruction started, and reading and spelling skills were again assessed in Grade 1 (T2). The data analysis included 353 first-grade participants. Linear regression analyses assessed predictive patterns, while logistic regression analyses explained children's likelihood of belonging to different proficiency groups (at-risk or typical skills).ResultsResults revealed phonological processing, letter knowledge, and intelligence, as significant predictors for Spelling in grade 1 (T2), even after adding the autoregressor (Spelling in kindergarten at T1) and the respective other literacy skill (Reading T2). For Reading in grade 1 (T2), phonological processing, and rapid automatized naming (RAN) surfaced as significant predictors after adding the autoregressor (Reading T1). However, only RAN surfaced as a significant predictor for Reading T2 after adding the respective other literacy skill (Spelling T2). In line with these findings, logistic regression analyses revealed that phonological processing predicted group allocation for Spelling T2 and RAN predicted group allocation for Reading T2.ConclusionsOverall, the study underscores the importance of phonological processing and letter knowledge as early predictors of spelling and reading skills in Grade 1. Moreover, intelligence is identified as a predictor for early spelling, while rapid automatized naming (RAN) emerges as a predictor for early reading.

Empowering inclusivity: improving readability of living kidney donation information with ChatGPT.

Addressing disparities in living kidney donation requires making information accessible across literacy levels, especially important given that the average American adult reads at an 8th-grade level. This study evaluated the effectiveness of ChatGPT, an advanced AI language model, in simplifying living kidney donation information to an 8th-grade reading level or below. We used ChatGPT versions 3.5 and 4.0 to modify 27 questions and answers from Donate Life America, a key resource on living kidney donation. We measured the readability of both original and modified texts using the Flesch-Kincaid formula. A paired t-test was conducted to assess changes in readability levels, and a statistical comparison between the two ChatGPT versions was performed. Originally, the FAQs had an average reading level of 9.6 ± 1.9. Post-modification, ChatGPT 3.5 achieved an average readability level of 7.72 ± 1.85, while ChatGPT 4.0 reached 4.30 ± 1.71, both with a p-value <0.001 indicating significant reduction. ChatGPT 3.5 made 59.26% of answers readable below 8th-grade level, whereas ChatGPT 4.0 did so for 96.30% of the texts. The grade level range for modified answers was 3.4-11.3 for ChatGPT 3.5 and 1-8.1 for ChatGPT 4.0. Both ChatGPT 3.5 and 4.0 effectively lowered the readability grade levels of complex medical information, with ChatGPT 4.0 being more effective. This suggests ChatGPT's potential role in promoting diversity and equity in living kidney donation, indicating scope for further refinement in making medical information more accessible.

Gender-affirming vaginoplasty online resources exceed patient health literacy: A readability analysis

PurposeEnsuring that educational materials geared toward transgender and gender-diverse patients are comprehensible can mitigate barriers to accessing gender-affirming care and understanding postoperative care. This study evaluates the readability of online patient resources related to gender-affirming vaginoplasty. MethodsOnline searches for vaginoplasty were conducted in January 2023 using two search engines. The readability scores of the top ten websites and their associated hyperlinked webpages were derived using ten validated readability tests. ResultsA total of 40 pages were assessed from the vaginoplasty searches. The average reading grade level for all the webpages with relevant educational materials was 13.3 (i.e., college level), exceeding the American Medical Association's recommended 6th grade reading level. ConclusionComplex patient resources may impede patients’ understanding of gender-affirming vaginoplasty. Online patient education resources should be created that are more accessible to patients with diverse reading comprehension capabilities.

Evaluating the Readability of Patient Education Materials for Anterior Vertebral Body Tethering, Distraction-Based Methods, and Posterior Spinal Fusion for the Treatment of Pediatric Spinal Deformity.

The Internet is an important source of information for patients, but its effectiveness relies on the readability of its content. Patient education materials (PEMs) should be written at or below a sixth-grade reading level as outlined by agencies such as the American Medical Association. This study assessed PEMs' readability for the novel anterior vertebral body tethering (AVBT), distraction-based methods, and posterior spinal fusion (PSF) in treating pediatric spinal deformity. An online search identified PEMs using the terms "anterior vertebral body tethering," "growing rods scoliosis," and "posterior spinal fusion pediatric scoliosis." We selected the first 20 general medical websites (GMWs) and 10 academic health institution websites (AHIWs) discussing each treatment (90 websites total). Readability tests for each webpage were conducted using Readability Studio software. Reading grade levels (RGLs), which correspond to the US grade at which one is expected to comprehend the text, were calculated for sources and independent t tests compared with RGLs between treatment types. The mean RGL was 12.1 ± 2.0. No articles were below a sixth-grade reading level, with only 2.2% at the sixth-grade reading level. AVBT articles had a higher RGL than distraction-based methods (12.7 ± 1.6 vs 11.9 ± 1.9, P = 0.082) and PSF (12.7 ± 1.6 vs 11.6 ± 2.3, P = 0.032). Materials for distraction-based methods and PSF were comparable (11.9 ± 1.9 vs 11.6 ± 2.3, P = 0.566). Among GMWs, AVBT materials had a higher RGL than distraction-based methods (12.9 ± 1.4 vs 12.1 ± 1.8, P = 0.133) and PSF (12.9 ± 1.4 vs 11.4 ± 2.4, P = 0.016). Patients' health literacy is important for shared decision-making. Assessing the readability of scoliosis treatment PEMs guides physicians when sharing resources and discussing treatment with patients. Both GMWs and AHIWs exceed recommended RGLs, which may limit patient and parent understanding. Within GMWs, AVBT materials are written at a higher RGL than other treatments, which may hinder informed decision-making and patient outcomes. Efforts should be made to create online resources at the appropriate RGL. At the very least, patients and parents may be directed toward AHIWs; RGLs are more consistent.

Responses of Five Different Artificial Intelligence Chatbots to the Top Searched Queries About Erectile Dysfunction: A Comparative Analysis

The aim of the study is to evaluate and compare the quality and readability of responses generated by five different artificial intelligence (AI) chatbots—ChatGPT, Bard, Bing, Ernie, and Copilot—to the top searched queries of erectile dysfunction (ED). Google Trends was used to identify ED-related relevant phrases. Each AI chatbot received a specific sequence of 25 frequently searched terms as input. Responses were evaluated using DISCERN, Ensuring Quality Information for Patients (EQIP), and Flesch-Kincaid Grade Level (FKGL) and Reading Ease (FKRE) metrics. The top three most frequently searched phrases were “erectile dysfunction cause”, “how to erectile dysfunction,” and “erectile dysfunction treatment.” Zimbabwe, Zambia, and Ghana exhibited the highest level of interest in ED. None of the AI chatbots achieved the necessary degree of readability. However, Bard exhibited significantly higher FKRE and FKGL ratings (p = 0.001), and Copilot achieved better EQIP and DISCERN ratings than the other chatbots (p = 0.001). Bard exhibited the simplest linguistic framework and posed the least challenge in terms of readability and comprehension, and Copilot’s text quality on ED was superior to the other chatbots. As new chatbots are introduced, their understandability and text quality increase, providing better guidance to patients.

Adolescent eHealth Literacy

In adults, the literature indicates that health literacy is a robust predictor of health. However, there remains a paucity of literature regarding adolescent health literacy, based on Manganello’s framework for studying adolescent health literacy, one potentially related construct of adolescent health literacy is eHealth literacy. Therefore, the purpose of this study is to describe (1) the relationship between adolescent health literacy and adolescent eHealth literacy and (2) the influencing factors of adolescent eHealth literacy. This is a cross-sectional, correlational study evaluating results from the online administration of the Newest Vital Sign (NVS) health literacy instrument and eHEALS eHealth literacy instrument to 125 adolescents. Bivariate correlations and linear regression were performed to examine relationships among variables. There was a significant, negative correlation between eHEALS and NVS scores (r = −.304, p = .001). There were significant, positive relationships between eHEALS scores, age (r = .346, p < .01) and grade level (r = .354, p < .01). There was a significant, positive correlation between NVS scores and reading grade (r = .326, p = < .01). Reading grade and eHEALS scores were significant predictors of NVS scores (R = .093, F = 12.57, p < .001). Therefore, the influencing factors of adolescent health literacy and eHealth literacy are related, but inconsistent. Furthermore, the negative correlation between NVS and eHEALS scores demonstrates that adolescent health literacy and eHealth literacy are associated, but independent constructs. Further research is needed to better understand facilitators and barriers to both constructs as well as their relationship to each other.

Adequacy of prostate cancer prevention and screening recommendations provided by an artificial intelligence-powered large language model.

We aimed to assess the appropriateness of ChatGPT in providing answers related to prostate cancer (PCa) screening, comparing GPT-3.5 and GPT-4. A committee of five reviewers designed 30 questions related to PCa screening, categorized into three difficulty levels. The questions were formulated identically for both GPTs three times, varying the prompts. Each reviewer assigned a score for accuracy, clarity, and conciseness. The readability was assessed by the Flesch Kincaid Grade (FKG) and Flesch Reading Ease (FRE). The mean scores were extracted and compared using the Wilcoxon test. We compared the readability across the three different prompts by ANOVA. In GPT-3.5 the mean score (SD) for accuracy, clarity, and conciseness was 1.5 (0.59), 1.7 (0.45), 1.7 (0.49), respectively for easy questions; 1.3 (0.67), 1.6 (0.69), 1.3 (0.65) for medium; 1.3 (0.62), 1.6 (0.56), 1.4 (0.56) for hard. In GPT-4 was 2.0 (0), 2.0 (0), 2.0 (0.14), respectively for easy questions; 1.7 (0.66), 1.8 (0.61), 1.7 (0.64) for medium; 2.0 (0.24), 1.8 (0.37), 1.9 (0.27) for hard. GPT-4 performed better for all three qualities and difficulty levels than GPT-3.5. The FKG mean for GPT-3.5 and GPT-4 answers were 12.8 (1.75) and 10.8 (1.72), respectively; the FRE for GPT-3.5 and GPT-4 was 37.3 (9.65) and 47.6 (9.88), respectively. The 2nd prompt has achieved better results in terms of clarity (all p < 0.05). GPT-4 displayed superior accuracy, clarity, conciseness, and readability than GPT-3.5. Though prompts influenced the quality response in both GPTs, their impact was significant only for clarity.

Evaluating the Adaptability of Large Language Models for Knowledge-aware Question and Answering

Abstract Large language models (LLMs) have transformed open-domain abstractive summarization, delivering coherent and precise summaries. However, their adaptability to user knowledge levels is largely unexplored. This study investigates LLMs’ efficacy in tailoring summaries to user familiarity. We assess various LLM architectures across different familiarity settings using metrics like linguistic complexity and reading grade levels. Findings expose current capabilities and constraints in knowledge-aware summarization, paving the way for personalized systems. We analyze LLM performance across three familiarity levels: none, basic awareness, and complete familiarity. Utilizing established readability metrics, we gauge summary complexity. Results indicate LLMs can adjust summaries to some extent based on user familiarity. Yet, challenges persist in accurately assessing user knowledge and crafting informative, comprehensible summaries. We highlight areas for enhancement, including improved user knowledge modeling and domain-specific integration. This research informs the advancement of adaptive summarization systems, offering insights for future development.