High Reading Level Research Articles

Quality, Reliability, and Readability of Peripheral Nerve Intervention Websites for Patients.

This study aims to evaluate the readability, quality, and reliability of online resources about peripheral nerve surgeries to determine if they meet recommended literacy standards. We analyzed a total of 137 peripheral nerve surgery website by performing a Google search using the search terms "nerve transfer," "nerve repair," "nerve graft," "nerve decompression," "neurolysis," "targeted muscle reinnervation," "regenerative peripheral nerve interface," and "vascularized denervated muscle target." The reading level of the website text was assessed using Simple Measures of Gobbledygook, Flesch-Kincaid, and Gunning Fog. Quality was evaluated using the DISCERN Instrument. Reliability was determined using the Journal of American Medical Association Benchmark Criteria. All the websites exceeded the sixth-grade reading level, with median readability scores corresponding to a high school reading level or above. Conceptually harder peripheral nerve surgeries such as targeted muscle reinnervation and regenerative peripheral nerve interface websites were generally written at a significantly higher reading level than conceptually easier surgeries such as nerve repair and nerve graft. The median quality of the websites was rated as poor, and the median reliability of the websites was rated as low. The findings indicate that the current peripheral nerve surgery websites texts do not adhere to recommended reading levels and are constructed with poor quality and low reliability. This potentially hinders patients understanding and utilization of peripheral nerve surgeries, suggesting a need for standardized guidelines to enhance the accessibility of medical information online.

Enhancing Spanish Patient Education Materials: Comparing the Readability of Artificial Intelligence-Generated Spanish Patient Education Materials to the Society of Pediatric Dermatology Spanish Patient Brochures.

Patient education materials (PEMs) are crucial for improving patient adherence and outcomes; however, they may not beaccessibledue to high reading levels. Our study used seven readability measures to compare the readability of SpanishPEMsfromthe Society of Pediatric Dermatology (SPD) with those generated by Open-AI's ChatGPT 4.0 and Google Gemini. Our results showed that when prompted to produce material at a 6th grade level, both AI ChatBots generated significantly improved readability scores when compared to the SPD handouts. These findings suggest that AI-generated PEMs couldbetter meet readability standards and potentially improve patient outcomes, although further studies are needed to confirm this.

The impact of interword spacing on inference processing during text reading: Evidence from eye movements

ABSTRACT Removing interword spacing has been shown to disrupt lower-level oculomotor processes and word identification during text reading. However, the impact of these disruptions on higher-level processes remains unclear. To examine the influence of spacing on inferential processing, we monitored eye movements while participants read spaced and unspaced passages that were strongly or weakly constrained toward an inference. Removing spaces disrupted reading fluency, as evidenced by longer reading times, longer fixation durations, reduced skipping, and shorter saccades. We also observed the effects of inferential constraint for spaced passages as characterized by longer reading times, more regressions, and longer regression-path durations for weakly than strongly constrained passages. However, these constraint effects were absent for unspaced passages, suggesting that removing spaces disrupts inferential processing. Our results are consistent with models of reading and discourse processing that assume that higher-level reading processes depend on the quality of lexical representations developed at earlier, word-level reading stages.

Development and Validation of an Artificial Intelligence-Assisted Patient Education Material for Ostomy Patients: A Methodological Study.

To develop and test the validity of an artificial intelligence-assisted patient education material for ostomy patients. A methodological study. The study was carried out in two main stages and five steps: (1) determining the information needs of ostomy patients, (2) creating educational content, (3) converting the educational content into patient education material, (4) validation of patient education material based on expert review and (5) measuring the readability of the patient education material. We used ChatGPT 4.0 to determine the information needs and create patient education material content, and Publuu Online Flipbook Maker was used to convert the educational content into patient education material. Understandability and applicability scores were assessed using the Patient Education Materials Assessment Tool submitted to 10 expert reviews. The tool inter-rater reliability was determined via the intraclass correlation coefficient. Readability was analysed using the Flesch-Kincaid Grade Level, Gunning Fog Index and Simple Measure of Gobbledygook formula. The mean Patient Education Materials Assessment Tool understandability score of the patient education material was 81.91%, and the mean Patient Education Materials Assessment Tool actionability score was 85.33%. The scores for the readability indicators were calculated to be Flesch-Kincaid Grade Level: 8.53, Gunning Fog: 10.9 and Simple Measure of Gobbledygook: 7.99. The AI-assisted patient education material for ostomy patients provided accurate information with understandable and actionable responses to patients, but is at a high reading level for patients. Artificial intelligence-assisted patient education materials can significantly increase patient information rates in the health system regarding ease of practice. Artificial intelligence is currently not an option for creating patient education material, and their impact on the patient is not fully known. The study followed the STROBE checklist guidelines. No patient or public contributions.

Is Information About Musculoskeletal Malignancies From Large Language Models or Web Resources at a Suitable Reading Level for Patients?

Patients and caregivers may experience immense distress when receiving the diagnosis of a primary musculoskeletal malignancy and subsequently turn to internet resources for more information. It is not clear whether these resources, including Google and ChatGPT, offer patients information that is readable, a measure of how easy text is to understand. Since many patients turn to Google and artificial intelligence resources for healthcare information, we thought it was important to ascertain whether the information they find is readable and easy to understand. The objective of this study was to compare readability of Google search results and ChatGPT answers to frequently asked questions and assess whether these sources meet NIH recommendations for readability. (1) What is the readability of ChatGPT-3.5 as a source of patient information for the three most common primary bone malignancies compared with top online resources from Google search? (2) Do ChatGPT-3.5 responses and online resources meet NIH readability guidelines for patient education materials? This was a cross-sectional analysis of the 12 most common online questions about osteosarcoma, chondrosarcoma, and Ewing sarcoma. To be consistent with other studies of similar design that utilized national society frequently asked questions lists, questions were selected from the American Cancer Society and categorized based on content, including diagnosis, treatment, and recovery and prognosis. Google was queried using all 36 questions, and top responses were recorded. Author types, such as hospital systems, national health organizations, or independent researchers, were recorded. ChatGPT-3.5 was provided each question in independent queries without further prompting. Responses were assessed with validated reading indices to determine readability by grade level. An independent t-test was performed with significance set at p < 0.05. Google (n = 36) and ChatGPT-3.5 (n = 36) answers were recorded, 12 for each of the three cancer types. Reading grade levels based on mean readability scores were 11.0 ± 2.9 and 16.1 ± 3.6, respectively. This corresponds to the eleventh grade reading level for Google and a fourth-year undergraduate student level for ChatGPT-3.5. Google answers were more readable across all individual indices, without differences in word count. No difference in readability was present across author type, question category, or cancer type. Of 72 total responses across both search modalities, none met NIH readability criteria at the sixth-grade level. Google material was presented at a high school reading level, whereas ChatGPT-3.5 was at an undergraduate reading level. The readability of both resources was inadequate based on NIH recommendations. Improving readability is crucial for better patient understanding during cancer treatment. Physicians should assess patients' needs, offer them tailored materials, and guide them to reliable resources to prevent reliance on online information that is hard to understand. Level III, prognostic study.

Improving readability and comprehension levels of otolaryngology patient education materials using ChatGPT

ObjectiveA publicly available large language learning model platform may help determine current readability levels of otolaryngology patient education materials, as well as translate these materials to the recommended 6th-grade and 8th-grade reading levels. Study designCross-sectional analysis. SettingOnline using large language learning model, ChatGPT. MethodsThe Patient Education pages of the American Laryngological Association (ALA) and American Academy of Otolaryngology-Head and Neck Surgery (AAO-HNS) websites were accessed. Materials were input into ChatGPT (OpenAI, San Francisco, CA; version 3.5) and Microsoft Word (Microsoft, Redmond, WA; version 16.74). Programs calculated Flesch Reading Ease (FRE) scores, with higher scores indicating easier readability, and Flesch-Kincaid (FK) grade levels, estimating U.S. grade level required to understand text. ChatGPT was prompted to “translate to a 5th-grade reading level” and provide new scores. Scores were compared for statistical differences, as well as differences between ChatGPT and Word gradings. ResultsPatient education materials were reviewed and 37 ALA and 72 AAO-HNS topics were translated. Overall FRE scores and FK grades demonstrated significant improvements following translation of materials, as scored by ChatGPT (p < 0.001). Word also scored significant improvements in FRE and FK following translation by ChatGPT for AAO-HNS materials overall (p < 0.001) but not for individual topics or for subspecialty-specific categories. Compared with Word, ChatGPT significantly exaggerated the change in FRE grades and FK scores (p < 0.001). ConclusionOtolaryngology patient education materials were found to be written at higher reading levels than recommended. Artificial intelligence may prove to be a useful resource to simplify content to make it more accessible to patients.

TrachGPT: Appraisal of tracheostomy care recommendations from an artificial intelligent Chatbot.

Safe home tracheostomy care requires engagement and troubleshooting by patients, who may turn to online, AI-generated information sources. This study assessed the quality of ChatGPT responses to such queries. In this cross-sectional study, ChatGPT was prompted with 10 hypothetical tracheostomy care questions in three domains (complication management, self-care advice, and lifestyle adjustment). Responses were graded by four otolaryngologists for appropriateness, accuracy, and overall score. The readability of responses was evaluated using the Flesch Reading Ease (FRE) and Flesch-Kincaid Reading Grade Level (FKRGL). Descriptive statistics and ANOVA testing were performed with statistical significance set to p < .05. On a scale of 1-5, with 5 representing the greatest appropriateness or overall score and a 4-point scale with 4 representing the highest accuracy, the responses exhibited moderately high appropriateness (mean = 4.10, SD = 0.90), high accuracy (mean = 3.55, SD = 0.50), and moderately high overall scores (mean = 4.02, SD = 0.86). Scoring between response categories (self-care recommendations, complication recommendations, lifestyle adjustments, and special device considerations) revealed no significant scoring differences. Suboptimal responses lacked nuance and contained incorrect information and recommendations. Readability indicated college and advanced levels for FRE (Mean = 39.5, SD = 7.17) and FKRGL (Mean = 13.1, SD = 1.47), higher than the sixth-grade level recommended for patient-targeted resources by the NIH. While ChatGPT-generated tracheostomy care responses may exhibit acceptable appropriateness, incomplete or misleading information may have dire clinical consequences. Further, inappropriately high reading levels may limit patient comprehension and accessibility. At this point in its technological infancy, AI-generated information should not be solely relied upon as a direct patient care resource.

Hakikat Media Pembelajaran Membaca di Kelas Tinggi

The use of learning media in high-level reading classes plays an important role in improving students' reading abilities. Various types of learning media, including visual, audio-visual, and interactive computer-based materials, are used for this purpose. The process of utilizing learning media includes analysis of student characteristics, design, development, implementation and evaluation of learning outcomes. This learning media has various functions in teaching reading, such as facilitating access to information, motivating students, helping understanding and retention of information, fostering critical thinking skills, enabling differentiated learning, encouraging independent learning, and developing digital literacy skills. In addition, many studies have investigated the effectiveness of various learning media, such as comics, modules, computer-based interactive learning, and picture cards, in increasing elementary school students' interest in reading and academic achievement.

The impact of orthographic knowledge on reading development in German third graders

AbstractOrthographic knowledge (i.e., the knowledge of conventions of a written language) has been identified as a predictor of both basic and higher-level reading processes, however, mostly examined in a cross-sectional design. It remains unclear, whether and how orthographic knowledge contributes uniquely in explaining differences in the acquisition of reading skills. The aim of this study was therefore to examine the effects of orthographic knowledge on basic (word-reading) and higher level (sentence- and text-comprehension) reading development in German third graders over the course of a schoolyear. 325 German speaking children (mean age = 8.35 years, standard deviation = 0.57; 49% girls) participated in the study, 100 of whom also provided data on vocabulary and naming speed. Data were analyzed via multiple regression in hierarchical models with random intercepts, since the children were nested in classes. The longitudinal analyses showed that orthographic knowledge contributes to the development of reading on the word- and text-level, over and above vocabulary knowledge and naming speed. These findings support and extend previous results, showing that orthographic knowledge not only supports reading processes, but also plays an important role in reading development over the course of a schoolyear. The fact that no significant interaction between orthographic knowledge and reading comprehension could be found implies that orthographic knowledge is relevant for reading development in German third graders, independent of the level of reading proficiency. The present findings suggest that orthographic knowledge should be considered in reading instruction at school as well as during reading related interventions.

Why higher-level reading is important

Societies are facing fundamental transformations as digital technologies are changing the ways we live, interact, work, study and read. The social and cultural impact of the digitization process on reading skills and practices remains under-researched. While digital technologies offer much potential for new forms of reading, recent empirical research shows that the digital environment is having a negative impact on reading, in particular on long-form reading and reading comprehension. It also remains unclear whether the transition to digital media actually lives up to its promise of improving learning outcomes. Recent studies of various kinds indicate a decline of crucial higher-level reading competencies and practices, such as critical and conscious reading, slow reading, non-strategic reading and long-form reading. Current educational policy, meanwhile, relies heavily on monocultural standardized testing of basic reading ca­pabilities and on growing use of digital technologies. Reading education, assessment, research and policy-making should focus more on higher-level reading practices in both adults and children in order to understand the development of reading skills and practices in an age increasingly dependent on a ubiquitous digital infrastructure.

Low Literacy Levels Among U.S. Adults and Difficult Ballot Propositions

High-level literacy skills are required for full participation in the democratic process through voting. Consequently, adults with low-level literacy skills are at a disadvantage. This work investigated the disparity between the readability of U.S. ballot propositions for year 2022 state elections and grade level reading estimates (≤eighth grade) for adults. Educational attainment was also examined. Propositions (n = 140) from 38 states were included. Mean readability was 18 (range 7.0-64.0). Only four measures (3%) fell within range of national estimates for adult reading ability. Thirty-nine percent of adults completed high school or less, yet 74% of ballots were written well above a high school reading level. There is a discrepancy between the literacy skills of the average voter and the readability of most propositions. The findings of this study have important implications for individuals with learning disabilities. Policy changes and educational support efforts should be initiated.

Readability Analysis of Patient Education Material on Rotator Cuff Injuries From the Top 25 Ranking Orthopaedic Institutions

Introduction: Rotator cuff injuries (RCIs) are incredibly common in the US adult population. Forty-three percent of adults have basic or below-basic literacy levels; nonetheless, patient educational materials (PEMs) are frequently composed at levels exceeding these reading capabilities. This study investigates the readability of PEMs on RCIs published by leading US orthopaedic institutions. Methods: The top 25 orthopaedic institutions on the 2022 U.S. News &amp; World Report Best Hospitals Specialty Ranking were selected. Readability scores of PEMs related to RCI were calculated using the www.readabilityformulas.com website. Results: Among the 25 analyzed PEM texts, all exceeded the sixth-grade reading level. Only four of 168 scores (2.4%) were below the eighth-grade level. Discussion: This study indicates that PEMs on rotator cuff injuries from top orthopedic institutions are too complex for many Americans, with readability levels ranging from 8.5 to 16th grade, well above the CDC-recommended eighth-grade level. The research highlights a widespread issue with high reading levels across healthcare information and underscores the need for healthcare providers to adopt patient-centered communication strategies to improve comprehension and accessibility. Conclusion: PEMs on rotator cuff injuries from leading orthopedic institutions often have a reading level beyond that of many Americans, exceeding guidelines from the NIH and CDC that recommend PEMs be written at an eighth-grade reading level. To increase accessibility, enhance healthcare literacy, and improve patient outcomes, institutions should simplify these materials to meet recommended readability standards.

Does Attitude towards Grammar Affect High-Level Reading Comprehension Skills?

Grammar is a learning area that functionally contributes to effective language skills. The aim of grammar instruction is the acquisition of the four basic language skills based on Turkish language rules. Reading skill, a lingual comprehension skill, aims accurate structuring of the ideas of the author of a text by the reader. The present study aimed to determine the possible effects of the attitudes towards grammar on high-level reading comprehension skills (HLRCS) based on certain variables (gender, final Turkish language course grade, and parental education level). In the current quantitative study, data were collected with the Grammar Attitude Scale and the High-Level Reading Comprehension Test (HLRCT). The study group included 427 8th grade middle school students. In the study, the regression analysis conducted to determine the causality between the Grammar Attitude Scale and HLRCT scores, revealed significant findings. Grammar attitude explained 4% of the variation in the HLRCT scores. Grammar attitude improved the achievements in high-level reading comprehension.

Readability of online patient education materials for shoulder instability surgery in English and Spanish

Online patient education materials (OPEMs) exist to inform patient medical decisions, yet the average adult in the United States reads at an eighth-grade level and 50% of Medicaid patients read at or below a fifth-grade level. To appropriately meet US health literacy needs, the American Medical Association and National Institutes of Health recommend that patient education materials not exceed a sixth-grade level. The purpose of this study was to assess and compare the readability of English and Spanish online patient education materials pertaining to shoulder instability surgery. Google searches of the terms "shoulder instability surgery" and "cirugía de inestabilidad de hombro'' were conducted to include 25 eligible online patient education materials OPEMs per language. English OPEM readability was calculated using Flesch-Kincaid Grade Level, Flesch Reading Ease, Flesch Reading Ease Grade Level, Gunning-Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook. Spanish OPEM readability was assessed using Fernandez-Huerta Index (FHI) (the Spanish equivalent of Flesch Reading Ease), FHI Grade Level, Gutiérrez de Polini's Fórmula de comprensibilidad, and INFLESZ. Readability index analysis revealed that the mean Flesch Reading Ease of English online patient education materials was significantly lower than the mean FHI of Spanish online patient education materials. English materials were also found to be written at a significantly higher grade level than Spanish materials. Shoulder instability surgery online patient education materials in both English and Spanish are written at higher reading levels than recommended by the American Medical Association and National Institutes of Health, though Spanish online patient education materials were more readable on average.

Dr. Google vs. Dr. ChatGPT: Exploring the Use of Artificial Intelligence in Ophthalmology by Comparing the Accuracy, Safety, and Readability of Responses to Frequently Asked Patient Questions Regarding Cataracts and Cataract Surgery

ABSTRACT Purpose Patients are using online search modalities to learn about their eye health. While Google remains the most popular search engine, the use of large language models (LLMs) like ChatGPT has increased. Cataract surgery is the most common surgical procedure in the US, and there is limited data on the quality of online information that populates after searches related to cataract surgery on search engines such as Google and LLM platforms such as ChatGPT. We identified the most common patient frequently asked questions (FAQs) about cataracts and cataract surgery and evaluated the accuracy, safety, and readability of the answers to these questions provided by both Google and ChatGPT. We demonstrated the utility of ChatGPT in writing notes and creating patient education materials. Methods The top 20 FAQs related to cataracts and cataract surgery were recorded from Google. Responses to the questions provided by Google and ChatGPT were evaluated by a panel of ophthalmologists for accuracy and safety. Evaluators were also asked to distinguish between Google and LLM chatbot answers. Five validated readability indices were used to assess the readability of responses. ChatGPT was instructed to generate operative notes, post-operative instructions, and customizable patient education materials according to specific readability criteria. Results Responses to 20 patient FAQs generated by ChatGPT were significantly longer and written at a higher reading level than responses provided by Google (p < .001), with an average grade level of 14.8 (college level). Expert reviewers were correctly able to distinguish between a human-reviewed and chatbot generated response an average of 31% of the time. Google answers contained incorrect or inappropriate material 27% of the time, compared with 6% of LLM generated answers (p < .001). When expert reviewers were asked to compare the responses directly, chatbot responses were favored (66%). Conclusions When comparing the responses to patients’ cataract FAQs provided by ChatGPT and Google, practicing ophthalmologists overwhelming preferred ChatGPT responses. LLM chatbot responses were less likely to contain inaccurate information. ChatGPT represents a viable information source for eye health for patients with higher health literacy. ChatGPT may also be used by ophthalmologists to create customizable patient education materials for patients with varying health literacy.

Heuristic evaluation and usability testing of a functional dementia stages app for Hispanic family caregivers

AbstractBackgroundHispanic family caregivers of persons with Alzheimer’s disease and other dementias are not getting the anticipatory guidance they need about how functional abilities will change as dementia progresses. Existing informational resources are overwhelming to navigate and written at a high reading level. Furthermore, professional assessment of functional abilities is not universally available. Innovative, tailored approaches are needed. Our aim was to develop and test a mobile application, the Interactive Functional Assessment Staging Navigator (I‐FASTN), to support Hispanic family caregivers’ assessment of care recipients’ functional stage of dementia in either English or Spanish.MethodWe conducted heuristic evaluation with informatics experts (N = 5, English only) and usability testing with caregivers (n = 10 English, n = 10 Spanish). Participants attempted a series of tasks with the app using a think‐aloud protocol and then rate the usability of the app using mobile app heuristics (experts) and two usability instruments (caregivers). Sessions were recorded and transcribed. For heuristic evaluation, we tabulated the number of usability problems by category: cosmetic, minor, major, and catastrophic. For usability testing, recordings, notes, and transcripts were content analyzed to extract usability problems, suggestions for improvement, desired app features, and other feedback.ResultRefinements to the app were made after each of the five evaluations to address the problems encountered. Caregivers’ mean usability scores were high (4.6 and 4.7 out of 5.0), reflecting good usability overall. Chief usability problems were a confusing tutorial and difficulty finding the app’s side menu. These will be addressed by creating a video tutorial and moving all menu items to the home screen. The app was very well received by caregivers, who indicated that the concise, illustrated content met their informational needs. However, analog alternatives are still needed by caregivers unaccustomed to using apps as they had difficulty moving on from even the simplest screens (see Figure 1a).ConclusionCaregivers are not receiving informational support in a way that meets their needs. The focus should be on providing concise information that addresses the practical aspects of caregiving, tailored to the care recipient’s functional stage.

SAT119 Artificial Intelligence Mediated Assistants: Will They Educate The Public About Health Disparities In Diabetes?

Abstract Disclosure: W. Zhao: None. G. Wu: None. E. Cheng: None. J. Lee: None. A. Wong: None. Background: In 2022, 37 million US adults had diabetes and 96 million had pre-diabetes. Diabetes and prediabetes affect 49.3% population in America. In the diabetic population, 12.1% are non-Hispanic blacks, 11.8% of Hispanics, and 9.5% of Asian Americans. 54% of Americans read at or below the sixth-grade level. ChatGPTTM, launched 11/30/22, has the newest Artificial Intelligence (AI) software versus Google AssistantTM, launched 05/2016. Purpose: Can ChatGPT use its natural language software to educate the lay public about diabetes and health disparities? Methods: Evaluate the readability of responses after five questions were posed to ChatGPTTM and Google AssistantTMQuestions: 1. Do African Americans have a higher risk of complications with Type 2 Diabetes? 2. Do Hispanic Americans have a higher risk of complications with Type 2 diabetes? 3. Do Asian Americans have a higher risk of complications with Type 2 diabetes? 4. Do women get Type 2 diabetes more than men? 5. What is pre-diabetes vs diabetes? Word count and Dale-Chall score assessed the readability level of both ChatGPT and Google Assistant responses. The Dale-Chall readability formula is calculated based on the percentage of difficult words and average sentence length in words. The U.S. Department of Education used the Dale-Chall formula to adjust the text to grade-level readability. Result: Word count: CHAT:Q1=104; Q2=135; Q3=136; Q4=133; Q5=232 vs GOOGLE: Q1=38, Q2=49, Q3=16; Q4=50; Q5=42. CHAT Word Count Avg=148, SD=48.81; GOOGLE Word Count Avg=39, SD=13.7; P-value=0.003908 Dale-Chall: CHAT: Q1=8.4; Q2=8.6; Q3=9.8, Q4=8.6, Q5=8.7 vs. GOOGLE: Q1=6.7, Q2=9, Q3=9.4, Q4=6.3, Q5=9.1. CHAT Dale-Chall Score Avg=8.82, SD=0.56; GOOGLE Dale-Chall Score Avg=8.1, SD=1.47; P-value=0.1306. ChatGPT has nearly the same grade level of comprehension as Google Assistant, including statistics about health disparities. However, ChatGPT was trained using information and transforming it into paragraphs, and google assistant extracts the data from websites and the variability is higher with a higher standard deviation than ChatGPT. Conclusion: ChatGPT with its artificial intelligence software is able to provide answers to our patients. While the answers may require a high school reading level and beyond, this is an excellent beginning for health education. Presentation: Saturday, June 17, 2023

Initial Steps in Creating a Patient-Centric Addendum to Clinical Trial Informed Consent Forms

IntroductionThe purpose of the informed consent form (ICF) is to outline the risks and benefits of an interventional clinical trial to potential participants. The aim of this study was to explore the feasibility of a short addendum to the ICF, summarizing key points most relevant to potential participants. MethodsA sample of 20 ICFs was reviewed against the requirements of the U.S. federal regulation documents and assessed for readability. Alongside the ICF review, we conducted focus groups and one-on-one interviews with people with lung cancer (n = 9) to learn what information was most important when considering participation in a clinical trial using a hypothetical phase 3 ICF. ResultsThe 20 ICFs reviewed were from phases 1 to 3, expanded-access, and single-patient trials covering predominantly NSCLC; 60% were global. The mean length of the ICFs was 21 (range: 15–34) pages. The average reading level was tenth grade whereas the average U.S. reading level was eighth grade. Readability varied by section, the “purpose of the study” section had the highest reading level. In the qualitative research component, participants were “overwhelmed” by the hypothetical ICF. Participants were also asked to list information for the addendum; their suggestions broadly map to federal regulations. An addendum with reference to sections in the ICF for additional details was well received. ConclusionsThe variations in ICF architecture and readability make it difficult for patients to make an informed decision to participate in a clinical trial. Implications extend beyond lung cancer, highlighting key areas for ICF improvements and providing a roadmap for developing a patient-centric addendum.

Evaluation of Online Artificial Intelligence-Generated Information on Common Hand Procedures

The purpose of this study was to analyze the quality and readability of the information generated by an online artificial intelligence (AI) platform regarding 4 common hand surgeries and to compare AI-generated responses to those provided in the informational articles published by the American Society for Surgery of the Hand (ASSH) HandCare website. An open AI model (ChatGPT) was used to answer questions commonly asked by patients on 4 common hand surgeries (carpal tunnel release, cubital tunnel release, trigger finger release, and distal radius fracture fixation). These answers were evaluated for medical accuracy, quality and readability and compared to answers derived from the ASSH HandCare materials. For the AI model, the Journal of the American Medical Association benchmark criteria score was 0/4, and the DISCERN score was 58 (considered good). The areas in which the AI model lost points were primarily related to the lack of attribution, reliability and currency of the source material. For AI responses, the mean Flesch Kinkaid Reading Ease score was 15, and the Flesch Kinkaid Grade Level was 34, which is considered to be college level. For comparison, ASSH HandCare materials scored 3/4 on the Journal of the American Medical Association Benchmark, 71 on DISCERN (excellent), 9 on Flesch Kinkaid Grade Level, and 60 on Flesch Kinkaid Reading Ease score (eighth/ninth grade level). An AI language model (ChatGPT) provided generally high-quality answers to frequently asked questions relating to the common hand procedures queried, but it is unclear when or where these answers came from without citations to source material. Furthermore, a high reading level was required to comprehend the information presented. The AI software repeatedly referenced the need to discuss these questions with a surgeon, the importance of shared decision-making and individualized care, and compliance with surgeon treatment recommendations. As novel AI applications become increasingly mainstream, hand surgeons must understand the limitations and ramifications these technologies have for patient care.

Evaluation High-Quality of Information from ChatGPT (Artificial Intelligence—Large Language Model) Artificial Intelligence on Shoulder Stabilization Surgery

To analyze the quality and readability of information regarding shoulder stabilization surgery available using an online AI software (ChatGPT), using standardized scoring systems, as well as to report on the given answers by the AI. An open AI model (ChatGPT) was used to answer 23 commonly asked questions from patients on shoulder stabilization surgery. These answers were evaluated for medical accuracy, quality, and readability using The JAMA Benchmark criteria, DISCERN score, Flesch-Kincaid Reading Ease Score (FRES) & Grade Level (FKGL). The JAMA Benchmark criteria score was 0, which is the lowest score, indicating no reliable resources cited. The DISCERN score was 60, which is considered a good score. The areas that open AI model did not achieve full marks were also related to the lack of available source material used to compile the answers, and finally some shortcomings with information not fully supported by the literature. The FRES was 26.2, and the FKGL was considered to be that of a college graduate. There was generally high quality in the answers given on questions relating to shoulder stabilization surgery, but there was a high reading level required to comprehend the information presented. However, it is unclear where the answers came from with no source material cited. It is important to note that the ChatGPT software repeatedly references the need to discuss these questions with an orthopaedic surgeon and the importance of shared discussion making, as well as compliance with surgeon treatment recommendations. As shoulder instability is an injury that predominantly affects younger individuals who may use the Internet for information, this study shows what information patients may be getting online.