Patient-Reported Outcome Measures in Surgical Oncology: A Cross-Sectional Study of NIH and AMA Readability Compliance.

Patient-reported outcome measures (PROMs) are essential tools for assessing surgical effectiveness and capturing patient perspectives on functionality, symptoms, and quality of life. However, readability is critical for ensuring patients can understand and accurately complete them. The National Institutes of Health (NIH) and American Medical Association (AMA) recommend that patient materials be written at or below a sixth-grade reading level. In pediatric orthopaedic surgery, the Pediatric Outcomes Data Collection Instrument (PODCI), Pediatric Quality of Life Inventory (PedsQL), and Patient-Reported Outcomes Measurement Information System (PROMIS) instruments are among the most commonly utilized PROMs. This study evaluates whether PROMs used in pediatric orthopaedic registries meet NIH and AMA readability standards. A readability analysis was performed on 59 PROMs reported in pediatric orthopaedic registries. PROMs were categorized as validated or not validated for patients under 18 years of age. Primary readability metrics were the Flesch Reading Ease Score (FRES) and the Simple Measure of Gobbledygook (SMOG) Index. Scores were obtained using an online calculator and compared against NIH and AMA guidelines. A FRES ≥80 or SMOG <7 was considered reflective of a sixth-grade or lower reading level. The average FRES and SMOG Index for all PROMs were 69 ± 14 and 8 ± 1, respectively, corresponding to an eighth to ninth-grade reading level. Only 4 PROMs met both FRES and SMOG readability thresholds: Hospital Anxiety and Depression Scale, Knee Society-Function, Children's Revised Impact of Event Scale, and PedsQL. PROMs such as the International Knee Documentation Committee, Lower Extremity Functional Scale, Single Assessment Numeric Evaluation, University of California Los Angeles Activity Scale, and Kujala Score were among the least readable. Although PedsQL met both readability thresholds, PODCI and PROMIS instruments were written above the recommended sixth-grade reading level. Most PROMs used in pediatric orthopaedic surgery are written above the recommended sixth-grade reading level, potentially impacting patient comprehension and the reliability of collected data. Level IV-observational study.

Introduction Osteoarthritis (OA) is an age-related degenerative joint disease. There is a 25% risk of symptomatic hip OA in patients who live up to 85 years of age. It can impair a person's daily activities and increase their reliance on healthcare services. It is primarily managed with education, weight loss and exercise, supplemented with pharmacological interventions. Poor health literacy is associated with negative treatment outcomes and patient dissatisfaction. A literature search found there are no previously published studies examining the readability of online information about hip OA. Objectives To assess the readability of healthcare websites regarding hip OA. Methods The terms "hip pain", "hip osteoarthritis", "hip arthritis", and "hip OA" were searched on Google and Bing. Of 240 websites initially considered, 74 unique websites underwent evaluation using the WebFX online readability software (WebFX®, Harrisburg, USA). Readability was determined using the Flesch Reading Ease Score (FRES), Flesch-Kincaid Reading Grade Level (FKGL), Gunning Fog Index (GFI), Simple Measure of Gobbledygook (SMOG), Coleman-Liau Index (CLI), and Automated Readability Index (ARI). In line with recommended guidelines and previous studies, FRES >65 or a grade level score of sixth grade and under was considered acceptable. Results The average FRES was 56.74±8.18 (range 29.5-79.4). Only nine (12.16%) websites had a FRES score >65. The average FKGL score was 7.62±1.69 (range 4.2-12.9). Only seven (9.46%) websites were written at or below a sixth-grade level according to the FKGL score. The average GFI score was 9.20±2.09 (range 5.6-16.5). Only one (1.35%) website was written at or below a sixth-grade level according to the GFI score.The average SMOG score was 7.29±1.41 (range 5.4-12.0). Only eight (10.81%) websites were written at or below a sixth-grade level according to the SMOG score. The average CLI score was 13.86±1.75 (range 9.6-19.7). All 36 websites were written above a sixth-grade level according to the CLI score. The average ARI score was 6.91±2.06 (range 3.1-14.0). Twenty-eight(37.84%) websites were written at or below a sixth-grade level according to the ARI score. One-sample t-tests showed that FRES (p<0.001, CI -10.2 to -6.37), FKGL (p<0.001, CI 1.23 to 2.01), GFI (p<0.001, CI 2.72 to 3.69), SMOG (p<0.001, CI 0.97 to 1.62), CLI (p<0.001, CI 7.46 to 8.27), and ARI (p<0.001, CI 0.43 to 1.39) scores were significantly different from the accepted standard. One-way analysis of variance (ANOVA) testing of FRES scores (p=0.009) and CLI scores (p=0.009) showed a significant difference between categories. Post hoc testing showed a significant difference between academic and non-profit categories for FRES scores (p=0.010, CI -15.17 to -1.47) and CLI scores (p=0.008, CI 0.35 to 3.29). Conclusions Most websites regarding hip OA are written above recommended reading levels, hence exceeding the comprehension levels of the average patient. Readability of these resources must be improved to improve patient access to online healthcare information which can lead to improved patient understanding of their own condition and treatment outcomes.

Observational study. To evaluate the readability of commonly utilized patient-reported outcome measures (PROMs) in spine surgery. Although studies have evaluated patient education materials, discharge instructions, and informed consent forms in spine surgery, there is a dearth of literature on the readability of PROMs despite widespread health illiteracy. Without knowledge of PROM readability, it is unclear whether these measures are able to be understood by the average spine patient. We analyzed all commonly utilized nonvisual PROMs within the spinal literature and uploaded PROMs into an online readability calculator. The Flesch Reading Ease Score (FRES) and Simple Measure of Gobbledygook (SMOG) Index were collected. A FRES>79 or SMOG<7 was considered readable by the general population per American Medical Association and Centers for Disease Control guidelines. A stricter threshold recommended in health care (SMOG <6 or FRES>89) was then used to further review readability. Seventy-seven PROMs were included. Based on FRES, the mean readability of all PROMs was 69.2 ± 17.2 (range, 10-96.4), indicating an average eighth to ninth-grade reading level. The mean readability score categorized by the SMOG Index was 8.12 ± 2.65 (range, 3.1-25.6), representing an eighth-grade reading level. Compared with the reading level of the general population, 49 (63.6%) PROMs are written above the United States literacy level, according to FRES. Using stricter definitions of readability, 8 PROMs were considered readable, including PROM Information System pain behavior (FRES: 96.4 and SMOG: 5.2), PROM Information System sleep disturbance (SMOG: 5.6), Neck Pain and Disability Scale (SMOG: 4.3), and Zung Depression Scale (SMOG: 3.1). Most PROMs utilized in spine surgery require an average reading competency far above the average patient's comprehension. This may have a meaningful impact on understanding PROM instruments and may affect the accuracy of complete surveys and the rates of incompletion.

Readability of the Most Commonly Used Patient-Reported Outcome Measures in Hand Surgery.

Sports Medicine Patient-Reported Outcomes Fail to Meet National Institutes of Health- and American Medical Association-Recommended Reading Levels.

Readability Assessment of Online Tracheostomy Care Resources

Accessibility to Puberphonia Online and Its Readability by Patients

Patient-reported outcome measures are important standardized survey tools in medicine. In anesthesiology, patient-reported outcome measures are used perioperatively to collect information about patients’ health status, quality of life, and healthcare experiences.1 As health care evolves, patient-reported outcome measures have the ability to enhance patient engagement and shared decision making, which could improve the safety and efficacy of anesthesia.2Although there are many benefits of patient-reported outcome measures, low health literacy is a significant barrier to their effective use. The National Institutes of Health (Bethesda, Maryland) and other healthcare organizations recommend healthcare-related materials to be published at or below a sixth-grade reading level.3 Recent studies in other fields have calculated many specialty-specific patient-reported outcome measures to be above the recommended complexity.3 We examined patient-reported outcome measures commonly used in the perioperative setting in relation to health equity.Eighteen patient-reported outcome measures were selected from those in a systematic review.1 Three additional patient-reported outcome measures highly cited in anesthesia literature were also included.4 The 21 patient-reported outcome measures evaluated are listed in table 1. Readability was assessed by a linguistics software (readable.com; Added Bytes, United Kingdom) using the Gunning Fog Index, Simple Measure of Gobbledygook (SMOG) Index, FORCAST Grade Level, and Flesch Reading Ease Score, indices used in previous readability studies and applicable to healthcare materials.3 Each numerical score computed per index correlates to a reading grade level. Consequently, an average readability score of six denotes a reading grade level of sixth grade.All 21 anesthesia patient-reported outcome measures analyzed had average readability scores above the sixth-grade level. The overall average readability was nine, corresponding to a ninth-grade reading level. The Hospital Anxiety and Depression Scale had the easiest readability at six, whereas the Bauer questionnaire had the most difficult at twelve. The average readability levels by patient-reported outcome measure are reported in table 1. Table 1 also shows the readability scores of each individual patient-reported outcome measure and the mean readability score and SD by index.Patient-reported outcome measures have become an important tool to measure not only vital outcomes such as mortality and postoperative complications but also postoperative symptoms that affect length of hospital stay, chronic health conditions, and cost of care.5 Measuring these outcomes allows for better patient engagement and shared decision making, thus enhancing overall medical care.2Recent readability studies in specialties such as otolaryngology have also reported the majority of their patient-reported outcome measures to be higher than recommended by healthcare organizations.3 Furthermore, several readability studies found online patient education materials in anesthesia to be even less readable.6,7 De Oliveira et al.6,7 determined a 13th-grade reading level is required to understand most patient education materials in anesthesia. Other studies reported the reading level of patient-reported experience measures in regional anesthesia and neuraxial labor analgesia to be well above the sixth grade.The complexity of healthcare reading materials poses a significant challenge for patients with lower health literacy, particularly in their ability to accurately complete patient-reported outcome measure questionnaires. Low health literacy disproportionately affects underserved populations, including minority groups, immigrants, and adults with lower education levels, further contributing to health disparities.8 Low health literacy is also associated with poor health outcomes, including delayed diagnoses, inadequate self-management skills, and higher rates of chronic conditions.9 Similarly, perioperative outcomes may be negatively affected.10This study has several limitations. First, the Gunning Fog and SMOG indices were developed to analyze running narratives, rather than questionnaires, which may have affected readability scores.3 Second, the algorithms are not designed to analyze complex medical terms, which may have also affected the readability scores.3 Last, although a comprehensive literature search was conducted by the authors, it is possible that some validated anesthesia patient-reported outcome measures were not included in this study. However, inclusion of more would likely not have altered data enough to affect conclusions, because the average readability of patient-reported outcome measures in this study is at a ninth-grade level.Developers of patient-reported outcome measure should consider performing readability analysis of text when designing questionnaires. And although these tools undergo a rigorous vetting before implementation, the authors would like to raise awareness of imperfections that exist. Improving the readability of these reporting tools could aid in improving data accuracy and health outcomes.The authors declare no competing interests.

MP02-16 CAN GENITOURINARY CANCER SURVIVORS UNDERSTAND QUALITY OF LIFE (QoL) QUESTIONNAIRES? ASSESSING THE READABILITY OF COMMONLY USED UROLOGIC ONCOLOGY QoL INSTRUMENTS

Readability Analysis of Commonly Cited Patient-Reported Outcome Measures in the Orthopaedic Joint Literature.

ObjectivesTo assess readability and understandability of online materials for vocal cord leukoplakia.Study DesignReview of online materials.SettingAcademic medical center.MethodsA Google search of “vocal cord leukoplakia” was performed, and the first 50 websites were considered for analysis. Readability was measured by the Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), and Simple Measure of Gobbledygook (SMOG). Understandability and actionability were assessed by 2 independent reviewers with the PEMAT-P (Patient Education Materials Assessment Tool for Printable Materials). Unpaired t tests compared scores between sites aimed at physicians and those at patients, and a Cohen’s kappa was calculated to measure interrater reliability.ResultsTwenty-two websites (17 patient oriented, 5 physician oriented) met inclusion criteria. For the entire cohort, FRES, FKGL, and SMOG scores (mean ± SD) were 36.90 ± 20.65, 12.96 ± 3.28, and 15.65 ± 3.57, respectively, indicating that materials were difficult to read at a >12th-grade level. PEMAT-P understandability and actionability scores were 73.65% ± 7.05% and 13.63% ± 22.47%. Statistically, patient-oriented sites were more easily read than physician-oriented sites (P < .02 for each of the FRES, FKGL, and SMOG comparisons); there were no differences in understandability or actionability scores between these categories of sites.ConclusionOnline materials for vocal cord leukoplakia are written at a level more advanced than what is recommended for patient education materials. Awareness of the current ways that these online materials are failing our patients may lead to improved education materials in the future.

ABSTRACTPurpose: To evaluate the readability of online uveitis patient education materials.Methods: A Google search in November 2016 was completed using search term “uveitis” and “uveitis inflammation.” The top 50 websites with patient-centered information were selected and analyzed for readability using the Flesch–Kincaid Grade Level (FKGL), Flesch Reading Ease Score (FRES), Gunning FOG Index (GFI), and Simple Measure of Gobbledygook (SMOG). Statistical analysis was performed with two-tailed t-tests.Results: The mean word count of the top 50 websites was 1162.7 words, and averaged 16.2 words per sentence. For these websites, the mean FRES was 38.0 (range 4–66, SD = 12.0), mean FKGL was 12.3 (range 6.8–19, SD = 2.4), mean SMOG score was 14.4 (range 9.8–19, SD = 1.8), and the mean Gunning FOG index was 14.0 (range 8.6–19, SD = 2.0).Conclusions: The majority of online patient directed uveitis materials are at a higher reading level than that of the average American adult.

BackgroundParticipant information sheets and consent forms (PICFs) used in interventional studies are often criticised for being hard to read and understand. We assessed the readability and its correlates of a broad range of Australian PICFs.MethodsWe analysed the participant information sheet portion of 248 PICFs. Readability scores were measured using three formulae: the Flesch Reading Ease, the Flesch-Kincaid Grade Level, and the Simple Measure of Gobbledygook (SMOG). We investigated how various features (including sponsor type and PICF type) correlated with PICF length and readability and examined compliance with other measures known to improve readability.ResultsFor a sample of 248 PICFs, the mean (standard deviation) Flesch Reading Ease score was 49.3 (5.7) and for the Flesch-Kincaid Grade Level 11.4 (1.1). The mean (SD) SMOG score was 13.2 (0.9). The median document length was 3848 words (8 pages). Commercial PICFs were more than twice as long as non-commercial, but statistically more readable (p = 0.03) when analysed using the SMOG formula. Subgroup analyses indicated that PICFs for self-consenters were statistically more readable than those for proxy consenters. The use of tables, but not the use of illustrations was associated with better readability scores.ConclusionsThe PICFs in our sample are long and complex, and only 3 of the 248 achieved the recommended readability score of grade 8 or below. The broader use of best practice principles for writing health information for consumers and the development of more context-sensitive templates could improve their utility.

Objective: This study aimed to evaluate the readability, quality, and reliability of online patient education materials (PEM) related to sarcopenia.Materials and methods: In August 2024, 278 websites obtained by searching the term ‘sarcopenia’ on Google were evaluated. After applying the exclusion criteria, 66 websites were included in the study. The ranks of the websites were evaluated with Blexb (New York, USA), readability evaluations were done with calculators, and quality and reliability were evaluated per the Journal of the American Medical Association (JAMA), Global Quality Scale (GQS), and modified Decision Index for Systematic Consumer Evaluation of Reviewed Narratives (DISCERN) criteria.Results: The readability values of the contents were compared with the 6th-grade level (GRL), and all results were significantly higher than the 6th GRL (p<0.01). The median values in the readability assessment of the 66 websites included in the study were as follows: Flesch Reading Ease Score (FRES) 40.81 (6.10-72.84), Gunning Fog (GFOG) 14.78 (9.10-22.20), Flesch-Kincaid Grade Level (FKGL) 12.18 (7.16-18.58), Coleman-Liau (CL) score 12.89 (7.14-18.45), Simple Measure of Gobbledygook (SMOG) 12.34 (7.77-17.65), Automated Readability Index (ARI) 12.49 (7.70-19.64), Linsear Write (LW) 13.30 (6.89-25.05), and GRL 15.25 (8.00-23.00). The types of websites were compared in terms of readability levels, and no significant difference was found. When the website typologies were compared, no significant difference was observed in GQS, JAMA, and DISCERN. A low-level negative correlation was found between Blexb rank and JAMA. The correlations between Blexb rank and readability scales were as follows: weak negative correlation with FRES and ARI scores and weak positive correlation with GFOG, FKGL, CL, and SMOG scores (p<0.05).Conclusion: The best category in terms of readability, reliability, and quality was determined to be health portals, but none of the websites had readability at or below the recommended 6th GRL.

So Difficult to Understand : Readability Index Analysis of Online Patient Information on Lymphoma from NCI- Designated Cancer Center.