Flesch-Kincaid Grade Level Scores Research Articles

Online Health Information for Penile Prosthesis Implants Lacks Quality and Is Unreadable to the Average US Patient.

Online health information (OHI) has become widely accessible and affects patient decisions regarding their healthcare. The purpose of this study was to assess the readability, quality, and accuracy of information available to patients online about penile prosthesis implants (PPIs). We performed a Google search using the keywords "penile implant" and "penile prosthesis." The first 30 search results for both terms were analyzed, and advertisements, news articles, duplicates, and videos were excluded. Websites were categorized as institutional, commercial, and personal/patient support. Readability of each website was determined using the Flesch-Kincaid grade level (FKGL) readability formula within the readable tool. Quality was measured by Health On the Net (HON) certification status and the DISCERN scoring method. For website accuracy, a score of 1-4 (1=0-25%, 2=25-50%, 3=50-75%, and 4=75-100%) was assigned. Forty-four websites met the criteria (23 institutional, 12 commercial, and 9 personal/patient support). The mean total FKGL score was 9.55. No statistical difference was detected between mean FKGL for each website category (p=0.69). Only eight websites (18%) scored ≤8th-grade reading level (average US adult level), while 36 (82%) were >8th-gradelevel. Mean total DISCERN sum score was 39.74/75, with no statistical difference in mean DISCERN score between website types (p=0.08). Over half (55%) of the websites were defined as "very poor" or "poor" quality by DISCERN scoring. Mean total overall quality rating was 2.67/5. HON certification was verified for only nine websites (20%). Twenty-five percent of websites were classified as 0-25% accurate, 23% were 25-50% accurate, 30% were 50-75% accurate, and 23% were 75-100% accurate. Most information on the Internet about PPIs is reasonably accurate; however, the majority of websites are deficient in quality and unreadable to the average patient, irrespective of website type.

Evaluation of accuracy, reliability, quality, and readability of online patient information materials on coccyx injury.

The aim of this research is to evaluate the websites containing "coccydynia, coccyx trauma or fracture" in terms of readability, reliability, accuracy, and quality. Searches for "coccydynia, coccyx trauma, coccyx fracture" were carried out in the 3 most used search engines in the USA: Google, Yahoo, and Bing in February 2022. A total of 141 websites were rated by 2 different neurosurgeons for the "Global Quality Score" and "Alexa Popularity Rank." 97.2% of the sites examined include the definition of the disease, 66% include the importance of the disease, 92.9% include the symptoms of the disease, 88.7% include the treatment of the disease, 78% include the signs of the disease, 77.3% include the mechanism of the disease It has been determined that the websites examined within the scope of the research have high global quality score (GQS) and APR and are enriched with images to a large extent.

Growing Taller without Hormones? Dr. Consult Google-An Evaluation of Online Information Related to Limb Lengthening.

The aim of this study was to investigate the reliability, content and readability of the information available on the Internet related to limb lengthening surgeries, which have recently been progressively in fashion. The three most commonly used browsers on the Internet were determined and a search term for "Limb Lengthening Surgery" was typed for each browser. The websites were categorized by their type, and the content and the quality of them was evaluated using the DISCERN score, the Journal of American Medical Association (JAMA) benchmark and the Global Quality Score (GQS). The Flesch Kincaid Grade Level (FKGL) and the Flesch Reading Ease Score (FKRS) were used to evaluate the readability. Each website also assessed the presence (or absence) of the Health on Net (HON) code. The academic category was found to be significantly higher than the medical and commercial categories. Mean FKGL and FCRS scores, DISCERN score values, JAMA, GQS and LLCS score values of Websites with HON code were significantly higher than those without. The quality of online information related to limb lengthening was of low quality. Although some websites, especially academic resources, were of higher quality, the readability of their content is just about 2.5 degrees higher than the sixth-grade reading level.

Evaluation of Reading Level of Result Letters Sent to Patients from an Academic Primary Care Practice.

In primary care, low health literacy, particularly reading ability, is associated with worse health outcomes. Most physicians do not receive feedback on the reading levels of written communication that they may provide to patients, including result letters. Our study compares the readability of result letters, written by resident versus attending physicians, to patients with positive or negative screens for reading ability, as determined by the single-item literacy screener (SILS). Result letters to 50 patients at high risk and 50 patients at low risk of low reading ability were randomly selected starting from January 1st, 2020 at Albany Medical Center. Flesch-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG), and Flesch Reading Ease (FRE) were used to compare the readability of resident versus attending result letters. For all SILS levels, attending physicians wrote result letters at a lower grade level than resident physicians based on the FKGL, GFI, and SMOG indices. The FKGL, GFI, and SMOG readability scores of result letters written to patients with SILS 3-5 were also lower when written by attending physicians compared to resident physicians. Result letters written by attending physicians may be easier to read than result letters written by resident physicians, especially for patients with low reading ability. Future electronic health record (EHR) software should give physicians and providers feedback on the reading level of their written communication.

Feasibility and acceptability of ChatGPT generated radiology report summaries for cancer patients.

Patients now have direct access to their radiology reports, which can include complex terminology and be difficult to understand. We assessed ChatGPT's ability to generate summarized MRI reports for patients with prostate cancer and evaluated physician satisfaction with the artificial intelligence (AI)-summarized report. We used ChatGPT to summarize five full MRI reports for patients with prostate cancer performed at a single institution from 2021 to 2022. Three summarized reports were generated for each full MRI report. Full MRI and summarized reports were assessed for readability using Flesch-Kincaid Grade Level (FK) score. Radiation oncologists were asked to evaluate the AI-summarized reports via an anonymous questionnaire. Qualitative responses were given on a 1-5 Likert-type scale. Fifty newly diagnosed prostate cancer patient MRIs performed at a single institution were additionally assessed for physician online portal response rates. Fifteen summarized reports were generated from five full MRI reports using ChatGPT. The median FK score for the full MRI reports and summarized reports was 9.6 vs. 5.0, (p < 0.05), respectively. Twelve radiation oncologists responded to our questionnaire. The mean [SD] ratings for summarized reports were factual correctness (4.0 [0.6], understanding 4.0 [0.7]), completeness (4.1 [0.5]), potential for harm (3.5 [0.9]), overall quality (3.4 [0.9]), and likelihood to send to patient (3.1 [1.1]). Current physician online portal response rates were 14/50 (28%) at our institution. We demonstrate a novel application of ChatGPT to summarize MRI reports at a reading level appropriate for patients. Physicians were likely to be satisfied with the summarized reports with respect to factual correctness, ease of understanding, and completeness. Physicians were less likely to be satisfied with respect to potential for harm, overall quality, and likelihood to send to patients. Further research is needed to optimize ChatGPT's ability to summarize radiology reports and understand what factors influence physician trust in AI-summarized reports.

Online information for incisional hernia repair: What are patients reading?

PurposePatients increasingly access online materials for health-related information. Using validated assessment tools, we aim to assess the quality and readability of online information for patients considering incisional hernia (IH) repair. MethodsThe top three online search engines (Google, Bing, Yahoo) were searched in July 2022 for “Incisional hernia repair” and “Surgical hernia repair”. Included websites were classified as academic, hospital-affiliated, commercial, and unspecified. The quality of information was assessed using the Journal of the American Medical Association (JAMA) benchmark criteria (0–4), DISCERN instrument (16–80), and the presence of Health On the Net code (HONcode) certification. Readability was assessed using the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) tests. Results25 unique websites were included. The average JAMA and DISCERN scores of all websites were 0.68 ± 1.02 and 36.50 ± 10.91, respectively. Commercial sites showed a significantly higher DISCERN mean score than academic sites (p = 0.034), while no significant difference was demonstrated between other website categories. 3 (12%) websites reported HONcode certification and had significantly higher JAMA (p = 0.016) and DISCERN (p = 0.045) mean scores than sites without certification. Average FRE and FKGL scores were 39.84 ± 13.11 and 10.62 ± 1.76, respectively, corresponding to college- and high school-level comprehensibility. ConclusionsOur findings suggest online patient resources on IH repair are of poor overall quality and may not be comprehensible to the public. Patients accessing internet resources for additional information on IH repair should be made aware of these inadequacies and directed to sites bearing HONcode certification.

ODP198 Free Diabetes Apps: An Intervention to Reduce Health Disparities in our Diabetic Patients

Abstract Background 85% of Americans own a smartphone and there were 230 billion downloads in 2021. 1,2 American diabetic patients have been reported to have limited health literacy. 3 55% of Americans do not graduate from high school. 4 African American, Hispanic patients and Asian patients comprise 20%, 22% and 19% of diabetic patients, respectively, but a greater proportion of hospitalization rates. 5 Many of these patients and their families use mobile apps for health questions. The accessibility of eHealth apps is changing how patients access health information. Purpose To evaluate free diabetes health apps as an intervention to reduce health disparities in our diabetic patients. Methods Using search term, "diabetes" in GooglePlayStore(Android/AD) and AppleAppStore(iOS) to find the best, free diabetes apps. Exclusion criteria: less than 100,000 downloads (DL) on Google and&amp;lt;1600 reviews (RE) on Apple (Apple does not provide download data). The app introduction was "pasted" into readabilityformulas.com and webfx.com for analysis via Flesch-Kincaid Reading Ease (FRE), Flesch-Kincaid Grade Level Score (FGL), Simple Measure of Gobbledygook Index (SI), and the Dale-Chall Adjusted Grade Level (DCGL). Results AD(DL): 1-mySugr(1M),2-OnetouchReveal(1M),3-DiabetesBloodSugar(1M),4-BloodGlucoseTracker(500K),5-Diabetes(500K),6-Health2Sync(500K),7-DiabetesM(500K), 8-GlucoseBuddy(500K), 9-BeatDiabetes(100K), 10- Contour Diabetes App (100K);iOS(RE): 1-OnetouchReveal(87K), 2-GlucoseBuddyDiabetesTracker(21K), 3-Onedrop(20K), 4-GlucoseBloodSugarTracker(17K), 5-Livongo(13K), 6-mySugr(9.6K), 7-Sugarmate(9.4K),8-Klinio(3K), 9- Glycemic Index Load Net Carbs (2.2K), 10- Diabetes Logs (1.6K). Push notification=19/20 apps(AD=9, iOS=10). Connect users to GoodRx=0/20 apps. Connect users to professional care team/diabetes education=8/20 apps(AD=3, iOS=5). Connect users to free/discounted insulin/health products=0/20 apps. Spanish=11/20, Chinese=5/20 apps (AD=3, iOS=8, some apps have both). Only 2/20 apps were at 7-8th grade and the rest were high school and beyond. Free test strips were offered only on 1/20 apps(Livongo). Conclusion Free diabetes apps are helpful and may serve as an easily accessible educational tool for our patients. The physician can play a role in navigating the eHealth space with the patient.

ODP168 Can Virtual Assistants Help our Diverse Population of Diabetic Patients?

Abstract Background: Diabetes affects 34 million in the US and an additional 88 million are prediabetic. 1 According to the CDC, Diabetes affects Hispanic Americans (22%) and Asian Americans (19%) more than the Caucasian (12%) population. 2 Many of these patients do not speak English. In addition, 55% of US adults did not graduate from high school. 3 Health literacy is a barrier to health care. There are 4.2 billion Virtual Assistant AI devices, e. g. Amazon Echo Dot and Google Home Mini, which are used worldwide. Virtual assistant devices are used in nursing homes and assisted living facilities. 4 Purpose: To evaluate Virtual Assistant Devices, Alexa and Google, as a educational tool for our multicultural Diabetic Population Methods: Evaluation of Amazon Echo Dot (Alexa) and Google Home Mini (Google) with voice activation in English, Spanish and Mandarin was performed. Alexa did not have Mandarin Chinese speaking capability. Responses from both devices were recorded and then transcribed to text using Google Docs "speech to text" tool and assistance from native Mandarin Chinese and Spanish speakers. Transcribed English language responses were pasted into WebFx.com and readabilityformulas.com for readability analysis via Flesch-Kincaid Reading Ease (FRE), Flesch-Kincaid Grade Level Score (FGL), Simple Measure of Gobbledygook Index (SI), and the Dale-Chall Adjusted Grade Level (DCGL). Verbal Questions: 1) I have diabetes- what do I do? 2) I have Type 1 Diabetes Mellitus - What do I do? 3) I have Type 2 Diabetes - What do I do? 4) Does diabetes cause me to have problems seeing? 5) Does diabetes cause kidney problems? 6) Does diabetes cause problems with my feet? Results: Word Counts: English: Google (36.7 words +/- 10.4), Alexa (39.2 words +/- 25.2) Spanish: Google (31.2 words +/- 13.6), Alexa (45.2 words +/- 27.9) Chinese: Google (73.3 words +/- 31.6), Alexa (NA) No significant differences between the number of words for English vs. Chinese responses or English vs. Spanish responses. English Responses Readability: Avg FRE: Google (51.3 +/-12. 0), Alexa (71.1 +/- 9.9) (higher = easier reading) Avg FGL: Google (9.8 grade +/-1.7), Alexa (6.5 grade +/- 0.8) (higher = higher grade level) Avg SI: Google (9.5 +/-1.5), Alexa (6.5 +/-0.9) (higher = harder reading) Avg DCGL: Google (College), Alexa (Grades 9-10) Conclusion: Digital Voice Assistant Devices are helpful to patients who are visually impaired, have poor mobility or need language translation. Under physician guidance, these devices provide diabetes education for our patients and for the lay public.

Assessing the Accuracy, Quality, and Readability of Online Patient Resources on Achilles Tendonitis

Category: Other; Hindfoot Introduction/Purpose: The internet is full of websites focused on different medical topics. With such a large accumulation of information, patients should be able to rely upon the internet as a reliable source for information on common medical conditions such as Achilles tendonitis. However, the question remains as to the accuracy, quality and readability of such online patient resources. Consumers themselves commonly evaluate online medical sites by trustworthiness, expertise, and objectivity. The purpose of this study was to investigate the top search-engine-generated online resources for Achilles tendonitis for their content and accuracy. Methods: The term 'Achilles tendonitis' was searched using the three most population search engines. The first 50 sites from each search were recorded. Duplicate sites were eliminated. Sites were categorized based on authorship type. Each website was independently evaluated by two reviewers. Quality was evaluated utilizing the DISCERN score and the Journal of the American Medical Association (JAMA) benchmark criteria. Accuracy was evaluated using a novel Achilles tendinosis-specific content rubric developed with a foot and ankle fellowship-trained orthopaedic surgeon and the AAOS OrthoInfo webpage, similar to previous studies. Readability was evaluated using the Flesch Kincaid grade level (FKGL) and Flesch Reading Ease (FRE) formulas. Higher FRE scores indicate better readability. FKGL was also compared against an 8th grade level, which represents the average US reading level. Results: The search resulted in 77 unique websites with 60 sites included for analysis. The average DISCERN score in our investigation is 42.1 which places it in the 'fair' category. Commercial websites were found to have higher DISCERN scores as compared to websites created by physicians or medical groups as well as academic websites. Our study found an average JAMA quality benchmark score of 2.2 out of 4. Our study found that on average, online patient resources on Achilles tendinosis was written at a 9th grade reading level with a FKGL score of 9.8. Academic websites were found to be written at a significantly lower reading level of 8.8. The average FRE score was 53.28 which translates to the 'fairly difficult' category. Utilizing our own content scoring system to assess the accuracy of online resources, the websites had an average accuracy of 11.5 points out of 20 possible points. Conclusion: The average online patient resource on Achilles tendinosis is too complex for the average patient, inaccurate, and is of low quality. This study found that the average website had low quality as measured by the DISCERN and JAMA benchmark scores. The average accuracy was also low as measured by our novel content specific score. The websites were also difficult to read as measured by the FRE and FKGL scores. More emphasis on improving the readability, quality, and accuracy of online patient resources on Achilles tendinosis should be undertaken in order to improve patient healthcare.

How readable are orthognathic surgery consent forms?

The valid consent process for medical intervention requires the disclosure of information in a format that is easily understandable by the patient. The aim of this investigation was to assess the readability of orthognathic surgery informed consent forms (OSICFs). An online search methodology was conducted to identify OSICFs for analysis. The forms that satisfied inclusion/exclusion criteria were evaluated according to a standardised protocol. The readability of the content was assessed using three validated tools: the Simple Measure of Gobbledegook (SMOG) score, Flesch-Kincaid Grade-Level (FKGL) score and Flesch Reading Ease (FRE) score. Most of the 26 evaluated OSICFs were sourced from websites within the United States (69.2%) and from oral and maxillo-facial surgery practices (76.9%). Two of the assessed forms were template OSICFs available from oral and maxillo-facial professional societies to its members. The scores from the three tools found that the content of 84.6% to 92.3% of the forms were "difficult" to read. The mean (SD) SMOG score for all evaluated OSICFs was 12.31(2.22) [95% CI: 11.42 to 13.21]. The SMOG and FKGL scores were closely correlated (r=0.99, P<0.0001; 95% CI: 0.9864 to 0.9973). There was no association between SMOG scores and the number of words contained within each consent form (r=-0.047;95% CI: -0.44 to 0.36). The OSICFs surveyed in this investigation failed to meet recommended readability levels. A significant number of patients are not likely to understand the information contained within the forms. Orthodontists are advised that poor literacy skills of their patients may preclude them from validly consenting to orthognathic surgery treatment procedures.

Academics versus the Internet: Evaluating the readability of patient education materials for cerebrovascular conditions from major academic centers.

Background: Health literacy profoundly impacts patient outcomes as patients with decreased health literacy are less likely to understand their illness and adhere to treatment regimens. Patient education materials supplement in-person patient education, especially in cerebrovascular diseases that may require a multidisciplinary care team. This study aims to assess the readability of online patient education materials related to cerebrovascular diseases and to contrast the readability of those materials produced by academic institutions with those of non-academic sources.Methods: The readability of online patient education materials was analyzed using Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) assessments. Readability of academic-based online patient education materials was compared to nonacademic online patient education materials. Online patient education materials from 20 academic institutions and five sources from the web were included in the analysis.Results: Overall median FKGL for neurovascular-related patient online education documents was 11.9 (95% CI: 10.8–13.1), reflecting that they are written at a 12th grade level, while the median FRE was 40.6 (95% CI: 34.1–47.1), indicating a rating as “difficult” to read. When comparing academic-based online patient education materials to other internet sources, there was no significant difference in FRE and FKGL scores (P = 0.63 and P = 0.26 for FKGL and FRE, respectively).Conclusion: This study demonstrates that online patient education materials pertaining to cerebrovascular diseases from major academic centers and other nonacademic internet sites are difficult to understand and written at levels significantly higher than that recommended by national agencies. Both academic and nonacademic sources reflect this finding equally. Further study and implementation are warranted to investigate how improvements can be made.

Quality of web-based information on attention deficit hyperactivity disorder.

The limited literature available on quality of online health information about attention deficit hyperactivity disorder (ADHD), reported mixed findings and was more than five years old. This study aimed to assess the content quality, readability, aesthetics, and interactivity of current web-based information on ADHD. We screened first 50 links obtained on searching for ADHD using two popular search engines (Google and Bing), out of which 44 websites were selected for quality assessment. They were analysed using pre-decided study Performa, which included previously validated DISCERN instrument for content quality. Flesch-Kincaid grade level score and Flesch-Kincaid readability index were calculated for assessing readability. About 61.4% of websites had DISCERN score ≥40, suggestive of good content quality. However, only 38.6% and 13.6% of websites were written at recommended reading level of eighth and sixth grade respectively. The DISCERN score had significant positive correlation with aesthetics and interactivity scores. Websites with the Health On Net (HON) label had better content quality. No significant difference in the readability, aesthetics, and interactivity of websites with and without the HON label. This study underscores the need for improving readability of web-based information on ADHD, and highlight important areas for improving the overall quality of websites. The use of HONCode label might guide general public in gauging the content quality of online health information, but not its readability or presentation.

Evaluation of Quality and Readability of Online Information on Treatments of Traumatic Dental Injuries

Background: Traumatic dental injuries is a common public health issue all over the world. The aim of this study was to investigate the quality and readability of web based information regarding treatments of traumatic dental injuries. Methods: An internet search was carried out using Google search engine with four different keywords (''treatment of broken teeth'', ''treatment of tooth trauma'', ''treatment of anterior tooth fractures in children'', ''fractured teeth treatments'') related to treatments of traumatic dental injuries. The duplicate websites, links to research studies, advertisements, discussion groups, videos and images were excluded from the evaluation. The quality of the written information regarding treatments of traumatic dental injuries in websites were analyzed with DISCERN toolkit. The readability of the written information was evaluated with Flesch-Kincaid grade level (FKGL), Flesch-Kincaid Reading Ease (FKRE) and Simple Measure of Gobbledygook reading grade level (SMOG) index. Results: A total of 120 webpages were identified but only 42 of them were remained after exclusion criterias. Among these websites, 25 (59.5%) of them were belonged to private dental clinics, 9 (21.4%) of them were belonged to informative pages about health and dentistry and 8 (19.1%) of them were belonged to professional organizations' websites. The average DISCERN score was 42.4 and ranged between 24 to 62. The average FKRE was 50.7 ± 14. which is equivalent to a reading level of 10th to 12nd grade. The average FKGL score for was 8.5 ± 2.8 and the average SMOG score was 7.3 ± 2.0 for all the websites. Conclusions: The results of the present study showed that the quality of information on the investigated websites was fair according to the DISCERN toolkit and too difficult to read and interpret for the general population. Healthcare information providers should focus on improving the quality and readability of the information regarding treatment of traumatic dental injuries.

Academic and community hernia center websites in the United States fail to meet healthcare literacy standards of readability.

Health literacy is considered the single best predictor of health status. Organizations including the American Medical Association (AMA) and the National Institutes of Health (NIH) have recommended that the readability of patient education materials not exceed the sixth-grade level. Our study focuses on the readability of self-designated hernia centers websites at both academic and community organizations across the United States to determine their ability to dispense patient information at an appropriate reading level. A search was conducted utilizing the Google search engine. The key words "Hernia Center" and "University Hernia Center" were used to identify links to surgical programs within the United States. The following readability tests were conducted via the program: Flesch-Kincaid Grade Level (FKGL), Gunning Fox Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG), and Flesch Reading Ease (FRE) score. Of 96 websites, zero (0%) had fulfilled the recommended reading level in all four tests. The mean test scores for all non-academic centers (n = 50) were as follows: FKGL (11.14 ± 2.68), GFI (14.39 ± 3.07), CLI (9.29 ± 2.48) and SMOG (13.38 ± 2.03). The mean test scores [SK1] for all academic programs (n = 46) were as follows: FKGL (11.7 ± 2.66), GFI (15.01 ± 2.99), CLI (9.34 ± 1.91) and SMOG (13.71 ± 2.02). A one-sample t test was performed to compare the FKGL, GFI, CLI, and SMOG scores for each hernia center to a value of 6.9 (6.9 or less is considered an acceptable reading level) and a p value of 0.001 for all four tests were noted demonstrating statistical significance. The Academic and Community readability scores for both groups were compared to each other with a two-sample t test with a p value of > 0.05 for all four tests and there were no statistically significant differences. Neither Academic nor Community hernia centers met the appropriate reading level of sixth-grade or less. Steps moving forward to improve patient comprehension and/or involving with their care should include appropriate reading level material, identification of a patient with a low literacy level with intervention or additional counseling when appropriate, and the addition of adjunct learning materials such as videos.

Readability of Online Spine Patient Education Resources

We assessed the readability of spine-related patient education materials on professional society websites to determine whether this had improved since last studied. We also compared the readability of these materials to a more patient-centered source, such as WebMD. Patient education pages from the American Association of Neurologic Surgeons (AANS), North American Spine Society (NASS), and spine-related pages from the American Academy of Orthopaedic Surgeons (AAOS), and WebMD were reviewed. Readability was evaluated using the Flesch Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) formulas. The mean FKGL and FRE scores of the societies were compared using one-way analysis of variance. The rate of a reading level at or below an eighth grade level was compared using the χ2 test. We analyzed a total of 156 sites. The mean FKGL score for the professional society sites was 11.4. The mean FRE score for the professional societies was 45.8, with 14.4% written at or below an eighth grade reading level. We found a significant difference in the FKGL scores and materials at or below the eighth grade level between the AAOS and AANS and AAOS and NASS. The mean FKGL and FRE scores for WebMD were 7.57 and 68.1, respectively, with a significant difference compared with the scores for the AAOS, NASS, and AANS. In addition, 80% of the WebMD materials had been written at or below the eighth grade reading level. A significant difference compared with the AANS and NASS (P < 0.0001) but not for the AAOS (P= 0.059). The average readability of spine-related topics exceeded the eighth grade reading level. The AAOS resources had better readability compared with the NASS and AANS. We found no improvement in readability since last studied. The readability of professional societies' materials was significantly worse than those from WebMD.

Web-Based Patient Educational Material on Osteosarcoma: Quantitative Assessment of Readability and Understandability.

BackgroundPatients often turn to web-based resources following the diagnosis of osteosarcoma. To be fully understood by average American adults, the American Medical Association (AMA) and National Institutes of Health (NIH) recommend web-based health information to be written at a 6th grade level or lower. Previous analyses of osteosarcoma resources have not measured whether text is written such that readers can process key information (understandability) or identify available actions to take (actionability). The Patient Education Materials Assessment Tool (PEMAT) is a validated measurement of understandability and actionability.ObjectiveThe purpose of this study was to evaluate web-based osteosarcoma resources using measures of readability, understandability, and actionability.MethodsUsing the search term “osteosarcoma,” two independent Google searches were performed on March 7, 2020 (by AGS), and March 11, 2020 (by TRG). The top 50 results were collected. Websites were included if they were directed at providing patient education on osteosarcoma. Readability was quantified using validated algorithms: Flesh-Kincaid Grade Ease (FKGE), Flesch-Kincaid Grade-Level (FKGL). A higher FKGE score indicates that the material is easier to read. All other readability scores represent the US school grade level. Two independent PEMAT assessments were performed with independent scores assigned for both understandability and actionability. A PEMAT score of 70% or below is considered poorly understandable or poorly actionable. Statistical significance was defined as P≤.05.ResultsTwo searches yielded 53 unique websites, of which 37 (70%) met the inclusion criteria. The mean FKGE and FKGL scores were 40.8 (SD 13.6) and 12.0 (SD 2.4), respectively. No website scored within the acceptable NIH or AHA recommended reading level. Only 4 (11%) and 1 (3%) website met the acceptable understandability and actionability threshold. Both understandability and actionability were positively correlated with FKGE (ρ=0.55, P<.001; ρ=0.60, P<.001), but were otherwise not significantly associated with other readability scores. There were no associations between readability (P=.15), understandability (P=.20), or actionability (P=.31) scores and Google rank.ConclusionsOverall, web-based osteosarcoma patient educational materials scored poorly with respect to readability, understandability, and actionability. None of the web-based resources scored at the recommended reading level. Only 4 achieved the appropriate score to be considered understandable by the general public. Authors of patient resources should incorporate PEMAT and readability criteria to improve web-based resources to support patient understanding.

Discharge summaries provided to owners of pets newly diagnosed with cancer exceed recommended readability levels.

To analyze the readability of discharge summaries distributed to owners of pets newly diagnosed with cancer. 118 discharge summaries provided to pet owners following initial consultation. A database search identified records of new patients that had been presented to the North Carolina State Veterinary Hospital medical oncology service between June 2017 and January 2019. Owner-directed portions of the summaries provided at the time of discharge were copied and pasted into a document and stripped of all identifying information. Readability of summaries was assessed with the use of 2 previously established readability calculators: the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) tests. Mean ± SD FKGL was 11.9 ± 1.1 (median, 11.9; range, 8.6 to 15.5; target ≤ 6), and the mean ± SD FRE score was 43 ± 5.9 (median, 42.7; range, 25.5 to 58.1; target ≥ 60). There were no significant differences in FKGL or FRE scores among discharge summaries for patients with the 4 most common tumor types diagnosed or the described treatment options. Ninety-three percent (110/118) of summaries were scored as difficult or very difficult to read. Owner-directed written information regarding a diagnosis of cancer at a single teaching hospital exceeded readability levels recommended by the American Medical Association and NIH and was above the average reading level of most US adults. Efforts to improve readability are an important component of promoting relationship-centered care and may improve owner compliance and patient outcomes.

Readability and Suitability of Online Patient Education Materials for Glaucoma

To assess the quality, content, readability, and accountability of information about glaucoma found online. Cross-sectional study. Thirteen websites containing patient education materials for glaucoma were analyzed in this study. An online Google search was conducted using the keyword "glaucoma." Thirteen medical website results were selected for analysis. Each website was assessed by 3 independent reviewers for suitability, readability, and accountability. The standardized Suitability Assessment of Materials (SAM) tool was used to evaluate the quality and content of information on each website. The Flesch Reading Ease (FRE) score, Flesch-Kincaid Grade Level (FKGL), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG) index, Automated Readability Index (ARI), and Linsear Write Formula (LWF) score were used to assess the readability of the websites. The Journal of the American Medical Association (JAMA) accountability benchmarks were used to evaluate each website's accountability. These included SAM, FRE, FKGL, CLI, SMOG index, ARI, and LWF scores as well as JAMA accountability benchmarks. The average SAM score for all the websites included in this study was 18 points out of a possible 34 points. Eyewiki.org was the lowest-scoring website (11.7 ± 0.6 points), whereas aao.org and nei.nih.gov were the highest-scoring websites (26.0 ± 1.0 points and 26.0 ± 2.6 points, respectively). Three content graders in this study were in moderate agreement (kappa statistic= 0.50). The average FRE score among all the websites was 47.0 (95% confidence interval [CI], 39.3-54.7). The average reading grade score among all the websites was 11.2 (95% CI, 10.0-12.4). Two of the 13 websites (15.4%) satisfied all 4 JAMA accountability criteria. There is significant variation in the content and quality of freely available, online glaucoma education material. The material is generally either not suitable or only adequate for use. Most websites reviewed are written at a reading grade level higher than that recommended for patient education materials.

What Did They Learn? Objective Assessment Tools Show Mixed Effects of Training on Science Communication Behaviors

There is widespread agreement about the need to assess the success of programs training scientists to communicate more effectively with non-professional audiences. However, there is little agreement about how that should be done. What do we mean when we talk about “effective communication”? What should we measure? How should we measure it? Evaluation of communication training programs often incorporates the views of students or trainers themselves, although this is widely understood to bias the assessment. We recently completed a 3-year experiment to use audiences of non-scientists to evaluate the effect of training on STEM (Science, Technology, Engineering and Math) graduate students’ communication ability. Overall, audiences rated STEM grad students’ communication performance no better after training than before, as we reported in Rubega et al. 2018. However, audience ratings do not reveal whether training changed specific trainee communication behaviors (e.g., jargon use, narrative techniques) even if too little to affect trainees’ overall success. Here we measure trainee communication behavior directly, using multiple textual analysis tools and analysis of trainees’ body language during videotaped talks. We found that student use of jargon declined after training but that use of narrative techniques did not increase. Flesch Reading Ease and Flesch-Kincaid Grade Level scores, used as indicators of complexity of sentences and word choice, were no different after instruction. Trainees’ movement of hands and hesitancy during talks was correlated negatively with audience ratings of credibility and clarity; smiling, on the other hand, was correlated with improvement in credibility, clarity and engagement scores given by audience members. We show that objective tools can be used to measure the success of communication training programs, that non-verbal cues are associated with audience judgments, and that an intensive communication course does change some, if not all, communication behaviors.

Zenker's Diverticulum: Readability and Quality of Online Written Education Materials.

To assess the readability and quality of online materials for Zenker's diverticulum. A Google search of "Zenker's diverticulum" was performed and the first 50 websites were reviewed. Readability was measured by Flesch Reading Ease (FRES), Flesch-Kincaid Grade Level (FKGL) and the Simple Measure of Gobbledygook (SMOG) scores using an online calculator. Understandability and actionability were assessed with the Patient Education Materials Assessment Tool for Printed Materials (PEMAT-P). Websites were categorized as either patient-targeted or physician-targeted and unpaired t tests were used to compare scores between these two groups. A total of 31 websites (23 patient-oriented sites and 8 physician-oriented sites) were analyzed. The average FRES, FKGL, and SMOG scores were 29.96 ± 14.72, 13.75 ± 2.78, and 12.19 ± 2.02, respectively, for the entire cohort. Readability scores for the patient-oriented sites were better than the physician-oriented sites for each measure (FRES 36.21 ± 8.86 vs 11.96 ± 13.54, FKGL 12.68 ± 1.58 vs 16.85 ± 3.25, and SMOG 11.37 ± 1.26 vs 14.53 ± 1.98; p < 0.001 for all comparisons). PEMAT-P understandability and actionability scores for the entire cohort were 68.20% ± 10.19% and 10.75% ± 16.52% respectively, and did not differ between patient-oriented and physician oriented sites. Though patient-oriented sites had better readability than physician-oriented sites for Zenker's Diverticulum, both categories had readability levels more advanced than what is recommended for medical education materials.