American Medical Association Recommends Research Articles

Expanding Accessibility in Cleft Care: The Role of Artificial Intelligence in Improving Literacy of Alveolar Bone Grafting Information.

The American Medical Association (AMA) recommends patient education materials (PEMs) be written at or below a sixth grade reading level. This study seeks to determine the quality, readability, and content of available alveolar bone grafting (ABG) PEMs and determine if artificial intelligence can improve PEM readability. Review of free online PEMs. Online ABG PEMs were retrieved from different authoring body types (hospital/academic center, medical society, or private practice). None. Content was assessed by screening PEMs for specific ABG-related topics. Quality was evaluated with the Patient Education Material Assessment Tool (PEMAT), which has measures of understandability and actionability. Open-access readability software (WebFX) determined readability with Flesch Reading Ease, Flesch-Kincaid Grade Level, and Gunning-Fog Index. PEMs were rewritten with ChatGPT, and readability metrics were reassessed. Quality, readability, and content of ABG PEMs. 34 PEMs were analyzed. Regarding quality, the average PEMAT-understandability score was 67.0 ± 16.2%, almost at the minimum acceptable score of 70.0% (p = 0.281). The average PEMAT-actionability score was low at 33.0 ± 24.1%. Regarding readability, the average Flesch Reading Ease score was 64.6 ± 12.8, categorized as "standard/plain English." The average Flesch-Kincaid Grade Level was 8.0 ± 2.3, significantly higher than AMA recommendations (p < 0.0001). PEM rewriting with ChatGPT improved Flesch-Kincaid Grade Level to 6.1 ± 1.3 (p < 0.0001). Available ABG PEMs are above the recommended reading level, yet ChatGPT can improve PEM readability. Future studies should improve areas of ABG PEMs that are most lacking, such as actionability.

Assessing the Readability, Reliability, and Quality of AI-Modified and Generated Patient Education Materials for Endoscopic Skull Base Surgery.

Despite National Institutes of Health and American Medical Association recommendations to publish online patient education materials at or below sixth-grade literacy, those pertaining to endoscopic skull base surgery (ESBS) have lacked readability and quality. ChatGPT is an artificial intelligence (AI) system capable of synthesizing vast internet data to generate responses to user queries but its utility in improving patient education materials has not been explored. To examine the current state of readability and quality of online patient education materials and determined the utility of ChatGPT for improving articles and generating patient education materials. An article search was performed utilizing 10 different search terms related to ESBS. The ten least readable existing patient-facing articles were modified with ChatGPT and iterative queries were used to generate an article de novo. The Flesch Reading Ease (FRE) and related metrics measured overall readability and content literacy level, while DISCERN assessed article reliability and quality. Sixty-six articles were located. ChatGPT improved FRE readability of the 10 least readable online articles (19.7 ± 4.4 vs. 56.9 ± 5.9, p < 0.001), from university to 10th grade level. The generated article was more readable than 48.5% of articles (38.9 vs. 39.4 ± 12.4) and higher quality than 94% (51.0 vs. 37.6 ± 6.1). 56.7% of the online articles had "poor" quality. ChatGPT improves the readability of articles, though most still remain above the recommended literacy level for patient education materials. With iterative queries, ChatGPT can generate more reliable and higher quality patient education materials compared to most existing online articles and can be tailored to match readability of average online articles.

Readability Analysis of Spanish-Language Patient-Reported Outcome Measures in Orthopaedic Surgery.

Patient-reported outcome measures (PROMs) are increasingly used to quantify patient symptomatology when assessing intervention effectiveness as well as to guide patient-centered health-care management and research. Previous studies have analyzed the readability of PROMs used in orthopaedics in the English language; however, the readability of PROMs used in orthopaedics in the Spanish language has not been evaluated. Given the increasing number of Spanish-speaking individuals seeking orthopaedic care in the United States, it is imperative that PROMs are at an adequate reading level for patients to answer appropriately in their native language. This study aimed to assess the readability of publicly available Spanish-language PROMs used in orthopaedics. Searches of the PubMed, Google Scholar, and Embase databases were conducted to identify publicly available Spanish-language versions of PROMs used in orthopaedics. Additionally, the PROMIS Health Organization was contacted to obtain Spanish-language versions of the PROMIS (Patient-Reported Outcomes Measurement Information System) questionnaires. A total of 42 Spanish-language PROMs were identified and included in this study. The text of each PROM was inserted into multilingual readability software (legible.es), which analyzed the readability of each PROM using the Fernández Huerta and Índice de Legibilidad de Flesch-Szigriszt (INFLESZ) readability formulas. The mean and standard deviation (SD) of the raw readability scores were calculated for the PROMs. The percentage of PROMs at or below the 6th-grade reading level was also calculated. The mean readability of PROMs using the INFLESZ formula was at a 7th to 10th-grade reading level (mean = 63, SD = 16), and the mean readability using the Fernández Huerta formula was 68 (SD = 15). On average, 57% of the PROMs used in orthopaedics were at or below the INFLESZ 6th-grade readability level. Approximately half of Spanish-language PROMs used in orthopaedics are written at a reading level that does not meet the National Institutes of Health and the American Medical Association recommendations at or below the 6th-grade level. PROM developers and translators should consider the importance of readability when translating PROMs, to maximize their use and efficacy among orthopaedic patients speaking that language.

Validation of ChatGPT 3.5 as a Tool to Optimize Readability of Patient-facing Craniofacial Education Materials.

To address patient health literacy, the American Medical Association recommends that readability of patient education materials should not exceed a sixth grade reading level; the National Institutes of Health recommend no greater than an eigth-grade reading level. However, patient-facing materials in plastic surgery often remain at an above-recommended average reading level. The purpose of this study was to evaluate ChatGPT 3.5 as a tool for optimizing patient-facing craniofacial education materials. Eighteen patient-facing craniofacial education materials were evaluated for readability by a traditional calculator and ChatGPT 3.5. The resulting scores were compared. The original excerpts were then inputted to ChatGPT 3.5 and simplified by the artificial intelligence tool. The simplified excerpts were scored by the calculators. The difference in scores for the original excerpts between the online calculator and ChatGPT 3.5 were not significant (P = 0.441). Additionally, the simplified excerpts' scores were significantly lower than the originals (P < 0.001), and the mean of the simplified excerpts was 7.78, less than the maximum recommended 8. The use of ChatGPT 3.5 for simplification and readability analysis of patient-facing craniofacial materials is efficient and may help facilitate the conveyance of important health information. ChatGPT 3.5 rendered readability scores comparable to traditional readability calculators, in addition to excerpt-specific feedback. It was also able to simplify materials to the recommended grade levels. With human oversight, we validate this tool for readability analysis and simplification.

Readability of Spanish-Language Online Patient Educational Materials for Peripheral Artery Disease Do Not Meet Recommended Standards and Represent a Literacy Barrier to Care

Online resources are a valuable source of information for patients and have been reported to improve engagement and adherence to medical care. However, readability of online patient educational materials (OPEMs) is crucial for them to serve their intended purpose. The American Medical Association (AMA) recommends that OPEM be written at or below the sixth grade reading level. To avoid disparities in access to comprehensible health information on peripheral artery disease (PAD), it is imperative that the readability of PAD OPEM is appropriate for both English-speaking and Spanish-speaking patients. The aim of this study is to evaluate the readability of PAD OPEM in Spanish and compare to English-language OPEM. We conducted a Google search in English and Spanish using "peripheral arterial disease" and "enfermedad arterial periferica", respectively, and the top 25 patient-accessible articles were collected for each. Articles were categorized by source type: hospital, professional society, or other. Readability of English-language OPEM was measured using the Flesch Reading Ease Readability Formula, Automated Readability Index, Coleman-Liau Index, Flesch-Kincaid Grade Level, Gunning Fog, Linsear Write Formula, and the Simple Measure of Gobbledygook Index. Readability of Spanish OPEM was measured using the Fernández-Huerta Index and Índice Flesch-Szigriszt Scale. Readability of the articles was compared to the AMA recommendation, between English- and Spanish-language, and across sources using statistical tests appropriate to the data. OPEM from professional societies represented the fewest number of English- (n=7, 28%) and Spanish-language (n=6, 24%) articles. Most English-speaking (n=18, 72%) and Spanish-language (n=20, 80%) OPEM were considered difficult as measured by the Flesch Reading Ease Readability Formula and Fernández-Huerta Index, respectively, but did not significantly differ between languages (P=0.59). There were no significant differences in the average readability of all readability measurements across sources (hospital, professional society, or other). All the average readability grade levels for English-speaking and Spanish-language OPEM was significantly higher than the sixth grade reading level (P<0.01). Only 3 (6%) OPEM met the AMA recommended reading level and there was no significant difference between English-language and Spanish-language OPEM (P=1.0). Nearly all Spanish-language and English-language PAD OPEM assessed were written at a reading grade level higher than recommended by the AMA. There was no significant difference in the readability of materials from hospitals or professional societies. To prevent further widening of health disparities related to literacy, health content creators, particularly hospitals and professional societies, should prioritize, develop, and ensure that English-language and Spanish-language patient education materials are written at a level appropriate for the public.

LB895 Readability, Understandability, and Actionability of Online Patient Education Materials for Sunscreen

Most adults in the United States obtain health information online. It is crucial that online patient education materials (PEMs) are easily interpreted in order to successfully impact patients’ health behaviors. This is especially important for sunscreen PEMs, as proper use is associated with decreased risks of melanoma, non-melanoma skin cancers, and photoaging. American Medical Association recommends that PEMs be written at the sixth grade or lower reading level. We analyzed the readability of PEMs for sunscreen in order to identify opportunities to optimize PEMs.

Readability of Online Patient Education Materials for Glaucoma.

We assessed the readability of online glaucoma patient education materials using seven validated instruments. Overall, glaucoma materials were written at a 10th to 11th grade level, above the recommended seventh grade reading level. Online health information is increasingly used by patients, yet previous studies show online patient education materials are often difficult to understand. As such, the American Medical Association recommends that patient education materials are written at or below a seventh grade reading level. This study aimed to assess the readability of online glaucoma patient education materials. Glaucoma was entered into the Google search engine, and the first 30 search results were assessed for readability using seven validated readability instruments. Scientific articles, forums, and dictionary entries were excluded. Single sample t tests were used to assess whether online glaucoma materials were written above the recommended seventh grade level. Overall, glaucoma materials were written at a mean grade level of 10.33 (SD: 2.02). Across 6 grade level readability instruments, these patient education materials were written above the recommended seventh grade reading level (P<0.0001 for all). Glaucoma education materials only on the first page of Google search results were of a similar reading level: mean 10.56 (SD: 2.13). The readability instruments used in this study showed strong consistency. Glaucoma patient education materials are written above the recommended reading level to promote accessibility of education materials. This may contribute to lower patient engagement, worse clinical outcomes, and greater racial and ethnic disparities in glaucoma management. There is a need for reliable, simple glaucoma information to improve patient outcomes.

AMA Releases New Substance Use, Pain Recommendations

Back to table of contents Previous article Next article Clinical & ResearchFull AccessAMA Releases New Substance Use, Pain RecommendationsTerri D'ArrigoTerri D'ArrigoSearch for more papers by this authorPublished Online:27 Nov 2021https://doi.org/10.1176/appi.pn.2021.12.15AbstractThe recommendations by the Substance Use and Pain Care Task Force take a multipronged approach to ensuring evidence-based care is available to all who need it.In September the American Medical Association (AMA) released recommendations for physicians and lawmakers to help end drug overdoses and deaths. The recommendations were created by a new task force, the Substance Use and Pain Care Task Force, which combines AMA’s Opioid Task Force and Pain Care Task Force and includes APA and more than two dozen other health organizations.The recommendations focus on broad efforts to remove barriers and improve access to evidence-based care for patients with pain and/or substance use disorder (SUD) as well as to increase access to strategies for harm reduction.The recommendations include the following strategies:Support patients with pain and/or mental illness, including SUD, by building an evidence-based, sustainable, and resilient infrastructure and health care workforce rather than continuing a crisis-driven approach that has led to multiple unintended negative consequences, including one-size-fits-all strategies; continued stigma; and widespread gaps in data, evidence-based treatment, and prevention efforts.Support coverage for, access to, and payment of comprehensive, multidisciplinary, multimodal evidence-based treatment for patients with pain, SUD, and/or co-occurring mental illness.Include—through coverage, access, and payment—ways of directly addressing racial, gender, sexual orientation, ethnic, and economic inequities as well as social determinants of health. This includes removing barriers to evidence-based treatment for SUD, co-occurring mental illness, and pain.Broaden public health and harm reduction strategies to save lives from overdose, limit the spread of infectious disease, eliminate stigma, and reduce harms for people who take illicit substances. This includes increased support for naloxone and sterile needle and syringe exchange services.Improve stakeholder and multisector collaboration to ensure that the patients, policymakers, employers, and communities benefit from evidence-based decisions.“Physicians must continue to lead by example to help our patients with pain, SUDs, and mental illness,” AMA Board Chair and Task Force Chair Bobby Mukkamala, M.D., an otolaryngologist in Flint, Mich., said in a statement. “Removing barriers requires policymakers to join us in establishing a more effective and humane approach. Failure to adopt these policies will prolong the epidemic and our patients’ suffering.”APA President Vivian B. Pender, M.D., hailed the recommendations as a foundation for meeting the epidemic of drug overdose and mortality head-on.“We must approach this crisis from every angle and ensure that patients with comorbid pain and patients in marginalized communities have access to treatment that is offered by health and mental health professionals who understand the intricacies of substance use disorders,” Pender told Psychiatric News. “As psychiatrists, it is our responsibility not only to advocate for accessible, evidence-based care, but to offer it in a respectful way that addresses each patient as an individual with individual needs.”Smita Das, M.D., Ph.D., M.P.H., chair of APA’s Council on Addiction Psychiatry and a clinical assistant professor of psychiatry and behavioral health sciences at Stanford University School of Medicine, voiced support for the recommendations on behalf of the council.“Substance use disorders and mental illness are underaddressed and undertreated, and these recommendations will hopefully have an impact on our patients, colleagues, and systems,” Das told Psychiatric News. “Many of our patients also suffer from pain, so the joint recommendations are helpful. We especially appreciate the emphasis on evidence-based strategies and reducing barriers and stigma.”Das said that expanding access to care and expanding the treatment workforce are critical to meeting the recommendations regarding patient support.“APA has been advocating for improved education and training of all physicians to better screen, identify, and treat substance use disorders concurrently with other dual diagnoses,” Das said. “In addition to improved training and workforce strengthening, the other challenge will be to update systems of care to deliver on the AMA recommendations.”Das encourages psychiatrists to be proactive in helping the AMA recommendations materialize, particularly with respect to the recommendation about supporting coverage for, access to, and payment of comprehensive, evidence-based treatment for patients with pain, SUD, and/or co-occurring mental illness.“For individual psychiatrists, approaching SUD and pain with special attention to [this recommendation] can reduce the stigma and barriers our patients face,” Das said. “Screening for substance use, being familiar with treatment options, and [offering] harm reduction [strategies] can improve a patient’s journey and outcomes.” ■The AMA recommendations are posted here.Information about APA’s educational resources on SUD, medication treatment for opioid use disorder, and curriculum development for SUD education are posted here. ISSUES NewArchived

Are English-language online patient education materials related to breast cancer risk assessment understandable, readable, and actionable?

PurposeTo evaluate the readability, understandability, and actionability of online patient education materials (OPEM) related to breast cancer risk assessment. Material and methodsWe queried seven English-language search terms related to breast cancer risk assessment: breast cancer high-risk, breast cancer risk factors, breast cancer family history, BRCA, breast cancer risk assessment, Tyrer-Cuzick, and Gail model. Websites were categorized as: academic/hospital-based, commercial, government, non-profit or academic based on the organization hosting the site. Grade-level readability of qualifying websites and categories was determined using readability metrics and generalized estimating equations based on written content only. Readability scores were compared to the recommended parameters set by the American Medical Association (AMA). Understandability and actionability of OPEM related to breast cancer high-risk were evaluated using the Patient Education Materials Assessment Tool (PEMAT) and compared to criteria set at ≥70%. Descriptive statistics and inter-rater reliability analysis were utilized. Results343 websites were identified, of which 162 met study inclusion criteria. The average grade readability score was 12.1 across all websites (range 10.8–13.4). No website met the AMA recommendation. Commercial websites demonstrated the highest overall average readability of 13.1. Of the 26 websites related to the search term breast cancer high-risk, the average understandability and actionability scores were 62% and 34% respectively, both below criteria. ConclusionsOPEM on breast cancer risk assessment available to the general public do not meet criteria for readability, understandability, or actionability. To ensure patient comprehension of medical information online, future information should be published in simpler, more appropriate terms.

Evaluating the quality and readability of online information on keratoconus treatment

ObjectiveThis study aims to evaluate the quality and readability of online resources on keratoconus treatment. MethodsA Google.com search was conducted on August 9, 2020; 32 web sites were selected for analysis. Popularity was assessed by Google and Alexa rank. The quality of web sites was analyzed using the quality criteria for consumer health information (DISCERN) tool, the Journal of the American Medical Association (JAMA) benchmark, and the Health On the Net Code of Conduct Certification (HONcode). The readability of the web sites was assessed using the Fleschwebr hea Reading Ease, the Automated Readability Index, and the Fleschted Readability ResultsThe JAMA benchmark scores, unlike the DISCERN scores, were correlated with the Google and Alexa rank. One web site (3.1%) met all the JAMA benchmark criteria, and 3 (9.3%) others had HONcode certification. The median DISCERN score was 33 (range, 29.6–43.1; maximum possible, 80). Rnib.org.uk scored the highest at 57 (71.0%). The mean Flesch–Kincaid Reading Ease score (52.9 ± 7.1) corresponded to uk" n DIdifficult to read.” Thirty-one web sites (96.8%) had a Flesch–Kincaid Grade higher than the American Medical Association recommendation of sixth grade level. The median Automated Readability Index score was 7 (range, 6.2–7.3). ConclusionThe majority of online information currently available on keratoconus treatment is complex and highly variable. Rnib.org.uk is the best currently available source. Clinicians should inform patients on how to assess the credibility of online information and recommend suitable information sources.

The Readability of Patient Education Materials Pertaining to Gastrointestinal Procedures.

Introduction Due to the ubiquity and ease of access of Internet, patients are able to access online health information more easily than ever. The American Medical Association recommends that patient education materials be targeted at or below the 6th grade level in order to accommodate a wider audience. In this study, we evaluate the difficulty of educational materials pertaining to common GI procedures; we analyze on the readability of online education materials for colonoscopy, flexible sigmoidoscopy, and esophagogastroduodenoscopy (EGD). Methods Google search was performed using keywords of “colonoscopy,” “sigmoidoscopy,” and “EGD” with “patient information” at the end of each search term. The texts from a total of 18 studies, 6 for each of the procedures, were then saved. Each study was also subdivided into “Introduction,” “Preparation,” “Complications,” and if available, “Alternatives.” Furthermore, medical terminology that was properly explained, proper nouns, medication names, and other copyright text were removed in order to prevent inflation of the difficulty. Five validated readability tests were used to analyze each study and subsections: Coleman-Liau, New Dale-Chall, Flesch-Kincaid, Gunning Fog, SMOG. Results Studies on colonoscopy, flexible sigmoidoscopy, and EGD had median readability grades of 9.7, 10.2, and 11.0, respectively. Analysis of the subsections revealed that the “Alternative” subsection was the most difficult to comprehend with a readability score of 11.4, whereas the “Introduction” subsection was the easiest to comprehend with a readability score of 9.5. Conclusion Despite modifications to the studies that improved the readability scores, patient education materials were still significantly above the recommended 6th grade level across all websites. This study emphasizes that clear and simple language is warranted in order to create information that is suitable for most patients.

In Case You Haven't Heard…

In a new chapter in the history of the American Medical Association (AMA), which might be titled “AMA vs. Physicians for Responsible Opioid Prescribing (PROP),” the AMA sharply criticizes PROP (known for its affiliation with anti‐opioid activist and expert witness for the government Andrew Kolodny, M.D.) for accusing it of siding with the pharmaceutical industry by backing off from the Centers for Disease Control and Prevention's (CDC's) 2016 opioid guidelines. For the PROP letter, see https://www.bmj.com/content/372/bmj.m4901/rr‐1. For the AMA's response, praised by pain patient advocacy organizations, see https://searchlf.ama‐assn.org/letter/documentDownload?uri=/unstructured/binary/letter/LETTERS/2021‐2‐19‐AMA‐Response‐Letter‐to‐PROP‐FINAL.pdf. The AMA has “always focused on a multipronged approach to help patients with pain, patients with a substance use disorder, and harm reduction efforts to help save lives from overdose and mitigate other harms,” the letter states. One of the harms of the prescribing guidelines, which the CDC itself said have been “misapplied” (although this admission wasn't made until opioid overdoses continued to increase long after prescriptions were cut back), was that pain patients continued to suffer. People didn't want to get needed procedures because they were told they would only get Tylenol for pain. The AMA's recommendations, which PROP opposed, “highlighted that the nation now faced a drug overdose epidemic fueled by multiple substances, combined, adulterated, and resulting in a staggering increase in death from illicitly manufactured fentanyl, fentanyl analogs, methamphetamine, cocaine, and heroin.” Also see https://onlinelibrary.wiley.com/doi/10.1002/adaw.32764.

The quality, suitability, content and readability of online health-related information regarding sexual dysfunction after rectal cancer surgery.

Patients are not well informed about sexual dysfunction after rectal cancer surgery and often turn to the Internet for information. The purpose of this study was to assess online information for patients on sexual dysfunction after rectal cancer surgery. An online search of Google, Yahoo and Bing was performed using specific (e.g., rectal cancer surgery and vaginal pain) and general (e.g., rectal cancer surgery and sex) search terms. Inclusion criteria were websites in English, designed for patients, and including content regarding sexual dysfunction after rectal cancer surgery. Websites were assessed for readability (nine standardized tests), quality (DISCERN tool), suitability (Suitability Assessment of Materials tool), and content. Of 5040 websites identified, 99 unique websites met inclusion criteria. Three (3%) websites fulfilled the American Medical Association recommendation of a 6th-grade reading level. Using the DISCERN instrument, two (2%) websites were assigned good/excellent quality, nineteen (19%) referenced their sources of information, and thirty-one (31%) fully discussed the impact of sexual dysfunction on quality of life. Using the SAM instrument, three (3%) websites were classified as highly suitable for rectal cancer patients, sixty-five (66%) were adequate, and thirty-one (31%) were inadequate. With regards to content, nine (9%) websites fully discussed the impact of sexual dysfunction on patients partners and fifty-one (52%) websites did not cover prognosis. Online health information available to patients on sexual dysfunction after rectal cancer surgery is suboptimal. Websites are not suitable, lack important content, and are written at too complex a reading level for patients.

Readability of Patient Education Materials From High-Impact Medical Journals: A 20-Year Analysis.

Comprehensive patient education is necessary for shared decision-making. While patient–provider conversations primarily drive patient education, patients also use published materials to enhance their understanding. In this investigation, we evaluated the readability of 2585 patient education materials published in high-impact medical journals from 1998 to 2018 and compared our findings to readability recommendations from national groups. For all materials, mean readability grade levels ranged from 11.2 to 13.8 by various metrics. Fifty-four (2.1%) materials met the American Medical Association recommendation of sixth grade reading level, and 215 (8.2%) met the National Institutes of Health recommendation of eighth grade level. When stratified by journal and material type, general medical education materials from Annals of Internal Medicine were the most readable (P < .001), with 79.8% meeting the eighth grade level. Readability did not differ significantly over time. Efforts to standardize publication practice with the incorporation of readability evaluation during the review process may improve patients’ understanding of their disease processes and treatment options.

Evaluating the Readability of Online Patient Education Materials Related to Orthopedic Oncology.

The internet is increasingly used to access patient education materials. The average American reading level has been found to be that of a 7th- to 8th-grade student, prompting the National Institutes of Health (NIH) and the American Medical Association (AMA) to advise that patient education materials be written between the 4th- to 6th-grade reading level. The purpose of this study was to evaluate the reading level of current patient education materials for the most common musculoskeletal oncological tumors. A Google search was performed with all location filters off to account for geographic variability for patient education materials related to 28 orthopedic primary or secondary tumors. All patient education articles from the first 10 website hits for each tumor type were analyzed. Patient education materials from these websites were evaluated using 8 validated readability scales. Patient resources were found to be written at an average grade level nearly double the NIH and AMA recommendation. Patient education materials for soft tissue chondromas were written at the highest level (14.8±1.9), whereas education materials for chordomas (10.1±1.0) most closely approached national recommendations, despite still being written at a readability level nearly 4 grade levels higher than has been recommended. The Flesch Reading Ease assessment provided a mean score of 46.5±7.7, corresponding with a "difficult to read" result. Current patient education materials regarding oncological musculoskeletal-related patient education materials are written significantly above the recommended reading level. Further modification of these resources is warranted to ensure adequate comprehension and informed decision making in the clinical setting. [Orthopedics. 2021;44(1):38-42.].

Assessing Readability and Quality of Online Patient Directed Information on Hysterectomies

Study Objective Hysterectomy is a commonly performed procedure. Accessible online patient information is highly important due to associated risks, implications of the procedure and potential treatment alternatives. The study aims to evaluate quality, comprehensiveness and readability of commonly accessed online patient information on hysterectomies. Design We included websites whose purpose was to provide hysterectomy information for patients. The first 25 webpages on 5 search engines were assessed by two reviewers. Quality of websites was assessed by validated tools for online material (DISCERN and JAMA benchmark). We assessed whether the websites included information on alternatives (including minimally invasive approaches), risks, and benefits. Lastly, readability scores were calculated using standardized tools to assess the grade level or years of education needed to comprehend the material (FKLG, Gunning FOG, SMOG, FRES). Setting N/A Patients or Participants Women seeking online information on hysterectomy to inform their treatment choices. Interventions Hysterectomy (On-line patient information). Measurements and Main Results Forty-seven websites were included according to our selection strategy. On average, websites were of good quality (53.2 ± 10.7 of 80, DISCERN criteria; 2.8 ± 1.2 of 4, JAMA criteria). The majority of websites described alternatives to hysterectomy (33/47, 70.2%) including minimally invasive options (21/47, 44.7%), and also described surgical risks (36/47, 76.6%) and benefits (46/47, 97.9%). However, website article readability corresponded to a grade 12 reading level (FKGL= 11.9 ± 2.3) or reading levels of 11 or 15 years of education (SMOG= 11.1 ± 1.7, Gunning FOG= 14.9 ± 2.4) suggesting these websites are difficult to read for the general population (40.7 ± 12.0 FRES). Conclusion Online patient information on hysterectomy is overall of good quality and contains the majority of the required information. However, the content is more difficult to read than the American Medical Association's recommendation for a grade six level. Website content creators should consider readability to make the content more accessible to patients.

Quality and Readability Assessment of Websites on Human Papillomavirus and Oropharyngeal Cancer.

The incidence of human papillomavirus-positive (HPV+) oropharyngeal cancer is rising, but public knowledge about this diagnosis remains low. This study aimed to investigate the quality and readability of online information about HPV+ oropharyngeal cancer. Cross-sectional website analysis. This study conducted a total of 12 web searches across Google, Yahoo, and Bing to identify websites related to HPV+ oropharyngeal cancer. The QUality Evaluation Scoring Tool (QUEST) was used to measure quality based on seven website criteria. The Flesch Reading Ease Score (FRES) and Flesch-Kincaid Grade Level (FKGL) were used to measure readability, with scores estimating the education level a reader would require to understand a piece of text. Readability improves as FRES increases and FKGL decreases. Twenty-seven unique web pages were evaluated. The mean USA reading grade level as measured by FKGL was 10.42 (standard deviation = 1.54). There was an inverse relationship between quality and readability, with a significant positive correlation between QUEST score and FKGL (r = 0.343, P = .040) and a significant negative correlation between QUEST score and FRES (r = -0.537, P = .002). With a mean USA reading grade level more than four grades above the American Medical Association's recommendation and results indicating that readability suffers as quality improves, these findings suggest that the currently available online information about HPV+ oropharyngeal cancer is insufficient. Improved patient education practices and resources about this diagnosis are needed. NA Laryngoscope, 131:87-94, 2021.

Assessment of readability, quality and popularity of online information on ureteral stents.

To evaluate the quality and readability of online information on ureteral stents. Google.com was queried using the search terms "ureteric stent", "ureteral stent", "double J stent" and, "Kidney stent" derived from Google AdWords. Website popularity was determined using Google Rank and the Alexa tool. Website quality assessment was performed using the following criteria: Journal of the American Medical Association (JAMA) benchmarks, Health on the Net (HON) criteria, and a customized DISCERN questionnaire. The customized DISCERN questionnaire was developed by combining the short validated DISCERN questionnaire with additional stent-specific items including definition, placement, complications, limitations, removal and "when to seek help". Scores related to stent items were considered as the "stent score" (SS). Readability was evaluated using five readability tests. Thirty-two websites were included. The mean customized DISCERN score and "stent score" were 27.1±7.1 (maximum possible score=59) and 14.6±3.8 (maximum possible score=24), respectively. A minority of websites adequately addressed "stent removal" and "when to seek medical attention". Only two websites (6.3%) had HON certification (drugs.com, radiologyinfo.org) and only one website (3.3%) met all JAMA criteria (bradyurology.blogspot.com). Readability level was higher than the American Medical Association recommendation of sixth-grade level for more than 75% of the websites. There was no correlation between Google rank, Alexa rank, and the quality scores (P>0.05). Among the 32 most popular websites on the topic of ureteral stents, online information was highly variable. The readability of many of the websites was far higher than standard recommendations and the online information was questionable in many cases. These findings suggest a need for improved online resources in order to better educate patients about ureteral stents and also should inform physicians that popular websites may have incomplete information.

Quality and readability of online patient information regarding sclerotherapy for venous malformations.

Patients often use the internet as a source of information about their condition and treatments. However, this information is unregulated and varies in quality. To evaluate the readability and quality of online information for pediatric and adult patients and caregivers regarding sclerotherapy for venous malformations. "Venous malformation sclerotherapy" was entered into Google, and results were reviewed until 20 sites that satisfied predefined inclusion criteria were identified. Scientific and non-patient-focused web pages were excluded. Readability was assessed using the Flesch Reading Ease Score and American Medical Association reading difficulty recommendations and quality was assessed using Journal of the American Medical Association standards and assessing if the site displayed HONcode (Health on the Net Code) certification. Assessment of the breadth of relevant information was made using a predefined checklist. Forty-nine search engine results were reviewed before 20 sites were identified for analysis. Average Flesch Reading Ease Score was 44 (range: 24.2-70.1), representing a "fairly difficult" reading level. None of the sites had a Flesch Reading Ease Score meeting the American Medical Association recommendation of 80-90. Only one site met all four Journal of the American Medical Association quality criteria (average: 2.1). None of the sites displayed a HONcode seal. The information most frequently found was: sclerotherapy is performed by radiologists, multiple treatments may be needed and surgery is an alternative treatment. Online information regarding sclerotherapy for venous malformations is heterogeneous in quality and breadth of information, and does not meet readability recommendations for patient information. Radiologists should be aware of and account for this when meeting patients.

Radiation Oncology and Online Patient Education Materials: Deviating From NIH and AMA Recommendations

Physicians encourage patients to be informed about their health care options, but much of the online health care-related resources can be beneficial only if patients are capable of comprehending it. This study's aim was to assess the readability level of online patient education resources for radiation oncology to conclude whether they meet the general public's health literacy needs as determined by the guidelines of the United States National Institutes of Health (NIH) and the American Medical Association (AMA). Radiation oncology-related internet-based patient education materials were downloaded from 5 major professional websites (American Society for Radiation Oncology, American Association of Physicists in Medicine, American Brachytherapy Society, RadiologyInfo.org, and Radiation Therapy Oncology Group). Additional patient education documents were downloaded by searching for key radiation oncology phrases using Google. A total of 135 articles were downloaded and assessed for their readability level using 10 quantitative readability scales that are widely accepted in the medical literature. When all 10 assessment tools for readability were taken into account, the 135 online patient education articles were written at an average grade level of 13.7±2.0. One hundred nine of the 135 articles (80.7%) required a high school graduate's comprehension level (12th-grade level or higher). Only 1 of the 135 articles (0.74%) met the AMA and NIH recommendations for patient education resources to be written between the third-grade and seventh-grade levels. Radiation oncology websites have patient education material written at an educational level above the NIH and AMA recommendations; as a result, average American patients may not be able to fully understand them. Rewriting radiation oncology patient education resources would likely contribute to the patients' understanding of their health and treatment options, making each physician-patient interaction more productive and efficient.