DISCERN Instrument Research Articles

Tiktok as a Source of Education and Misinformation in Lichen Sclerosus.

The purpose of this study is to evaluate the content, delivery, and quality of medical information for vulvar lichen sclerosus on the social media platform TikTok. This is a descriptive, cross-sectional study. Using the third-party data scraping tool Apify, TikTok videos tagged with #lichensclerosus or "lichen sclerosus" were identified and sorted by view count. A sample of 100 videos was reviewed by 2 independent reviewers, excluding those not discussing lichen sclerosus. Videos were assessed using a coding document, the Patient Educational Materials Assessment Tool, and the DISCERN instrument. Interrater reliability was measured, and statistical analyses included Fleiss' kappa, intraclass correlation coefficient, t tests, and Wilcoxon rank sum test with Holm-Bonferroni correction. Content creators included patients (46%), health care professionals (30%), and others. Topics focused on clinical disease (52%) and treatment (48%). Evidence-based medicine was discussed in 71.7% of treatment-related videos, while 51.7% included nonevidence-based approaches, with a neutral or positive sentiment. Videos discussing topical steroids often had negative sentiments. Quality assessment revealed 61% of videos were understandable, 27% actionable, and 46% contained misinformation. Videos by health care professionals had less misinformation and higher quality scores compared to patient-generated content. Commercially biased videos were more understandable but contained more misinformation. TikTok serves as a significant platform for sharing information on lichen sclerosus, but nearly half of the content contains misinformation. Health care professionals need to engage in social media to provide accurate information and counteract misinformation. Enhanced collaboration with patient advocates and careful resource sharing can improve the quality and reliability of medical information available online.

Accuracy of natural language processors for patients seeking inguinal hernia information.

NLPs such as ChatGPT are novel sources of online healthcare information that are readily accessible and integrated into internet search tools. The accuracy of NLP-generated responses to health information questions is unknown. We queried four NLPs (ChatGPT 3.5 and 4, Bard, and Claude 2.0) for responses to simulated patient questions about inguinal hernias and their management. Responses were graded on a Likert scale (1 poor to 5 excellent) for relevance, completeness, and accuracy. Responses were compiled and scored collectively for readability using the Flesch-Kincaid score and for educational quality using the DISCERN instrument, a validated tool for evaluating patient information materials. Responses were also compared to two gold-standard educational materials provided by SAGES and the ACS. Evaluations were performed by six hernia surgeons. The average NLP response scores for relevance, completeness, and accuracy were 4.76 (95% CI 4.70-4.80), 4.11 (95% CI 4.02-4.20), and 4.14 (95% CI 4.03-4.24), respectively. ChatGPT4 received higher accuracy scores (mean 4.43 [95% CI 4.37-4.50]) than Bard (mean 4.06 [95% CI 3.88-4.26]) and Claude 2.0 (mean 3.85 [95% CI 3.63-4.08]). The ACS document received the best scores for reading ease (55.2) and grade level (9.2); however, none of the documents achieved the readibility thresholds recommended by the American Medical Association. The ACS document also received the highest DISCERN score of 63.5 (57.0-70.1), and this was significantly higher compared to ChatGPT 4 (50.8 [95% CI 46.2-55.4]) and Claude 2.0 (48 [95% CI 41.6-54.4]). The evaluated NLPs provided relevant responses of reasonable accuracy to questions about inguinal hernia. Compiled NLP responses received relatively low readability and DISCERN scores, although results may improve as NLPs evolve or with adjustments in question wording. As surgical patients expand their use of NLPs for healthcare information, surgeons should be aware of the benefits and limitations of NLPs as patient education tools.

Assessing the readability and quality of online written information on epistaxis.

The objective of this study was to assess the readability and quality of online written information on epistaxis. The terms 'epistaxis' and 'nosebleed' were entered into Google. The first six webpages generated for each search term were screened. Readability was assessed using the Flesch-Kincaid Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG) Index and Gunning Fog Index (GFOG). Quality was assessed using the DISCERN instrument. Spearman's correlation between quality and readability was calculated. A total of 37 websites met the inclusion criteria. The mean and 95% confidence intervals for FRES, FKGL, SMOG and GFOG were 58.9 (55.3-62.5), 9.65 (8.74-10.6), 9.18 (8.57-9.8) and 12.5 (11.5-13.5), respectively. The DISCERN score was 34.3 (32.0-36.5). Weak negative correlation was noted between DISCERN and FRES (rs = -0.15, p = 0.36). Online information on epistaxis is generally of poor quality and low readability.

Potential Reinforcement of Health Misconceptions in YouTube Videos: Example of Elbow Enthesopathy (Tennis Elbow)

Background and Objectives: We evaluated the prevalence of potential reinforcement of common unhealthy misinterpretations of bodily sensations in social media (YouTube videos) addressing elbow enthesopathy (eECRB, enthesopathy of the extensor carpi radialis brevis, tennis elbow). Methods: We recorded video metric data on 139 unique YouTube videos when searching “lateral epicondylitis” and “tennis elbow.” We designed a rubric to assess the level of potential reinforcement of unhelpful thinking in videos about eECRB. Informational quality was scored with an adapted version of the DISCERN instrument. We then assessed the factors associated with these scores. Results: Sixty-five percent (91 of 139) of videos contained information reinforcing at least one common misconception regarding eECRB. Potential reinforcement of misconceptions was associated with longer video duration, higher likes per day, and higher likes per view. No factors were associated with information quality scores. Conclusions: These findings of a high prevalence of potential reinforcement of misconceptions in YouTube videos, in combination with the known associations of misconceptions with greater discomfort and incapability, point to the potential of such videos to harm health. Producers of patient facing health material can add avoidance of reinforcement of unhelpful thinking along with readability, accuracy, and relevance as a guiding principle.

Information Regarding Male Infertility in the Nordic Countries’ Fertility Clinics’ Websites: An Evaluation of Readability, Suitability, and Quality

Background There is a shortage of information on infertility targeted at men on the Internet, and the quality of websites on fertility-related information varies. This may lead men to seek information on the Internet, as the anonymity of online forums may appeal to some men. However, this information is not a substitute for proper diagnosis and treatment. Aim To examine whether online information on male infertility on private or university infertility clinics in Finland, Denmark, Norway, and Sweden is readable, suitable, and of good quality. Methods A cross-sectional design was used to evaluate 83 fertility clinics’ websites focusing on male infertility by using the Suitability Assessment of Materials (SAM), and the DISCERN instrument. Results The results indicated that information pertaining to male infertility was scarce but had an “adequate” readability and was easy to comprehend. The findings also pinpoint several elements on websites that necessitate enhancement, notably updated evidence-based information. Conclusions The conclusion was that the information on the websites must be reliable, so that men can base their choices concerning their fertility issues on trustworthy facts. It is crucial that healthcare professionals develop high-quality sources of information regarding male infertility.

Reading Between the Tabloids: An Objective Analysis of the Quality and Readability of Patient Encountered Online Material for Face Transplantation.

Face transplantation is a highly sensationalized procedure in the media. The purpose of this study is to assess the content and readability of online materials that prospective patients/public encounter regarding face transplantation. A search for face transplantation was performed on Google. Sites were categorized under 3 groups: established face transplant programs, informational third-party sources (eg, Wikipedia), and news article/tabloid sites. Each site was assessed for readability using 6 different readability metrics, while quality was assessed utilizing JAMA benchmark criteria and DISCERN instrument. One-way ANOVA with post hoc Tukey's multiple comparisons test was used for analysis. News sources were significantly easier to read than face transplant program sites (10.4 grade reading level vs. 12.4). For the JAMA benchmark, face transplant programs demonstrated the lowest average score relative to third-party sites, and news sources (2.05 vs. 2.91 vs. 3.67, respectively; P<0.001), but had significantly greater DISCERN scores than news sources (53.50 vs. 45.83, P=0.019). News sources were significantly more accessible, readable, and offered greater transparency of authorship compared with reputable sources, despite their lack of expertise on face transplantation. Face transplant programs should update their websites to ensure readability and accessibility of the information provided to the public.

Online Patient Education in Obstructive Sleep Apnea: ChatGPT versus Google Search.

The widespread implementation of artificial intelligence technologies provides an appealing alternative to traditional search engines for online patient healthcare education. This study assessed ChatGPT-3.5's capabilities as a source of obstructive sleep apnea (OSA) information, using Google Search as a comparison. Ten frequently searched questions related to OSA were entered into Google Search and ChatGPT-3.5. The responses were assessed by two independent researchers using the Global Quality Score (GQS), Patient Education Materials Assessment Tool (PEMAT), DISCERN instrument, CLEAR tool, and readability scores (Flesch Reading Ease and Flesch-Kincaid Grade Level). ChatGPT-3.5 significantly outperformed Google Search in terms of GQS (5.00 vs. 2.50, p < 0.0001), DISCERN reliability (35.00 vs. 29.50, p = 0.001), and quality (11.50 vs. 7.00, p = 0.02). The CLEAR tool scores indicated that ChatGPT-3.5 provided excellent content (25.00 vs. 15.50, p < 0.001). PEMAT scores showed higher understandability (60-91% vs. 44-80%) and actionability for ChatGPT-3.5 (0-40% vs. 0%). Readability analysis revealed that Google Search responses were easier to read (FRE: 56.05 vs. 22.00; FKGL: 9.00 vs. 14.00, p < 0.0001). ChatGPT-3.5 delivers higher quality and more comprehensive OSA information compared to Google Search, although its responses are less readable. This suggests that while ChatGPT-3.5 can be a valuable tool for patient education, efforts to improve readability are necessary to ensure accessibility and utility for all patients. Healthcare providers should be aware of the strengths and weaknesses of various healthcare information resources and emphasize the importance of critically evaluating online health information, advising patients on its reliability and relevance.

Quality assessment of heatstroke videos on TikTok.

The prevalence of heatstroke is rising due to global warming, making it a serious but preventable condition, highlighting the urgent need for effective dissemination of relevant health education to the general public. Advances in technology have made accessing health information more convenient and rapid. In recent years, short videos have become a primary medium for delivering health education, with TikTok gaining considerable popularity among the general public. However, the quality of heatstroke-related health education content on TikTok deserves closer scrutiny. This study aimed to evaluate the quality and content of heatstroke-related videos available on TikTok. The present study analyzed the top 100 heatstroke-related short videos on TikTok, focusing on their characteristics, quality, and the content they conveyed. The quality of these videos was assessed using the DISCERN instrument. In addition, the completeness of the videos was assessed by examining six key aspects: disease definition, clinical manifestations, risk factors, assessment, management, and outcomes. The study included a total of 90 videos. The results showed that news organizations and healthcare professionals were the primary contributors to these videos, with those from news organizations receiving the most attention. In contrast, those from healthcare professionals received comparatively less engagement. Overall, the quality of the information was found to be moderately low, with the highest quality videos posted by non-profit organizations, followed by those posted by healthcare professionals. The majority of videos uploaded described the disease definition, clinical presentation, risk factors, assessment, management, and outcomes of heatstroke. The quality of information provided in heatstroke-related short videos on TikTok is generally inadequate and requires significant improvement. In addition, such content should be subject to government review to ensure its accuracy and reliability.

Quality and Readability Analysis of Turbinoplasty on the Web.

This study aimed to assess the quality, credibility, and readability of online health information concerning turbinoplasty, given the increasing reliance on internet resources for health education. Using four search terms related to turbinoplasty, we analyzed 71 text-based webpages from Google.com, Bing.com, and Yahoo.com. Readability was evaluated using the Flesch-Kincaid grade level, Gunning-Fog Index, SMOG Index, and Coleman-Liau Index were utilized. Web page quality was computed through the DISCERN Instrument (DISCERN), the Journal of the American Medical Association benchmark criteria (JAMA), Novel Turbinoplasty Index (NTI), and presence of code certification by Health on The Net (HON). Seventy-one text-based web pages were assessed. Information quality was measured by an average DISCERN score of 47.4±7.40, indicating "fair" quality. The average readability was a grade level of 9.7±1.57, notably higher than AMA and NIH recommendations. Of all web pages, only 11 (15.49%) proficiently met all 4 listed JAMA criteria. Significant correlations between web page classification and average DISCERN (P=0.0042), as well as JAMA score (P<0.001) were discovered. The web pages that had HON code certification revealed significantly higher quality metrics such as DISCERN scores (P<0.001), JAMA scores (P<0.001), and NTI scores (P=0.038). Online health information for turbinoplasty is of "fair" quality, and the average readability is several grade levels above current AMA and NIH recommendations. Health care providers should aim to guide their patients on finding appropriate educational resources and should improve the readability of their patient education materials.

Assessment of Online Arabic Patient-Centered Knowledge About Orthognathic Surgery: An Infodemiologic Study.

Orthognathic surgery, addressing skeletal and dental irregularities, is pivotal for patients suffering from malocclusion, facial asymmetry, and related disorders. With the internet becoming a primary source of health information, the accuracy, quality, and reliability of online Arabic-language patient-centered information on orthognathic surgery necessitate thorough evaluation. This study aimed to assess the quality, reliability, and readability of Arabic online resources about orthognathic surgery to ascertain their potential as reliable patient education tools. An infodemiological approach was employed, analyzing the top 100 websites from Google, Yahoo, and Bing based on specific Arabic search terms related to orthognathic surgery. Websites were evaluated for affiliation, content specialization, and presentation and excluded based on predetermined criteria to ensure relevance and focus. Quality assessment was conducted using the DISCERN instrument, Journal of the American Medical Association (JAMA) benchmarks, and Health On the Net code evaluation. Readability was assessed using the Flesch-Kincaid Grade Level, Simplified Measure of Gobbledygook, and Flesch Reading Ease scales. Of 600 initial websites, 96 met the inclusion criteria. Most were affiliated with universities or medical centers (53.1%), followed by commercial (28.1%), and non-profit organizations (16.7%). The majority of websites (96.9%) were only partially related to orthognathic surgery. Content analysis revealed a moderate quality of information, with a median overall DISCERN quality rating of 3 out of 5. The JAMA benchmarks showed a lack of comprehensive adherence, with currency being the most achieved criterion. Readability assessments indicated the content was generally accessible yet highlighted the need for Arabic-specific readability evaluation tools. The study reveals a critical need for enhancing Arabic online resources on orthognathic surgery. Despite the available content's moderate quality and acceptable readability, there is a substantial gap in providing comprehensive, patient-centered, and easily understandable information. Future efforts should focus on developing high-quality, reliable, and readable online resources to aid Arabic-speaking patients in making informed decisions about orthognathic surgery.

Quality of Information About Kidney Stones from Artificial Intelligence Chatbots.

Introduction: Kidney stones are common and morbid conditions in the general population with a rising incidence globally. Previous studies show substantial limitations of online sources of information regarding prevention and treatment. The objective of this study was to examine the quality of information on kidney stones from artificial intelligence (AI) chatbots. Methods: The most common online searches about kidney stones from Google Trends and headers from the National Institute of Diabetes and Digestive and Kidney Diseases website were used as inputs to four AI chatbots (ChatGPT version 3.5, Perplexity, Chat Sonic, and Bing AI). Validated instruments were used to assess the quality (DISCERN instrument from 1 low to 5 high), understandability, and actionability (PEMAT, from 0% to 100%) of the chatbot outputs. In addition, we examined the reading level of the information and whether there was misinformation compared with guidelines (5 point Likert scale). Results: AI chatbots generally provided high-quality consumer health information (median DISCERN 4 out of 5) and did not include misinformation (median 1 out of 5). The median understandability was moderate (median 69.6%), and actionability was moderate to poor (median 40%). Responses were presented at an advanced reading level (11th grade; median Flesch-Kincaid score 11.3). Conclusions: AI chatbots provide generally accurate information on kidney stones and lack misinformation; however, it is not easily actionable and is presented above the recommended reading level for consumer health information.

Potential misinformation in websites on carpal tunnel syndrome

ObjectiveWe sought to evaluate the potential reinforcement of misconceptions in websites discussing carpal tunnel syndrome (CTS). MethodsAfter removing all cookies to limit personalization, we entered “carpal tunnel syndrome” into five search engines and collected the first 50 results displayed for each search. For each of the 105 unique websites, we recorded publication date, author background, and number of views. The prevalence of potential reinforcement and/or reorientation of misconceptions for each website was then scored using a rubric based on our interpretation of the best current evidence regarding CTS. The informational quality of websites was graded with the DISCERN instrument, a validated tool for assessing online health information. ResultsEvery website contained at least one potentially misleading statement in our opinion. The most common misconceptions reference “excessive motion” and “inflammation.” Greater potential reinforcement of misinformation about CTS was associated with fewer page views and lower informational quality scores. ConclusionsKeeping in mind that this analysis is based on our interpretation of current best evidence, potential misinformation on websites addressing CTS is common and has the potential to increase symptom intensity and magnitude of incapability via reinforcement of unhelpful thoughts regarding symptoms. InnovationThe prevalence of patient-directed health information that can increase discomfort and incapability by reinforcing common unhelpful thoughts supports the need for innovations in how we develop, oversee, and evolve healthy online material.

Evaluation of the Quality of Online Information About Perinatal Depression in China

This study assessed the quality of Chinese-language websites that provide information about perinatal depression, focusing on their compliance with recognized quality standards, the impact of information content providers on information quality, and the correlation between search engine ranking and quality. We conducted a systematic search on the Baidu search engine and screened 55 relevant websites. The quality of these websites was assessed using the DISCERN instruments. Our findings suggest that the quality of information on websites about perinatal depression is predominantly 'poor', with commercial sites constituting the entirety of the sample, although health information content provided by non-profit organizations demonstrating significantly higher quality. There was a significant negative correlation between search engine rankings and the quality of information on perinatal depression, suggesting that higher quality content is less likely to be searched for and may hinder access to reliable information. This study highlights the urgent need for mechanisms to prioritize the display of high-quality information in search engine results to provide better assistance to Chinese patients with perinatal depression.

Using ChatGPT to Improve Readability of Interventional Radiology Procedure Descriptions.

This project examines ChatGPT's potential to enhance the readability of patient educational materials about interventional radiology (IR) procedures. The descriptions of IR procedures from the Cardiovascular and Interventional Radiological Society of Europe (CIRSE) were used as the original text. Readability scores were calculated using three metrics: Flesch Reading Ease (FRE), Gunning Fog (GF), and the Automated Readability Index (ARI) using an online calculator ( https://readabilityformulas.com ). FRE is scored on a scale of 0-100, where 100 indicates easy-to-read texts, and GF and ARI represent the grade level required to comprehend the text. The DISCERN instrument measured credibility and reliability. ChatGPT was prompted to simplify the texts to a fifth-grade reading level, with subsequent recalculation of readability and DISCERN scores for comparison. Statistical significance was determined using a Wilcoxon Signed-Rank Test. Articles were subsequently organized by subgroups and analyzed. 73 interventional radiology procedures from CIRSE were analyzed. The original FRE score was 47.2 (Difficult), improved to 78.4 (Fairly Easy) by ChatGPT. GF and ARI scores dropped from 14.4 and 11.2 to 7.8 and 5.8, respectively, after simplification, showing significant improvement (p < 0.001). However, the average DISCERN score decreased from 3.73 to 2.99 (p < 0.001) post-ChatGPT simplification. This study shows ChatGPT's ability to make interventional radiology descriptions more readable but highlights its struggle to maintain the original's reliability, suggesting the need for human review and prompt engineering to enhance outcomes. Level 6.

O-161 #Infertility: a systematic review of information available on social media platforms

Abstract Study question How accurate, credible, reliable and readable is online information obtained from social media content in relation to the diagnosis and management of fertility problems? Summary answer Information available on social media regarding infertility often uses language that is inappropriate for a lay audience and lacks sufficient accuracy, credibility and reliability. What is known already Social media has become a popular source of information for people seeking guidance, advice and educational resources related to their own fertility journey. Social media can provide free and easily accessible information and can act as a space for community support to share real-world experiences. Many of the platforms delivering this information have not been verified and there is a growing concern regarding the accuracy of the content on these posts. A variation in expertise among content creators and the rapid spread of anecdotal experiences may contribute to the unreliability of information. Study design, size, duration Using real-time analytics software, the #infertility was tracked prospectively during National Infertility Awareness Week, an annual campaign organised by the National Infertility Association (USA) to raise awareness around infertility, on Twitter and Instagram platforms. During this one-week period, a total of 28,612 posts containing the #infertility was posted on Twitter and Instagram. Posts containing non-English language, commercial advertising and content related solely to the lived experience of infertility, for example, a patient’s blog, were excluded. Participants/materials, setting, methods Once screening was completed, 673 posts (310 on Twitter and 363 on Instagram) were identified as containing information pertaining to the diagnosis and management of fertility problems. Two independent researchers screened the included posts. Posts were evaluated using validated instruments across four domains, including assessments of 1) credibility (White Paper instrument) 2) quality (adapted DISCERN instrument) 3) accuracy (assessed by prioritized criteria developed in consultation with healthcare professionals and researchers) and 4) Readability (Flesch-Kincaid instrument). Main results and the role of chance A total of 28,612 posts, with a reach of 285,826,087 people were identified. Once screening had been completed, there were 310 posts on Twitter containing information pertaining to infertility, with 20,136 engagements and an overall reach of 75,184,476. For Instagram, 363 posts were identified as containing information pertaining to infertility, with 46,443 engagements. The most discussed topics included: 1) lifestyle interventions including nutrition and supplements, 2) polycystic ovarian syndrome, 3) endometriosis and 4) infertility in relation to COVID and the COVID vaccination programme. Only 87 Twitter posts (28%) had clear demonstration of credibility and of those posts the information contained within them was accurate 92% of the time. Of the remaining 72% of Twitter posts, no clear credentials were displayed, and the accuracy of those posts dropped to 34%. 139 Twitter posts (45%) appeared to have an undeclared Trial registration number Not applicable

Readability and Information Quality in Cancer Information From a Free vs Paid Chatbot

The mainstream use of chatbots requires a thorough investigation of their readability and quality of information. To identify readability and quality differences in information between a free and paywalled chatbot cancer-related responses, and to explore if more precise prompting can mitigate any observed differences. This cross-sectional study compared readability and information quality of a chatbot's free vs paywalled responses with Google Trends' top 5 search queries associated with breast, lung, prostate, colorectal, and skin cancers from January 1, 2021, to January 1, 2023. Data were extracted from the search tracker, and responses were produced by free and paywalled ChatGPT. Data were analyzed from December 20, 2023, to January 15, 2024. Free vs paywalled chatbot outputs with and without prompt: "Explain the following at a sixth grade reading level: [nonprompted input]." The primary outcome measured the readability of a chatbot's responses using Flesch Reading Ease scores (0 [graduate reading level] to 100 [easy fifth grade reading level]). Secondary outcomes included assessing consumer health information quality with the validated DISCERN instrument (overall score from 1 [low quality] to 5 [high quality]) for each response. Scores were compared between the 2 chatbot models with and without prompting. This study evaluated 100 chatbot responses. Nonprompted free chatbot responses had lower readability (median [IQR] Flesh Reading ease scores, 52.60 [44.54-61.46]) than nonprompted paywalled chatbot responses (62.48 [54.83-68.40]) (P < .05). However, prompting the free chatbot to reword responses at a sixth grade reading level was associated with increased reading ease scores than the paywalled chatbot nonprompted responses (median [IQR], 71.55 [68.20-78.99]) (P < .001). Prompting was associated with increases in reading ease in both free (median [IQR], 71.55 [68.20-78.99]; P < .001)and paywalled versions (median [IQR], 75.64 [70.53-81.12]; P < .001). There was no significant difference in overall DISCERN scores between the chatbot models, with and without prompting. In this cross-sectional study, paying for the chatbot was found to provide easier-to-read responses, but prompting the free version of the chatbot was associated with increased response readability without changing information quality. Educating the public on how to prompt chatbots may help promote equitable access to health information.

Exploring the Role of ChatGPT-4, BingAI, and Gemini as Virtual Consultants to Educate Families about Retinopathy of Prematurity.

Large language models (LLMs) are becoming increasingly important as they are being used more frequently for providing medical information. Our aim is to evaluate the effectiveness of electronic artificial intelligence (AI) large language models (LLMs), such as ChatGPT-4, BingAI, and Gemini in responding to patient inquiries about retinopathy of prematurity (ROP). The answers of LLMs for fifty real-life patient inquiries were assessed using a 5-point Likert scale by three ophthalmologists. The models' responses were also evaluated for reliability with the DISCERN instrument and the EQIP framework, and for readability using the Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), and Coleman-Liau Index. ChatGPT-4 outperformed BingAI and Gemini, scoring the highest with 5 points in 90% (45 out of 50) and achieving ratings of "agreed" or "strongly agreed" in 98% (49 out of 50) of responses. It led in accuracy and reliability with DISCERN and EQIP scores of 63 and 72.2, respectively. BingAI followed with scores of 53 and 61.1, while Gemini was noted for the best readability (FRE score of 39.1) but lower reliability scores. Statistically significant performance differences were observed particularly in the screening, diagnosis, and treatment categories. ChatGPT-4 excelled in providing detailed and reliable responses to ROP-related queries, although its texts were more complex. All models delivered generally accurate information as per DISCERN and EQIP assessments.

(040) EVALUATING CHATGPT AS A SOURCE OF PATIENT EDUCATION MATERIALS ON DYSPAREUNIA

Abstract Introduction Dyspareunia, or female sexual pain, has an estimated prevalence of up to 18% worldwide. It is a complex condition with various physical and psychological etiologies, making it a challenge for patients to understand. Physicians spend significant time on patient education and providing educational materials to patients. ChatGPT is an artificial intelligence (AI) language model that generates human-like conversational text. Objective The aim of our study is to evaluate the understandability, quality, and readability of written patient educational material on topics related to dyspareunia generated by ChatGPT in English. Methods Five existing and accessible patient education handouts related to the topic of dyspareunia were identified and used as reference material. We prompted ChatGPT to create patient education leaflets on dyspareunia, female sexual dysfunction, vaginismus, and vulvodynia. The DISCERN instrument and Patient Education Materials Assessment (PEMAT) were used to assess each leaflet’s quality, understandability, and actionability. Readability of the patient education materials was assessed with the Flesch-Kincaid Grade (FKG) level. Results ChatGPT created four different patient education materials. The document created on “dyspareunia” scored 90% on understandability 75% on actionability, 42 on DISCERN, and 14.4 FKG level compared to the UpToDate (88%, 50%, 50, and 7.6) and the International Urogynecologic Association (IUGA) (100%, 0%, 80, and 13.6) on the same subject. The ChatGPT handout on “female sexual dysfunction” scored 70% on understandability, 50% on actionability, 37 on DISCERN, and 12.9 FKG Level compared to the American Urogynecologic Society (AUGS) (90%, 50%, 52, and 13) handout on the same subject. The ChatGPT handout on “vaginismus” scored 81% on understandability, 25% on actionability, 41 on DISCERN, and 11.3 FKG Level compared to the International Society for the Study of Vulvovaginal Disease (ISSVD) (90%, 0%, 44, 11.4) handout on the same subject. The ChatGPT handout on “vulvodynia” scored 81% on understandability, 25% on actionability, 36 on DISCERN, and 11.5 FKG Level compared to the ISSVD (90%, 25%, 46, 10.5) handout on the same subject. Conclusions The average PEMAT understandability scores suggest that the reference handouts (91.6%) are more understandable than the AI-generated materials (80.5%). The ChatGPT-generated handouts also fell short in quality according to the average DISCERN scores (39 vs. 54.4). The average FKG level for the AI-generated information was 12.525 compared to 11.22 for the reference patient education materials. Overall, ChatGPT was not able to create patient education handouts on topics related to dyspareunia that were as understandable or readable as the existing handouts. The ChatGPT-generated handouts did, however, score higher in actionability on average (50) than the reference handouts (25), which may give patients a false sense of trust in these AI-generated materials. Our study indicates that ChatGPT is not yet an ideal tool for the creation of patient education materials, nor is it a reliable information source for patients. Disclosure No.

Exploring the online landscape of interventional radiology: a global analysis of search trends

This study aimed to analyze the online presence of interventional radiology (IR), and its popularity over time and location, given the reported under-awareness of the specialty. The study analyzed search volumes, searcher demographics, and query type related to IR and its four most searched procedures using Google Trends, Demographics.io, and Also Asked. Search strategies were stratified by volume and region ("Worldwide" and "United States"), and the quality of current patient materials in the first 10 Google search results was analyzed using the DISCERN instrument and Flesch Kincaid levels. The analyzed search trends demonstrated a slow uptrend in search volume over the past 15 years since 2013, with a CAGR rate of 0.6%. Demographics revealed that 80.9% of searchers were female and over half (51.8%) fell into the age range of 35-54 years old. Geographically, the US had the highest search volume (100) for the term "interventional radiology" and website search results mainly related to patient education about the specialty. The quality of online resources was poor with overall college-level readability, and "What is Interventional Radiology?" was the most popular query. There is a growing interest in IR procedures in recent years, particularly in the US, with middle-aged females being the most active demographic online. However, online resources containing information on specific IR procedures remain of poor quality. Actions should be taken to improve the quality, accessibility, and awareness of IR-related webpages to increase public knowledge of IR care in the US and abroad.

A Quality and Completeness Assessment of Testicular Cancer Health Information on TikTok

TikTok has become a hub for easily accessible medical information. However, the quality and completeness of this information for testicular cancer has not been examined. Our study aims to assess the quality and completeness of testicular cancer information on TikTok. A search was performed on TikTok using the search terms “Testicular Cancer” and “Testicle Cancer”. Inclusion criteria encompassed videos about testicular cancer in English. We excluded non-English videos, irrelevant videos, and videos without audio. We evaluated these videos using the DISCERN instrument and a completeness assessment. A total of 361 videos were considered for screening and 116 videos were included. Of these, 57 were created by healthcare professionals (HCPs). The median video length was 40 s (5–277 s), with &gt;25 million cumulative views and a median of 446,400 views per video. The average DISCERN score was 29.0 ± 5.7, with HCPs providing higher-quality videos than non-HCPs (30.8 vs. 5.5, p &lt; 0.05). HCPs also had more reliable videos (21.2 vs. 18.1, p &lt; 0.05). Overall quality levels were mostly poor or very poor (97.4%), with none being good or excellent. Most HCP videos were poor (63.2%), whilst many non-HCP videos were very poor (61.0%). The most viewed video had 2,800,000 views but scored a 31 on the DISCERN tool and one on the completeness assessment. The highest DISCERN score had 11,700 views. HCP videos better defined the disease and were more complete (p &lt; 0.05). Most videos discussed self-assessment but were lacking in definitions, risk factors, symptoms, evaluation, management, and outcomes. Most of TikTok’s testicular cancer information lacks quality and completeness, whilst higher-quality videos have limited reach.