Ensuring Quality Information Research Articles

Assessing the readability, quality and reliability of responses produced by ChatGPT, Gemini, and Perplexity regarding most frequently asked keywords about low back pain.

Patients who are informed about the causes, pathophysiology, treatment and prevention of a disease are better able to participate in treatment procedures in the event of illness. Artificial intelligence (AI), which has gained popularity in recent years, is defined as the study of algorithms that provide machines with the ability to reason and perform cognitive functions, including object and word recognition, problem solving and decision making. This study aimed to examine the readability, reliability and quality of responses to frequently asked keywords about low back pain (LBP) given by three different AI-based chatbots (ChatGPT, Perplexity and Gemini), which are popular applications in online information presentation today. All three AI chatbots were asked the 25 most frequently used keywords related to LBP determined with the help of Google Trend. In order to prevent possible bias that could be created by the sequential processing of keywords in the answers given by the chatbots, the study was designed by providing input from different users (EO, VH) for each keyword. The readability of the responses given was determined with the Simple Measure of Gobbledygook (SMOG), Flesch Reading Ease Score (FRES) and Gunning Fog (GFG) readability scores. Quality was assessed using the Global Quality Score (GQS) and the Ensuring Quality Information for Patients (EQIP) score. Reliability was assessed by determining with DISCERN and Journal of American Medical Association (JAMA) scales. The first three keywords detected as a result of Google Trend search were "Lower Back Pain", "ICD 10 Low Back Pain", and "Low Back Pain Symptoms". It was determined that the readability of the responses given by all AI chatbots was higher than the recommended 6th grade readability level (p < 0.001). In the EQIP, JAMA, modified DISCERN and GQS score evaluation, Perplexity was found to have significantly higher scores than other chatbots (p < 0.001). It has been determined that the answers given by AI chatbots to keywords about LBP are difficult to read and have low reliability and quality assessment. It is clear that when new chatbots are introduced, they can provide better guidance to patients with increased clarity and text quality. This study can provide inspiration for future studies on improving the algorithms and responses of AI chatbots.

Evaluation of the quality, reliability, readability of websites on temporomandibular joint pain

ABSTRACT OBJECTIVE: This study aimed to evaluate the reliability, quality, and readability of websites providing information on temporomandibular joint pain. MATERIALS AND METHOD: An internet search was performed on Google Chrome (Google, Mountain View, California, USA) on June 9, 2023, with the keywords “jaw pain” and the links of the first 50 websites were saved. Four different evaluation tools the DISCERN instrument, The Ensuring Quality Information for Patients (EQIP), the Journal of the American Medical Association (JAMA) benchmark, Readability Index were used to evaluate 37 sites that met the inclusion criteria. SPSS 25 statistical software was used for the statistical analyses and p&amp;lt;0.05 was considered statistically significant. RESULTS: Within the scope of the study, the first 50 websites were registered and a total of 37 websites met the inclusion criteria and were evaluated. The mean values for DISCERN, EQIP, and the JAMA index were 2.05±0.911, 62.4±11.6 and 0.486±0.692, respectively. In addition, those with high credibility constituted %5.4 (n=2) of the total number of sites and %50 (n=1) of these sites were uploaded by private hospitals, and %50 (n=1) by blogs (web logs). CONCLUSION: Based on the findings of this study, it can be concluded that the quality, reliability and readability of websites providing information about jaw pain on the internet vary but are at a medium-low level. Professional associations and organizations, such as the Turkish Orthodontic Society (TOD), may conduct similar research and publish the results to ensure the quality and reliability of the information in publicly available online resources.

Assessment of the quality, reliability, and readability of the websites about orthognathic surgery

Objectives: The purpose of this study is to evaluate the accuracy, dependability, and readability of websites providing information about orthognathic surgery. Material and Methods: A total of 300 websites encountered through Google, Yahoo, and Bing search engines using the search terms “orthognathic surgery” and “jaw surgery” were utilized. Websites containing duplicate advertisements, original articles, or course content with video links unrelated to the topic were excluded from the study. Ninety-three websites were included in the study. The included websites were assessed for reliability and quality using the DISCERN, Ensuring Quality Information for Patients (EQIP), and Journal of American Medical Association (JAMA) tools. Readability was evaluated by determining the Flesch Reading Ease Score (FRES) and Flesch–Kincaid Grade Level (FKGL) scores. The data that did not show normal distribution were analyzed using the Kruskal–Wallis test. Post hoc comparisons were conducted using the least significant difference and Mann– Whitney U-tests. A significance level of P &lt; 0.05 was accepted. Results: The mean DISCERN score was (38.47 ± 10.47), the EQIP score was (57.34 ± 10.73), and the JAMA score was (0.81 ± 0.99). The FRES was determined to be (43.89 ± 11.36), and the FKGL score was (12.06 ± 2.3). In the layperson group, the highest scores were found in the DISCERN (44.62 ± 9.73) and JAMA (1.85 ± 0.8) analyses compared to other groups. Conclusion: Website content on orthognathic surgery is generally of poor quality, and moderate reliability, with readability often being challenging for the average reader. There is a need for more user-friendly websites that offer comprehensive and easily digestible information to better serve orthognathic surgery patients.

Ensuring quality information for patients tool to assess patient information on CABG websites: Systemic search and evaluation.

Coronary artery bypass surgery (CABG) is the most common cardiac surgical procedure worldwide. The Internet is often the first place for patients to obtain medical information and may influence their health-seeking behaviour. This study aimed to assess the quality of online CABG information for patients. A review was conducted on the available CABG information online using four search terms in Google: "coronary artery bypass grafting", "CABG", "bypass surgery", and "coronary surgery". Websites on the first three pages of each search term were assessed by two independent assessors using the Ensuring Quality Information for Patients (EQIP) tool (score 0-36). 71 websites in total were eligible for the evaluation. Most websites originated from the USA and the UK (35% for both). The median EQIP score was 16.5. Most websites used everyday language (94%) in short sentences (83%). More than half (60%) of the websites addressed qualitative complications and risks of surgery. Only 1/3 of the websites described treatment alternatives, precautions before surgery, and warning signs after surgery. Only about 1/10 of websites could address quality-of-life issues and resolution of complications. Most online sites concerning CABG scores were low in EQIP tool assessment in terms of quality and readability with insufficient sources. Minimal information is provided regarding the quantitative risks and complications. Enhancement of the credibility and informativeness of websites is needed to reduce patients' anxiety about surgical procedures and help with treatment options for coronary artery disease. Education and investigation on the consistency of high-quality websites may help patient identify high quality medical information.

BN SO41 - A systematic search and evaluation of patient information quality on Anti-reflux surgery using modified EQIP Score

Abstract Background GORD affects 20% of western populations and internet searches are a key resource for patients considering surgery. An assessment of the quality of public intended electronic information can highlight a need for improved material to aide patient education. This study assesses the quality of internet information on anti-reflux surgery using the validated Modified Ensuring Quality Information for Patients (EQIP) tool. Method A systematic analysis of the top 200 results for the search engine query “Anti-reflux surgery” was conducted. Google (UK) was used. Search results aimed solely for healthcare professionals were excluded. Results were evaluated using the modified (EQIP) tool (score 0-36) and evaluation performed on information on surgery benefits, risks and complications. Results 107 of 200 websites were included in the main study. The majority (43%) originated from UK, while 40% were from USA. Median EQIP score for all websites was 16. Median EQIP scores for the 3 domains were 7/18, 1/6 and 8/12 for content, identification data and structure respectively. Benefits of surgery was described on 71%, surgery risks on 66%, and complications on 11%. However, only 16% of websites described the quantitative risks of surgery. Information on postoperative recovery was included on 63%, emergency advice on 22%, and mortality rates on 7% of websites. Conclusion We observed most public intended internet searches lacked information on the quantitative risks and complications of anti-reflux surgery. Moreover, the information provided was insufficient on morbidity and mortality to adequately assist patients make informed decisions. This study supports the need for improved quality of patient intended publicly available information accessible online on Anti-reflux surgery.

How Useful are Current Chatbots Regarding Urology Patient Information? Comparison of the Ten Most Popular Chatbots' Responses About Female Urinary Incontinence.

This research evaluates the readability and quality of patient information material about female urinary incontinence (fUI) in ten popular artificial intelligence (AI) supported chatbots. We used the most recent versions of 10 widely-used chatbots, including OpenAI's GPT-4, Claude-3 Sonnet, Grok 1.5, Mistral Large 2, Google Palm 2, Meta's Llama 3, HuggingChat v0.8.4, Microsoft's Copilot, Gemini Advanced, and Perplexity. Prompts were created to generate texts about UI, stress type UI, urge type UI, and mix type UI. The modified Ensuring Quality Information for Patients (EQIP) technique and QUEST (Quality Evaluating Scoring Tool) were used to assess the quality, and the average of 8 well-known readability formulas, which is Average Reading Level Consensus (ARLC), were used to evaluate readability. When comparing the average scores, there were significant differences in the mean mQEIP and QUEST scores across ten chatbots (p = 0.049 and p = 0.018). Gemini received the greatest mean scores for mEQIP and QUEST, whereas Grok had the lowest values. The chatbots exhibited significant differences in mean ARLC, word count, and sentence count (p = 0.047, p = 0.001, and p = 0.001, respectively). For readability, Grok is the easiest to read, while Mistral is highly complex to understand. AI-supported chatbot technology needs to be improved in terms of readability and quality of patient information regarding female UI.

ChatGPT as a Source for Patient Information on Patellofemoral Surgery-A Comparative Study Amongst Laymen, Doctors, and Experts.

In November 2022, OpenAI launched ChatGPT for public use through a free online platform. ChatGPT is an artificial intelligence (AI) chatbot trained on a broad dataset encompassing a wide range of topics, including medical literature. The usability in the medical field and the quality of AI-generated responses are widely discussed and are the subject of current investigations. Patellofemoral pain is one of the most common conditions among young adults, often prompting patients to seek advice. This study examines the quality of ChatGPT as a source of information regarding patellofemoral conditions and surgery, hypothesizing that there will be differences in the evaluation of responses generated by ChatGPT between populations with different levels of expertise in patellofemoral disorders. A comparison was conducted between laymen, doctors (non-orthopedic), and experts in patellofemoral disorders based on a list of 12 questions. These questions were divided into descriptive and recommendatory categories, with each category further split into basic and advanced content. Questions were used to prompt ChatGPT in April 2024 using the ChatGPT 4.0 engine, and answers were evaluated using a custom tool inspired by the Ensuring Quality Information for Patients (EQIP) instrument. Evaluations were performed independently by laymen, non-orthopedic doctors, and experts, with the results statistically analyzed using a Mann-Whitney U Test. A p-value of less than 0.05 was considered statistically significant. The study included data from seventeen participants: four experts in patellofemoral disorders, seven non-orthopedic doctors, and six laymen. Experts rated the answers lower on average compared to non-experts. Significant differences were observed in the ratings of descriptive answers with increasing complexity. The average score for experts was 29.3 ± 5.8, whereas non-experts averaged 35.3 ± 5.7. For recommendatory answers, experts also gave lower ratings, particularly for more complex questions. ChatGPT provides good quality answers to questions concerning patellofemoral disorders, although questions with higher complexity were rated lower by patellofemoral experts compared to non-experts. This study emphasizes the potential of ChatGPT as a complementary tool for patient information on patellofemoral disorders, although the quality of the answers fluctuates with the complexity of the questions, which might not be recognized by non-experts. The lack of personalized recommendations and the problem of "AI hallucinations" remain a challenge. Human expertise and judgement, especially from trained healthcare experts, remain irreplaceable.

Evaluating the reliability of the responses of large language models to keratoconus-related questions

ABSTRACT Clinical relevance Artificial intelligence has undergone a rapid evolution and large language models (LLMs) have become promising tools for healthcare, with the ability of providing human-like responses to questions. The capabilities of these tools in addressing questions related to keratoconus (KCN) have not been previously explored. Background In this study, the responses were evaluated from three LLMs – ChatGPT-4, Copilot, and Gemini – to common patient questions regarding KCN. Methods Fifty real-life patient inquiries regarding general information, aetiology, symptoms and diagnosis, progression, and treatment of KCN were presented to the LLMs. Evaluations of the answers were conducted by three ophthalmologists with a 5-point Likert scale ranging from ‘strongly disagreed’ to ‘strongly agreed’. The reliability of the responses provided by LLMs was evaluated using the DISCERN and the Ensuring Quality Information for Patients (EQIP) scales. Readability metrics (Flesch Reading Ease Score, Flesch-Kincaid Grade Level, and Coleman-Liau Index) were calculated to evaluate the complexity of responses. Results ChatGPT-4 consistently scored 3 points or higher for all (100%) its responses, while Copilot had five (10%) and Gemini had two (4%) responses scoring 2 points or below. ChatGPT-4 achieved a ‘strongly agree’ rate of 74% across all questions, markedly superior to Copilot at 34% and Gemini at 42% (p < 0.001); and recorded the highest ‘strongly agree’ rates in general information and symptoms & diagnosis categories (90% for both). The median Likert scores differed among LLMs (p < 0.001), with ChatGPT-4 scoring highest and Copilot scoring lowest. Although ChatGPT-4 exhibited more reliability based on the DISCERN scale, it was characterised by lower readability and higher complexity. While all LLMs provided responses categorised as ‘extremely difficult to read’, the responses provided by Copilot showed higher readability. Conclusions Despite the responses provided by ChatGPT-4 exhibiting lower readability and greater complexity, it emerged as the most proficient in answering KCN-related questions.

Assessing the quality and readability of online patient information: ENT UK patient information e-leaflets vs responses by a Generative Artificial Intelligence.

The evolution of artificial intelligence has introduced new ways to disseminate health information, including natural language processing models like ChatGPT. However, the quality and readability of such digitally-generated information remains understudied. This study is the first to compare the quality and readability of digitally-generated health information against leaflets produced by professionals. Patient information leaflets for five ENT UK leaflets and their corresponding ChatGPT responses were extracted from the Internet. Assessors with various degree of medical knowledge evaluated the content using the Ensuring Quality Information for Patients (EQIP) tool and readability tools including the Flesch-Kincaid Grade Level (FKGL). Statistical analysis was performed to identify differences between leaflets, assessors, and sources of information. ENT UK leaflets were of moderate quality, scoring a median EQIP of 23. Statistically significant differences in overall EQIP score were identified between ENT UK leaflets but ChatGPT responses were of uniform quality. Non-specialist doctors rated the highest EQIP scores while medical students scored the lowest. The mean readability of ENT UK leaflets was higher than ChatGPT responses. The information metrics of ENT UK leaflets were moderate and varied between topics. Equivalent ChatGPT information provided comparable content quality, but with reduced readability. ChatGPT patient information and professionally-produced leaflets had comparable content, but LLM content were required a higher reading age. With the increasing use of online health resources, this study highlights the need for a balanced approach that considers optimises both the quality and readability of patient education materials.

Comparative Analysis of AI Chatbots ChatGPT, Gemini, and Copilot’s Answers to Common Cataract Questions

Purpose: To compare the readability and quality of the answers given by artificial intelligence (AI) chatbots to the twenty-five most frequent searches about cataracts on Google. Study Design: Cross sectional comparative study. Place and Duration of study: Serdivan State Hospital's Ophthalmology Department from March 2024 to April 2024. Methods: The word 'Cataract' was entered into Google Trends, and 25 trending searches made worldwide since 2004 were identified. These 25 trending searches were entered separately into AI programs. The answers were examined for quality using the Ensuring Quality Information for Patients (EQIP) test, while the readability was evaluated using the Flesch-Kincaid Reading Ease (FKRE) and Flesch-Kincaid Grade Level (FKGL). Results: The keywords searched most frequently were ‘Cataract surgery,’‘Eye cataract and 'After Cataract’. The EQIP category of all three AI chatbots was found to be 'serious quality issues.' When EQIP scores were compared, ChatGPT had a lower median value than Gemini and Copilot (p:0.001, p:0.007, respectively), while there was no significant difference between Gemini and Copilot (p:0.098). When FKRE values were compared, ChatGPT had a lower median value than Gemini and Copilot (p:0.001, p:0.001, respectively), with no significant difference between Gemini and Copilot (p:0.557). When FKGL values were compared, ChatGPT had a higher median value than Gemini and Copilot (p:0.003, p:0.001, respectively), with no significant difference between Gemini and Copilot(p:0.245). Conclusion: All three AI chatbots had an EQIP category of 'serious issues with quality .'The readability of all three was not at the recommended level.

FP7.19 - PIXEL: Patient Information and eXplanation following Emergency Laparotomy

Abstract Aims 80% of patients that survive an emergency laparotomy (EmLap) report that they would have preferred more information on “what to expect” during their recovery; information is often delivered on a variable adhoc inpatient basis. This work aims to assess the robustness of a multi-disciplinary prepared patient information leaflet (PIL), by use of a patient questionnaire, and the Ensuring Quality Information for Patients (EQIP) toolkit. Methods A 21-page PIL was prepared by the EmLap MDT at our institute. The PIL’s objective was to outline common issues encountered during recovery, factors that could aid recovery, and signpost relevant resources. POLO enrolled patients (ClinicalTrials.Gov:NCT05281627) were asked their opinion by completion of an 8-point questionnaire at their 12-month follow-up. The PIL was also scored using the EQIP tool by an independent healthcare professional. Results 30 patients took part. 66.6% were female; average age was 63 years. 100% patients felt the PIL was easy to understand; 96.6% felt it covered all appropriate content. When asked about the length of the PIL: 93.3% felt the length of the leaflet was just right; 6.7% felt that the leaflet was too long. The PIL was found to be 93.3% compliant with the EQIP tool. Conclusions This is the first study to demonstrate the robustness of a PIL concerning EmLap recovery, suggesting it can be applied to a national audience. It is accepted that the leaflet is long; future iterations may benefit from being in a different format e.g. visuo-audio.

Evaluation of the Quality and Readability of Web-Based Information Regarding Foreign Bodies of the Ear, Nose, and Throat: Qualitative Content Analysis.

Foreign body (FB) inhalation, ingestion, and insertion account for 11% of emergency admissions for ear, nose, and throat conditions. Children are disproportionately affected, and urgent intervention may be needed to maintain airway patency and prevent blood vessel occlusion. High-quality, readable online information could help reduce poor outcomes from FBs. We aim to evaluate the quality and readability of available online health information relating to FBs. In total, 6 search phrases were queried using the Google search engine. For each search term, the first 30 results were captured. Websites in the English language and displaying health information were included. The provider and country of origin were recorded. The modified 36-item Ensuring Quality Information for Patients tool was used to assess information quality. Readability was assessed using a combination of tools: Flesch Reading Ease score, Flesch-Kincaid Grade Level, Gunning-Fog Index, and Simple Measure of Gobbledygook. After the removal of duplicates, 73 websites were assessed, with the majority originating from the United States (n=46, 63%). Overall, the quality of the content was of moderate quality, with a median Ensuring Quality Information for Patients score of 21 (IQR 18-25, maximum 29) out of a maximum possible score of 36. Precautionary measures were not mentioned on 41% (n=30) of websites and 30% (n=22) did not identify disk batteries as a risky FB. Red flags necessitating urgent care were identified on 95% (n=69) of websites, with 89% (n=65) advising patients to seek medical attention and 38% (n=28) advising on safe FB removal. Readability scores (Flesch Reading Ease score=12.4, Flesch-Kincaid Grade Level=6.2, Gunning-Fog Index=6.5, and Simple Measure of Gobbledygook=5.9 years) showed most websites (56%) were below the recommended sixth-grade level. The current quality and readability of information regarding FBs is inadequate. More than half of the websites were above the recommended sixth-grade reading level, and important information regarding high-risk FBs such as disk batteries and magnets was frequently excluded. Strategies should be developed to improve access to high-quality information that informs patients and parents about risks and when to seek medical help. Strategies to promote high-quality websites in search results also have the potential to improve outcomes.

Evaluating the quality and reliability of YouTube videos on myopia: a video content analysis.

To evaluate the quality and reliability of YouTube videos as an educational resource about myopia. The videos were identified by searching YouTube with the keywords 'myopia' and 'nearsightedness', using the website's default search settings. The number of views, likes, dislikes, view ratio, source of the upload, country of origin, video type, and described treatment techniques were assessed. Each video was evaluated using the DISCERN, Journal of the American Medical Association (JAMA), Ensuring Quality Information for Patients (EQIP), Health On the Net Code of Conduct Certification (HONcode), and the Global Quality Score (GQS) scales. A total of 112 videos were included. The classification of videos by source indicated that the top three contributors were health channels (30 videos [26.8%]), physicians (24 videos [21.4%]), and academic centers (19 videos [16.9%]). Most of these videos originated from the United States (74 videos [66.1%]) and focused on the pathophysiology (n = 89, 79.4%) and the treatment (n = 77, 68.7%) of myopia. Statistical comparisons among the groups of video sources showed no significant difference in the mean DISCERN score (p = 0.102). However, significant differences were noted in the JAMA (p = 0.011), GQS (p = 0.009), HONcode (p = 0.011), and EQIP (p = 0.002) scores. This study underscored the variability in the quality and reliability of YouTube videos related to myopia, with most content ranging from 'weak to moderate' quality based on the DISCERN and GQS scales, yet appearing to be 'excellent' according to the HONcode and EQIP scales. Videos uploaded by physicians generally exhibited higher standards, highlighting the importance of expert involvement in online health information dissemination. Given the potential risks of accessing incorrect medical data that can affect the decision-making processes of patients, caution should be exercised when using online content as a source of information.

Evaluating the Internet as a Source of Information for Patients With Hidradenitis Suppurativa: A Review of Websites' Content, Insights Into Engagement, and Access.

The Internet has become the primary information source for patients, with most turning to online resources before seeking medical advice. The aim of this study is to evaluate the quality of online information on hidradenitis suppurativa available to patients. The authors performed an Internet search using the search terms "hidradenitis suppurativa," "hidradenitis suppurativa treatment," "hidradenitis suppurativa surgery," and "acne inversa." They identified the initial 100 websites from Google, Yahoo, and Bing. Websites were evaluated based on the modified Ensuring Quality Information for Patients instrument. Of the 300 websites, 95 (31.7%) were incorporated after accounting for the exclusion criteria: duplicate entries, websites not pertinent to the subject matter, websites inaccessible due to location restrictions or necessitating user accounts for access, websites in languages other than English, and websites originating from scientific publications directed at a scientific audience rather than the general population. Ensuring Quality Information for Patients scores ranged from 5 to 30/36, with a median of 17. This analysis unveils a diverse array of websites that could confound patients navigating toward high-caliber resources. These barriers may hinder the access to top-tier online patient information and magnify disparities in referral rates, patient engagement, treatment satisfaction, and quality of life.

20 An Evaluation of the Quality and Readability of Online Information Regarding TMJ Dysfunction

Abstract Aim The accessibility and availability of the internet has allowed people to increasingly seek health information online, and therefore it plays an essential role in the decision-making process of the patient. The study aimed to assess the quality and readability of online health information on TMJ dysfunction. Method The search was conducted on Google using keywords: jaw pain, jaw locking, jaw discomfort, jaw clicking, jaw popping, temporomandibular joint, TMJ dysfunction, TMD and temporomandibular disorder. Ensuring Quality Information for Patient (EQIP) tool was used to assess quality of information and the application of readability score reflected through Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level(FKGL), Gunning-Fog Index(GFI) and Simple Measure of Gobbledygook(SMOG). Results 134 websites were eligible for assessment. The overall median EQIP score was 23, with an IQR of 7 and maximum score of 30. The most common source of information was healthcare industry (n=60; 45%), followed by news services (n=19; 14%), with a median EQIP score of 20 and 25 respectively. Statistically significant differences in overall EQIP scores were found between different sources of information (p=0.004). Only 31(23%) websites achieved the recommended 6th grade or below readability level, and only 14 (10%) discussed complication rates with none mentioning mortality rates at all. Conclusions Most online information on TMJ dysfunction is of suboptimal quality and readability. This study identified areas for improvements when producing patient material regarding TMJ dysfunction. To improve patient education and awareness, content creators should strive to be informative, accurate, and patient-centric.

Evaluating the web as a source of information for patients with chest wall deformities: insights into engagement and disparities.

Recent data highlight the internet's pivotal role as the primary information source for patients. In this study, we emulate a patient's/caregiver's quest for online information concerning chest deformities and assess the quality of available information. We conducted an internet search using combination of the terms "pectus excavatum," "pectus excavatum surgery," "funnel chest," "pectus excavatum repair" and identified the first 100 relevant websites from the three most popular search engines: Google, Yahoo, and Bing. These websites were evaluated using the modified Ensuring Quality Information for Patients (EQIP) instrument. Of the 300 websites generated, 140 (46.7%) were included in our evaluation after elimination of duplicates, non-English websites, and those targeting medical professionals. The EQIP scores in the final sample ranged from 8 to 32/36, with a median score of 22. Most of the evaluated websites (32.8%) originated from hospitals, yet none met all 36 EQIP criteria. None of the evaluated websites pertaining to pectus excavatum achieved a flawless "content quality" score. The diverse array of websites potentially complicates patients' efforts to navigate toward high-quality resources. Barriers in accessing high-quality online patient information may contribute to disparities in referral, patient engagement, treatment satisfaction, and overall quality of life. IV.

Evaluation of information accuracy and clarity: ChatGPT responses to the most frequently asked questions about premature ejaculation.

Premature ejaculation (PE) is the most prevalent sexual dysfunction in men, and like many diseases and conditions, patients use Internet sources like ChatGPT, which is a popular artificial intelligence-based language model, for queries about this andrological disorder. The objective of this research was to evaluate the quality, readability, and understanding of texts produced by ChatGPT in response to frequently requested inquiries on PE. In this study we used Google Trends to identify the most frequently searched phrases related to PE. Subsequently, the discovered keywords were methodically entered into ChatGPT, and the resulting replies were assessed for quality using the Ensuring Quality Information for Patients (EQIP) program. The produced texts were assessed for readability using the Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease Score (FRES), and DISCERN metrics. This investigation has identified substantial concerns about the quality of texts produced by ChatGPT, highlighting severe problems with reading and understanding. The mean EQIP score for the texts was determined to be 45.93 ± 4.34, while the FRES was 15.8 ± 8.73. Additionally, the FKGL score was computed to be 15.68 ± 1.67 and the DISCERN score was 38.1 ± 3.78. The comparatively low average EQIP and DISCERN scores suggest that improvements are required to increase the quality and dependability of the presented information. In addition, the FKGL scores indicate a significant degree of linguistic intricacy, requiring a level of knowledge comparable to about 14 to 15years of formal schooling in order to understand. The texts about treatment, which are the most frequently searched items, are more difficult to understand compared to other texts about other categories. The results of this research suggest that compared to texts on other topics the PE texts produced by ChatGPT exhibit a higher degree of complexity, which exceeds the recommended reading threshold for effective health communication. Currently, ChatGPT is cannot be considered a substitute for comprehensive medical consultations. This study is to our knowledge the first reported research investigating the quality and comprehensibility of information generated by ChatGPT in relation to frequently requested queries about PE. The main limitation is that the investigation included only the first 25 popular keywords in English. ChatGPT is incapable of replacing the need for thorough medical consultations.

Does the Internet Provide Quality Information on Metoidioplasty? Using the Modified Ensuring Quality Information for Patients Tool to Evaluate Artificial Intelligence-Generated and Online Information on Metoidioplasty.

Public interest in gender affirmation surgery has surged over the last decade. This spike in interest, combined with extensive free online medical knowledge, has led many to the Internet for more information on this complicated procedure. This study aimed to evaluate the quality of online information on metoidioplasty. Google Trends in searches on "metoidioplasty" from 2004 to present were assessed. "metoidioplasty" was searched on three popular search engines (Google, Yahoo, and Bing), and the first 100 websites from each search were extracted for inclusion (Fig. 1). Exclusion criteria included duplicates, websites requiring fees, photo libraries, and irrelevant websites. Websites were assigned a score (out of 36) using the modified Ensuring Quality Information for Patients (EQIP) instrument, which grades patient materials based on content (18), identification (6), and structure (12). ChatGPT was also queried for metoidioplasty-related information and responses were analyzed using EQIP. Google Trends analysis indicated relative search interest in "metoidioplasty" has more than quadrupled since 2013(Fig. 2). Of the 93 websites included, only 2 received an EQIP score greater than 27 (6%). Website scores ranged from 7 to 33, with a mean of 18.6 ± 4.8. Mean scores were highest for websites made by health departments (22.3) and lowest for those made by encyclopedias and academic institutions (16.0). Websites with the highest frequency were research articles, web portals, hospital websites, and private practice sites, which averaged scores of 18.2, 19.7, 19.0, and 17.8, respectively. Health department sites averaged the highest content points (11.25), and academic institutions averaged the lowest (5.5). The average content point across all websites was 7.9 of 18. ChatGPT scored a total score of 29: 17 content, 2 identification, and 10 structures. The artificial intelligence chatbot scored the second highest score among all included online resources. Despite the continued use of search engines, the quality of online information on metoidioplasty remains exceptionally poor across most website developers. This study demonstrates the need to improve these resources, especially as interest in gender-affirming surgery continues to grow. ChatGPT and other artificial intelligence chatbots may be efficient and reliable alternatives for those seeking to understand complex medical information.

Responses of Five Different Artificial Intelligence Chatbots to the Top Searched Queries About Erectile Dysfunction: A Comparative Analysis

The aim of the study is to evaluate and compare the quality and readability of responses generated by five different artificial intelligence (AI) chatbots—ChatGPT, Bard, Bing, Ernie, and Copilot—to the top searched queries of erectile dysfunction (ED). Google Trends was used to identify ED-related relevant phrases. Each AI chatbot received a specific sequence of 25 frequently searched terms as input. Responses were evaluated using DISCERN, Ensuring Quality Information for Patients (EQIP), and Flesch-Kincaid Grade Level (FKGL) and Reading Ease (FKRE) metrics. The top three most frequently searched phrases were “erectile dysfunction cause”, “how to erectile dysfunction,” and “erectile dysfunction treatment.” Zimbabwe, Zambia, and Ghana exhibited the highest level of interest in ED. None of the AI chatbots achieved the necessary degree of readability. However, Bard exhibited significantly higher FKRE and FKGL ratings (p = 0.001), and Copilot achieved better EQIP and DISCERN ratings than the other chatbots (p = 0.001). Bard exhibited the simplest linguistic framework and posed the least challenge in terms of readability and comprehension, and Copilot’s text quality on ED was superior to the other chatbots. As new chatbots are introduced, their understandability and text quality increase, providing better guidance to patients.

Artificial intelligence insights into osteoporosis: assessing ChatGPT's information quality and readability.

The use of artificial intelligence(AI) applications as a source of information in the field of health is increasing. Readable and accurate information plays a critical role in empowering patients to make decisions about their disease. The aim was to examine the quality and readability of responses provided by ChatGPT, an AI chatbot, to commonly asked questions regarding osteoporosis, representing a major public health problem. "Osteoporosis," "female osteoporosis," and "male osteoporosis" were identified by using Google trends for the 25 most frequently searched keywords on Google. A selected set of 38 keywords was sequentially inputted into the chat interface of the ChatGPT. The responses were evaluated with tools of the Ensuring Quality Information for Patients (EQIP), the Flesch-Kincaid Grade Level (FKGL), and the Flesch-Kincaid Reading Ease (FKRE). The EQIP score of the texts ranged from a minimum of 36.36 to a maximum of 61.76 with a mean value of 48.71 as having "serious problems with quality." The FKRE scores spanned from 13.71 to 56.06 with a mean value of 28.71 and the FKGL varied between 8.48 and 17.63, with a mean value of 13.25. There were no statistically significant correlations between the EQIP score and the FKGL or FKRE scores. Although ChatGPT is easily accessible for patients to obtain information about osteoporosis, its current quality and readability fall short of meeting comprehensive healthcare standards.