Frequently Asked Questions Research Articles

ChatGPT-4 Performs Clinical Information Retrieval Tasks Using Consistently More Trustworthy Resources Than Does Google Search for Queries Concerning the Latarjet Procedure

PurposeTo assess the ability for ChatGPT-4, an automated Chatbot powered by artificial intelligence (AI), to answer common patient questions concerning the Latarjet procedure for patients with anterior shoulder instability and compare this performance to Google Search Engine. MethodsUsing previously validated methods, a Google search was first performed using the query “Latarjet.” Subsequently, the top ten frequently asked questions (FAQs) and associated sources were extracted. ChatGPT-4 was then prompted to provide the top ten FAQs and answers concerning the procedure. This process was repeated to identify additional FAQs requiring discrete-numeric answers to allow for a comparison between ChatGPT-4 and Google. Discrete, numeric answers were subsequently assessed for accuracy based on the clinical judgement of two fellowship-trained sports medicine surgeons blinded to search platform. ResultsMean (±standard deviation) accuracy to numeric-based answers were 2.9±0.9 for ChatGPT-4 versus 2.5±1.4 for Google (p=0.65). ChatGPT-4 derived information for answers only from academic sources, which was significantly different from Google Search Engine (p=0.003), which used only 30% academic sources and websites from individual surgeons (50%) and larger medical practices (20%). For general FAQs, 40% of FAQs were found to be identical when comparing ChatGPT-4 and Google Search Engine. In terms of sources used to answer these questions, ChatGPT-4 again used 100% academic resources, while Google Search Engine used 60% academic resources, 20% surgeon personal websites, and 20% medical practices (p=0.087). ConclusionChatGPT-4 demonstrated the ability to provide accurate and reliable information about the Latarjet procedure in response to patient queries, using multiple academic sources in all cases. This was in contrast to Google Search Engine, which more frequently used single surgeon and large medical practice websites. Despite differences in the resources accessed to perform information retrieval tasks, the clinical relevance and accuracy of information provided did not significantly differ between ChatGPT-4 and Google Search Engine.

A comparative study of English and Japanese ChatGPT responses to anaesthesia-related medical questions

PurposeThe expansion of artificial intelligence (AI) within large language models (LLMs) has the potential to streamline healthcare delivery. Despite the increased use of LLMs, disparities in their performance particularly in different languages, remain underexplored. This study examines the quality of ChatGPT responses in English and Japanese, specifically to questions related to anaesthesiology. MethodAnaesthesiologists proficient in both languages were recruited as experts in this study. Ten Frequently asked Questions (FAQs) in anaesthesia were selected and translated for evaluation. Three non-sequential responses from ChatGPT were assessed for content quality (accuracy, comprehensiveness, and safety) and communication quality (understanding, empathy/tone, and ethics) by expert evaluators. ResultEight anaesthesiologists evaluated English and Japanese LLMs responses. The overall quality for all questions combined, was higher in the English responses compared to Japanese responses. Content and communication quality were significantly higher in the English compared to Japanese LLMs responses (both p<0.001) in all three responses. Comprehensiveness, safety and understanding were higher scores in English LLMs responses. In all 3 responses, more than half of evaluators marked overall English responses were better than Japanese response. ConclusionEnglish LLMs responses to anaesthesia related FAQs were superior in quality to Japanese responses when assessed by bilingual anaesthesia experts in this report. This study highlights the potential for language-related disparities in healthcare information and the need to improve the quality of AI responses in underrepresented languages. Future studies are needed to explore these disparities in other commonly spoken languages and to compare the performance of different LLMs.

Using Google web search to analyze and evaluate the application of ChatGPT in femoroacetabular impingement syndrome.

Chat Generative Pre-trained Transformer (ChatGPT) is a new machine learning tool that allows patients to access health information online, specifically compared to Google, the most commonly used search engine in the United States. Patients can use ChatGPT to better understand medical issues. This study compared the two search engines based on: (i) frequently asked questions (FAQs) about Femoroacetabular Impingement Syndrome (FAI), (ii) the corresponding answers to these FAQs, and (iii) the most FAQs yielding a numerical response. To assess the suitability of ChatGPT as an online health information resource for patients by replicating their internet searches. Cross-sectional study. The same keywords were used to search the 10 most common questions about FAI on both Google and ChatGPT. The responses from both search engines were recorded and analyzed. Of the 20 questions, 8 (40%) were similar. Among the 10 questions searched on Google, 7 were provided by a medical practice. For numerical questions, there was a notable difference in answers between Google and ChatGPT for 3 out of the top 5 most common questions (60%). Expert evaluation indicated that 67.5% of experts were satisfied or highly satisfied with the accuracy of ChatGPT's descriptions of both conservative and surgical treatment options for FAI. Additionally, 62.5% of experts were satisfied or highly satisfied with the safety of the information provided. Regarding the etiology of FAI, including cam and pincer impingements, 52.5% of experts expressed satisfaction or high satisfaction with ChatGPT's explanations. Overall, 62.5% of experts affirmed that ChatGPT could serve effectively as a reliable medical resource for initial information retrieval. This study confirms that ChatGPT, despite being a new tool, shows significant potential as a supplementary resource for health information on FAI. Expert evaluations commend its capacity to provide accurate and comprehensive responses, valued by medical professionals for relevance and safety. Nonetheless, continuous improvements in its medical content's depth and precision are recommended for ongoing reliability. While ChatGPT offers a promising alternative to traditional search engines, meticulous validation is imperative before it can be fully embraced as a trusted medical resource.

Evaluating ChatGPT-3.5 and ChatGPT-4.0 Responses on Hyperlipidemia for Patient Education.

Introduction Hyperlipidemia is prevalent worldwide and affects a significant number of USadults. It significantly contributes to ischemic heart disease and millions of deaths annually. With the increasing use of the internet for health information, tools like ChatGPT (OpenAI, San Francisco, CA, USA) have gained traction. ChatGPT version 4.0, launched in March 2023, offers enhanced features over its predecessor but requires a monthly fee. This study compares the accuracy, comprehensibility, and response length of the free and paid versions of ChatGPT for patient education on hyperlipidemia. Materials and methods ChatGPT versions 3.5 and 4.0 were prompted in three different ways and 25 questions from the Cleveland Clinic's frequently asked questions (FAQs) on hyperlipidemia. Prompts includedno prompting (Form 1), patient-friendly prompting (Form 2), and physician-level prompting (Form 3). Responses were categorized as incorrect, partially correct, or correct. Additionally, the grade level and word count from each response were recorded for analysis. Results Overall, scoring frequencies for ChatGPT version 3.5 were: five (6.67%) incorrect, 18 partially correct (24%), and 52 (69.33%) correct. Scoring frequencies for ChatGPT version 4.0 were: one (1.33%) incorrect, 18 (24.00%) partially correct, and 56 (74.67%) correct. Correct answers did not significantly differ between ChatGPT version 3.5 and ChatGPT version 4.0 (p = 0.586). ChatGPT version 3.5 had a significantly higher grade reading level than version 4.0 (p = 0.0002). ChatGPT version 3.5 had a significantly higher word count than version 4.0 (p = 0.0073). Discussion There was no significant difference in accuracy between the free and paid versions of hyperlipidemia FAQs. Both versions provided accurate but sometimes partially complete responses. Version 4.0 offered more concise and readable information, aligning with the readability of most online medical resources despite exceeding the National Institutes of Health's (NIH's) recommended eighth-grade reading level. The paid version demonstrated superior adaptability in tailoring responses based on the input. Conclusion Both versions of ChatGPT provide reliable medical information, with the paid version offering more adaptable and readable responses. Healthcare providers can recommend ChatGPT as a source of patient education, regardless of the version used. Future research should explore diverse question formulations and ChatGPT's handling of incorrect information.

The Royal Marsden BRIDgE TRE Transparency Project

BackgroundThe BRIDgE platform is the Royal Marsden’s Trusted Research environment, which aims to transform clinical practice and improve outcomes for patients by enabling data analytics and AI development using real-world cancer data within secure, collaborative, cloud-based workspaces. This project evaluated the BRIDgE website against the new UK Health Data Research UK Transparency Standards, leading to changes to implement the following: Open Access Application form and Guidance (Standard 1), Transparent Access Process and Criteria (Standard 2), Clear website navigation (Standard 3), Consider Target Audience (Standard 4), Regular Review of Website Content (Standard 5), and Transparency of Data Use and Auditing (Standard 6). Our primary aim was to improve the content and sign posting so that patients and researchers would be better informed, resulting in increased website traffic and requests via our Contact Us page. MethodsInformation about the transparency project and the updates to the website were presented to a patient panel for feedback and a leaflet was co-designed with patient representatives. ResultsChanges included a new BRIDgE home page, which was re-designed to make navigation simpler and clearer, improved signposting, so both patients and researchers were aware of the sections meant for their consumption, such as the new Researchers’ Frequently Asked Questions (FAQ) page, which contained detailed instructions on how to request data, the governance workflow and costing structure. Finally, we expanded the Research Projects and Publications page, and added a Case Study from one of the Researchers, detailing their personal experience on the platform. ConclusionsWe anticipate that the impact of this project will be that patients will be more informed about Trusted Research Environments in general and what BRIDgE specifically enables in terms of innovative research and data security, leading to greater assurance for patients that their data is being used securely for their benefit. Researchers will get the benefit of the new FAQ page, which includes a great deal more information than was previously available about the platform, the data access methods, and the governance processes. We envisage that this will lead to more traffic on the site and a greater number of requests for information and workspaces via the Contact Us page. Finally, the patient leaflet will be the first time that patients in our waiting areas will be able to read about research informatics at the Royal Marsden and the work of our data scientists, in paper form. The resources developed using the funding from the transparency project are the revamped BRIDgE website https://www.cancerbrc.org/BRIDgE and (soon to be published) patient leaflet.

Educating patients on osteoporosis and bone health: Can "ChatGPT" provide high-quality content?

The rise of artificial intelligence (AI) models like ChatGPT offers potential for varied applications, including patient education in healthcare. With gaps in osteoporosis and bone health knowledge and adherence to prevention and treatment, this study aims to evaluate the accuracy of ChatGPT in delivering evidence-based information related to osteoporosis. Twenty of the most common frequently asked questions (FAQs) related to osteoporosis were subcategorized into diagnosis, diagnostic method, risk factors, and treatment and prevention. These FAQs were sourced online and inputted into ChatGPT-3.5. Three orthopedic surgeons and one advanced practice provider who routinely treat patients with fragility fractures independently reviewed the ChatGPT-generated answers, grading them on a scale from 0 (harmful) to 4 (excellent). Mean response accuracy scores were calculated. To compare the variance of the means across the four categories, a one-way analysis of variance (ANOVA) was used. ChatGPT displayed an overall mean accuracy score of 91%. Its responses were graded as "accurate requiring minimal clarification" or "excellent," with a mean response score ranging from 3.25 to 4. No answers were deemed inaccurate or harmful. No significant difference was observed in the means of responses across the defined categories. ChatGPT-3.5 provided high-quality educational content. It showcased a high degree of accuracy in addressing osteoporosis-related questions, aligning closely with expert opinions and current literature, with structured and inclusive answers. However, while AI models can enhance patient information accessibility, they should be used as an adjunct rather than a substitute for human expertise and clinical judgment.

Pregnancy Loss: A Comparative Analysis of Information Delivery Between ACOG FAQs and ChatGPT [ID 2683594

INTRODUCTION: Pregnancy loss is a sensitive topic, and clear, reliable information is crucial. This study compares the readability and accuracy of ChatGPT's responses to ACOG's frequently asked questions (FAQ) sheets on the subject. METHODS: Within the five FAQ sheets under “Pregnancy Loss,” 66 questions were assessed using ChatGPT-3.5. Readability scores, ie, Flesch–Kincaid Reading Ease, Flesch–Kincaid Grade Level, Gunning Fog Score, Smog Index, Coleman Liau Index, and Automated Readability Index were computed for each response. The quality of responses were also graded by two maternal–fetal medicine specialists using a 1–4 scale, where 1 represents a comprehensive response and 4 indicates an incorrect response. Statistical analysis utilized a two-tailed t-test. A weighted Cohen’s kappa coefficient evaluated interrater reliability. RESULTS: ACOG attained better readability scores over ChatGPT across all six metrics with statistical significance (P&lt;.001). Among grading, ACOG also outperformed ChatGPT in quality of information with a summative mean of 1.53 (SD 0.60) versus 1.85 (SD 0.90) with statistical significance (P&lt;.001). Grading for ACOG possessed a Cohen's kappa coefficient of 0.493, implying moderate agreement between graders. Grading for ChatGPT possessed a Cohen's kappa coefficient of 0.744, implying substantial agreement between graders. CONCLUSION: The responses by ACOG to frequently asked questions on pregnancy loss were both clearer and more comprehensive than those of ChatGPT with consistent agreement between graders. Although ChatGPT can be a convenient source of medical queries, further improvements ought to be made to these large-language models to provide clear and accurate information regarding pregnancy loss to patients.

Navigating Intimate Partner Violence: A Comparative Analysis Between ACOG and ChatGPT on Patient Education [ID 2683552

INTRODUCTION: With the surge in artificial intelligence usage, there is an increasing need to assess its applicability in conveying public health information. We compare the readability and quality of responses between ChatGPT and the American College of Obstetricians and Gynecologists (ACOG) in answering frequently asked questions (FAQs) on intimate partner violence (IPV). METHODS: Twelve questions from ACOG's “Intimate Partner Violence” FAQs were posed to ChatGPT-3.5 (July 19 Version). Readability and grade-level scores were determined. The quality of responses were also graded by two obstetrician–gynecologists using a 1–4 scale, where 1 represents a comprehensive response and 4 indicates an incorrect response. Statistical analysis utilized a two-tailed t-test. A weighted Cohen’s kappa coefficient evaluated interrater reliability. RESULTS: Mean readability favored ACOG over ChatGPT, but only the Coleman Liau Index was statistically significant (ACOG: 12.71; ChatGPT: 15.76; P=.003). Other readability measures demonstrated no significant differences. The ACOG responses were graded with an average of 1.29, alongside a Cohen’s kappa coefficient of 0.375, implying fair agreement between graders. All ChatGPT’s responses were graded a 1 with a Cohen’s kappa coefficient of 1, indicating perfect agreement. The difference in grades between ACOG and ChatGPT was statistically significant (P=.013). CONCLUSION: The ACOG FAQ responses are relatively equivalent in readability when compared to ChatGPT-generated responses with only one statistically significant difference across the indices. However, ChatGPT's responses were more comprehensive and accurate. Although patients can use ACOG as a source, ChatGPT provides equivalently clear and more comprehensive information on IPV.

ChatGPT-3.5 Responses Require Advanced Readability for the General Population and May Not Effectively Supplement Patient-Related Information Provided by the Treating Surgeon Regarding Common Questions About Rotator Cuff Repair

The purpose of this study was to investigate the accuracy of ChatGPT's responses to frequently asked questions (FAQ) prior to rotator cuff repair surgery. The ten most common FAQs related to rotator cuff repair were compiled from four institution websites. Questions were then input into ChatGPT-3.5 in one session. The provided ChatGPT-3.5 responses were analyzed by two orthopedic surgeons for reliability, quality, and readability using The Journal of the American Medical Association (JAMA) Benchmark criteria, the DISCERN score, and the Flesch-Kincaid Grade Level. The JAMA Benchmark criteria score was 0, indicating the absence of reliable source material citations. The mean Flesch-Kincaid Grade Level was 13.4 (range, 11.2-15.0). The mean DISCERN score was 43.4 (range, 36-51) indicating that the quality of the responses overall was considered fair. All responses cited making final decision-making to be made with the treating physician. ChatGPT-3.5 provided substandard patient-related information in alignment with recommendations from the treating surgeon regarding common questions around rotator cuff repair surgery. Additionally, the responses lacked reliable source material citations and the readability of the responses was relatively advanced with a complex language style. The findings of this study suggest that ChatGPT-3.5 may not effectively supplement patient-related information in the context of recommendations provided by the treating surgeon prior to rotator cuff repair surgery.

Islamic QA with Chatbot System Using Convolutional Neural Network

Many questions and answers about Islamic law are scattered on the internet and have been explained repeatedly by various sites. One solution is presented by the website www.piss-ktb.com, which creates a web-based source of information in the form of Frequently Asked Questions (FAQ). However, web-based FAQs have a weakness: users still have to browse through the available questions one by one according to the questions they want to know the answers to. Browsing through thousands of FAQs is inefficient and exhausting. Thus, a chatbot system can become a better alternative to the FAQ website. Still, chatbots are difficult to use because most of their conversations are hard to understand. A single character error will cause the system to misunderstand its meaning. In reality, users expect a chatbot that can understand everyday language. Thus, it is necessary to develop a chatbot system that can understand various common sentence combinations in everyday language and understand the meaning of words. In addition, it should be able to predict answers automatically to various kinds of questions and requests, even though the initial training data is relatively low. Therefore, this study aims to develop a system that can provide answers automatically based on user commands in natural language using Global Vectors for Word Representations (GloVe), Convolutional Neural Networks (CNN), and Transfer Learning techniques. The result shows that the use of transfer learning and the Nadam optimizer can improve the system’s performance.

Evaluation of Patient Education Materials From Large-Language Artificial Intelligence Models on Carpal Tunnel Release.

ChatGPT, an artificial intelligence technology, has the potential to be a useful patient aid, though the accuracy and appropriateness of its responses and recommendations on common hand surgical pathologies and procedures must be understood. Comparing the sources referenced and characteristics of responses from ChatGPT and an established search engine (Google) on carpal tunnel surgery will allow for an understanding of the utility of ChatGPT for patient education. A Google search of "carpal tunnel release surgery" was performed and "frequently asked questions (FAQs)" were recorded with their answer and source. ChatGPT was then asked to provide answers to the Google FAQs. The FAQs were compared, and answer content was compared using word count, readability analyses, and content source. There was 40% concordance among questions asked by the programs. Google answered each question with one source per answer, whereas ChatGPT's answers were created from two sources per answer. ChatGPT's answers were significantly longer than Google's and multiple readability analysis algorithms found ChatGPT responses to be statistically significantly more difficult to read and at a higher grade level than Google's. ChatGPT always recommended "contacting your surgeon." A comparison of ChatGPT's responses to Google's FAQ responses revealed that ChatGPT's answers were more in-depth, from multiple sources, and from a higher proportion of academic Web sites. However, ChatGPT answers were found to be more difficult to understand. Further study is needed to understand if the differences in the responses between programs correlate to a difference in patient comprehension.

ChatGPT4's proficiency in addressing patients' questions on systemic lupus erythematosus: a blinded comparative study with specialists.

The efficacy of artificial intelligence (AI)-driven chatbots like ChatGPT4 in specialized medical consultations, particularly in rheumatology, remains underexplored. This study compares the proficiency of ChatGPT4' responses with practicing rheumatologists to inquiries from patients with SLE. In this cross-sectional study, we curated 95 frequently asked questions (FAQs), including 55 in Chinese and 40 in English. Responses for FAQs from ChatGPT4 and five rheumatologists were scored separately by a panel of rheumatologists and a group of patients with SLE across six domains (scientific validity, logical consistency, comprehensibility, completeness, satisfaction level and empathy) on a 0-10 scale (a score of 0 indicates entirely incorrect responses, while 10 indicates accurate and comprehensive answers). Rheumatologists' scoring revealed that ChatGPT4-generated responses outperformed those from rheumatologists in satisfaction level and empathy, with mean differences of 0.537 (95% CI, 0.252-0.823; P < 0.01) and 0.460 (95% CI, 0.227-0.693; P < 0.01), respectively. From the SLE patients' perspective, ChatGPT4-generated responses were comparable to the rheumatologist-provided answers in all six domains. Subgroup analysis revealed ChatGPT4 responses were more logically consistent and complete regardless of language and exhibited greater comprehensibility, satisfaction and empathy in Chinese. However, ChatGPT4 responses were inferior in comprehensibility for English FAQs. ChatGPT4 demonstrated comparable, possibly better in certain domains, to address FAQs from patients with SLE, when compared with the answers provided by specialists. This study showed the potential of applying ChatGPT4 to improve consultation in SLE patients.

Understanding Public Perception of Naloxone: A Study of FAQs and Answer Source Credibility

Purpose The most commonly used intervention for opioid overdoses is naloxone. With naloxone soon to be sold over-the-counter in the United States, the goal of this paper is to categorize frequently asked questions (FAQs) and answers about naloxone using internet sources in a cross-sectional fashion. Methods Terms “narcan” and “naloxone” were searched on a clean Google Chrome browser using the “People also asked” tab to find FAQs and their answer sources. We classified questions and sources and assessed each website’s quality and credibility grading with JAMA benchmark criteria. The Kruskal-Wallis H test was used to determine variance of mean JAMA score by source type and Post-Hoc Dunn’s test with Bonferroni corrected alpha of 0.005 used to compare source types. Results Of the 305 unique questions, 202 (66.2%) were classified as facts, 78 (25.6%) were policy, and 25 (8.2%) were value. Of the 144 unique answer sources, the two most common included 55 (38.2%) which were government entities and 47 (32.6%) which were commercial entities. Ninety-two (of 144, 63.9%) sources met three or more JAMA benchmark criteria. Statistical analysis showed a significant difference between the JAMA benchmark scores by source type H(4) = 12.75, p = 0.0126 and between the mean rank of academic and government sources (p = 0.0036). Conclusion We identified FAQs and their citations about naloxone, highlighting potential lack of understanding and knowledge of this important intervention. We recommend updating websites to accurately reflect current and useful information for those that may require naloxone.

Medical Question Summarization with Entity-driven Contrastive Learning

By summarizing longer consumer health questions into shorter and essential ones, medical question-answering systems can more accurately understand consumer intentions and retrieve suitable answers. However, medical question summarization is very challenging due to obvious distinctions in health trouble descriptions from patients and doctors. Although deep learning has been applied to successfully address the medical question summarization (MQS) task, two challenges remain: how to correctly capture question focus to model its semantic intention, and how to obtain reliable datasets to fairly evaluate performance. To address these challenges, this article proposes a novel medical question summarization framework based on e ntity-driven c ontrastive l earning (ECL). ECL employs medical entities present in frequently asked questions (FAQs) as focuses and devises an effective mechanism to generate hard negative samples. This approach compels models to focus on essential information and consequently generate more accurate question summaries. Furthermore, we have discovered that some MQS datasets, such as the iCliniq dataset with a 33% duplicate rate, have significant data leakage issues. To ensure an impartial evaluation of the related methods, this article carefully examines leaked samples to reorganize more reasonable datasets. Extensive experiments demonstrate that our ECL method outperforms the existing methods and achieves new state-of-the-art performance, i.e., 52.85, 43.16, 41.31, 43.52 in terms of ROUGE-1 metric on MeQSum, CHQ-Summ, iCliniq, HealthCareMagic dataset, respectively. The code and datasets are available at https://github.com/yrbobo/MQS-ECL

AI-driven translations for kidney transplant equity in Hispanic populations

Health equity and accessing Spanish kidney transplant information continues being a substantial challenge facing the Hispanic community. This study evaluated ChatGPT’s capabilities in translating 54 English kidney transplant frequently asked questions (FAQs) into Spanish using two versions of the AI model, GPT-3.5 and GPT-4.0. The FAQs included 19 from Organ Procurement and Transplantation Network (OPTN), 15 from National Health Service (NHS), and 20 from National Kidney Foundation (NKF). Two native Spanish-speaking nephrologists, both of whom are of Mexican heritage, scored the translations for linguistic accuracy and cultural sensitivity tailored to Hispanics using a 1–5 rubric. The inter-rater reliability of the evaluators, measured by Cohen’s Kappa, was 0.85. Overall linguistic accuracy was 4.89 ± 0.31 for GPT-3.5 versus 4.94 ± 0.23 for GPT-4.0 (non-significant p = 0.23). Both versions scored 4.96 ± 0.19 in cultural sensitivity (p = 1.00). By source, GPT-3.5 linguistic accuracy was 4.84 ± 0.37 (OPTN), 4.93 ± 0.26 (NHS), 4.90 ± 0.31 (NKF). GPT-4.0 scored 4.95 ± 0.23 (OPTN), 4.93 ± 0.26 (NHS), 4.95 ± 0.22 (NKF). For cultural sensitivity, GPT-3.5 scored 4.95 ± 0.23 (OPTN), 4.93 ± 0.26 (NHS), 5.00 ± 0.00 (NKF), while GPT-4.0 scored 5.00 ± 0.00 (OPTN), 5.00 ± 0.00 (NHS), 4.90 ± 0.31 (NKF). These high linguistic and cultural sensitivity scores demonstrate Chat GPT effectively translated the English FAQs into Spanish across systems. The findings suggest Chat GPT’s potential to promote health equity by improving Spanish access to essential kidney transplant information. Additional research should evaluate its medical translation capabilities across diverse contexts/languages. These English-to-Spanish translations may increase access to vital transplant information for underserved Spanish-speaking Hispanic patients.

ChatGPT Responses to Common Questions About Slipped Capital Femoral Epiphysis: A Reliable Resource for Parents?

We sought to evaluate the ability of ChatGPT, an AI-powered online chatbot, to answer frequently asked questions (FAQs) regarding slipped capital femoral epiphysis (SCFE). Seven FAQs regarding SCFE were presented to ChatGPT. Initial responses were recorded and compared with evidence-based literature and reputable online resources. Responses were subjectively rated as "excellent response requiring no further clarification," "satisfactory response requiring minimal clarification," "satisfactory response requiring moderate clarification," or "unsatisfactory response requiring substantial clarification." ChatGPT was frequently able to provide satisfactory responses that required only minimal clarification. One response received an excellent rating and required no further clarification, while only 1 response from ChatGPT was rated unsatisfactory and required substantial clarification. ChatGPT is able to frequently provide satisfactory responses to FAQs regarding SCFE while appropriately reiterating the importance of always consulting a medical professional.

ChatGPT vs. neurologists: a cross-sectional study investigating preference, satisfaction ratings and perceived empathy in responses among people living with multiple sclerosis.

ChatGPT is an open-source natural language processing software that replies to users' queries. We conducted a cross-sectional study to assess people living with Multiple Sclerosis' (PwMS) preferences, satisfaction, and empathy toward two alternate responses to four frequently-asked questions, one authored by a group of neurologists, the other by ChatGPT. An online form was sent through digital communication platforms. PwMS were blind to the author of each response and were asked to express their preference for each alternate response to the four questions. The overall satisfaction was assessed using a Likert scale (1-5); the Consultation and Relational Empathy scale was employed to assess perceived empathy. We included 1133 PwMS (age, 45.26 ± 11.50years; females, 68.49%). ChatGPT's responses showed significantly higher empathy scores (Coeff = 1.38; 95% CI = 0.65, 2.11; p > z < 0.01), when compared with neurologists' responses. No association was found between ChatGPT' responses and mean satisfaction (Coeff = 0.03; 95% CI = -0.01, 0.07; p = 0.157). College graduate, when compared with high school education responder, had significantly lower likelihood to prefer ChatGPT response (IRR = 0.87; 95% CI = 0.79, 0.95; p < 0.01). ChatGPT-authored responses provided higher empathy than neurologists. Although AI holds potential, physicians should prepare to interact with increasingly digitized patients and guide them on responsible AI use. Future development should consider tailoring AIs' responses to individual characteristics. Within the progressive digitalization of the population, ChatGPT could emerge as a helpful support in healthcare management rather than an alternative.

Evaluating the accuracy and relevance of ChatGPT responses to frequently asked questions regarding total knee replacement

BackgroundChat Generative Pretrained Transformer (ChatGPT), a generative artificial intelligence chatbot, may have broad applications in healthcare delivery and patient education due to its ability to provide human-like responses to a wide range of patient queries. However, there is limited evidence regarding its ability to provide reliable and useful information on orthopaedic procedures. This study seeks to evaluate the accuracy and relevance of responses provided by ChatGPT to frequently asked questions (FAQs) regarding total knee replacement (TKR).MethodsA list of 50 clinically-relevant FAQs regarding TKR was collated. Each question was individually entered as a prompt to ChatGPT (version 3.5), and the first response generated was recorded. Responses were then reviewed by two independent orthopaedic surgeons and graded on a Likert scale for their factual accuracy and relevance. These responses were then classified into accurate versus inaccurate and relevant versus irrelevant responses using preset thresholds on the Likert scale.ResultsMost responses were accurate, while all responses were relevant. Of the 50 FAQs, 44/50 (88%) of ChatGPT responses were classified as accurate, achieving a mean Likert grade of 4.6/5 for factual accuracy. On the other hand, 50/50 (100%) of responses were classified as relevant, achieving a mean Likert grade of 4.9/5 for relevance.ConclusionChatGPT performed well in providing accurate and relevant responses to FAQs regarding TKR, demonstrating great potential as a tool for patient education. However, it is not infallible and can occasionally provide inaccurate medical information. Patients and clinicians intending to utilize this technology should be mindful of its limitations and ensure adequate supervision and verification of information provided.

Performance of large language models on benign prostatic hyperplasia frequently asked questions.

Benign prostatic hyperplasia (BPH) is a common condition, yet it is challenging for the average BPH patient to find credible and accurate information about BPH. Our goal is to evaluate and compare the accuracy and reproducibility of large language models (LLMs), including ChatGPT-3.5, ChatGPT-4, and the New Bing Chat in responding to a BPH frequently asked questions (FAQs) questionnaire. A total of 45 questions related to BPH were categorized into basic and professional knowledge. Three LLM-ChatGPT-3.5, ChatGPT-4, and New Bing Chat-were utilized to generate responses to these questions. Responses were graded as comprehensive, correct but inadequate, mixed with incorrect/outdated data, or completely incorrect. Reproducibility was assessed by generating two responses for each question. All responses were reviewed and judged by experienced urologists. All three LLMs exhibited high accuracy in generating responses to questions, with accuracy rates ranging from 86.7% to 100%. However, there was no statistically significant difference in response accuracy among the three (p > 0.017 for all comparisons). Additionally, the accuracy of the LLMs' responses to the basic knowledge questions was roughly equivalent to that of the specialized knowledge questions, showing a difference of less than 3.5% (GPT-3.5: 90% vs. 86.7%; GPT-4: 96.7% vs. 95.6%; New Bing: 96.7% vs. 93.3%). Furthermore, all three LLMs demonstrated high reproducibility, with rates ranging from 93.3% to 97.8%. ChatGPT-3.5, ChatGPT-4, and New Bing Chat offer accurate and reproducible responses to BPH-related questions, establishing them as valuable resources for enhancing health literacy and supporting BPH patients in conjunction with healthcare professionals.

Parental concerns about oral health of children: Is ChatGPT helpful in finding appropriate answers?

Artificial intelligence (AI) is becoming an important part of our lives owing to increased data availability and improved power of computing. One of the recently launched modalities of AI, ChatGPT, is being enormously used worldwide for different types of tasks. In medical context, its use is being explored for clinical queries, academia, research help, etc. Further, literature suggests that parents seek information about health of their children using different Internet resources and would surely turn toward ChatGPT for the same, as this chatbot model is easy to use, generates "one" response, and is available without any subscription. ChatGPT generates a response using text cues and applying different algorithms on prepublished literature but is still in its naïve state; hence, it is imperative to validate the generated responses. Accordingly, we planned this study to determine the clarity, correctness, and completeness of some Frequently asked questions (FAQs) about child's oral health, from a mother's perspective. The study design was a vignette-based survey and included a set of 23 questions, for which ChatGPT was interviewed from the perspective of an imaginary parent. The answers responded by ChatGPT were copied "verbatim," and a Google survey form was designed. The survey form was validated and then sent to 15 pediatric dentists, and the responses were mainly collected on the Likert's scale with a provision of one open-ended question aiming to determine "what they would have added" to this generated response as an expert in the field. The responses on Likert's scale were condensed and values ≥4 were considered 'adequate and acceptable' while scores ≤3, were considered 'inadequate'. The generated responses and comments mentioned by different respondents in the open-ended question were critiqued in reference to the existing literature. Overall, the responses were found to be complete and logical and in clear language, with only some inadequacies being reported in few of the answers.