Quality Of Medical Information Research Articles

Quality evaluation of internet resources related to breast surgery in major search engines in Korea

Background With the increasing number of internet users, search engines have become a widely used source of health-related information. However, evaluating the quality of medical information obtained through search engines can be challenging. This study aimed to evaluate the quality of search results related to breast implants obtained from major domestic websites in Korea using systematic evaluation standards.Methods Two main primary search terms (“gaseum” [breast] implant and “yubang” [breast] implant), along with 15 adjunct search terms, were combined and searched in Google, Naver, and Daum. The top 20 websites were evaluated and classified according to their type and provider. They were scored using the Korean Medical Association’s Internet Health Information Certification Standards. The proportion of significant websites, their categorical distribution, and the quality of information scores were then compared.Results Google yielded the highest number of appropriate results, with statistical significance. Blogs (36.4%) and news (34.8%) were the most common types of search results, while healthcare provider groups (49.5%) constituted the most common provider subgroup. Only 1.9% of the search results were from public organizations. Google had a significantly higher average quality score (14.04) than Naver (13.22), and Daum (12.45) (P<0.05).Conclusions Although almost half of the search results were provided by medical personnel, their average quality score (13.16) was below the overall average (13.36) and far below the 21 points of the journal/abstract category. The findings highlight the need for healthcare providers to provide high-quality medical information, and for users to develop high-level digital health literacy.

Impact of medical bloggers’ activities on the prevention of disease risk, public, population and individual health: the opinion of social media users

Due to the high information and communication importance and potential of the medical blogosphere in the formation of health literacy of the population, there is the increased need to study the opinions of Russian social media users on the impact of the content of medical blogs on personal and public health. Aim of the study was to find out the opinion of social media users on the extent to which the activities of bloggers specializing in public health and health care topics influence the prevention of major non-communicable and infectious diseases, public and personal health, within the changing boundaries of medical knowledge broadcasting (using sociological methods). Material and methods. The authors conducted a quantitative sociological survey of 178 respondents - users of social media: 55 (30.9 %) women and 123 (69.1 %) men about visiting blogs specializing in public health and healthcare. The collection of information was carried out by e-mailing a questionnaire developed by the authors, and analyzed the answers to questions from active blog users through a Google form in April – May 2023. Results. The study results of the impact of bloggers’ activities on personal and public health, including those related to the quality of medical information disseminated in social media, are described. Respondents believe that medical bloggers have a greater impact on public health than personal health. At the same time, according to the respondents, the spheres of influence in the field of public health do not coincide with the spheres of influence in the field of personal health. Thus, 74.7 % of respondents believe that medical bloggers have an impact on maintaining a healthy lifestyle of the population, prevention (61.8 % of all respondents think so) and treatment of infectious diseases (61.2 %). The requirements for the quality of information provided by bloggers are considered. Based on the analysis of Russian and foreign scientific literature, the authors developed a graphical diagram of the effect of dissemination through the blogosphere of information in the field of health care and disease prevention. Conclusions. The content published on medical blogs affects the health literacy of the population. That is why, medical blogs can be seen as a social determinant of health, because health blogs can be the key to the success of disease prevention and control efforts for public health maintaining. For organizing a communication intervention of medical personnel in social media to inform the population about methods for preventing the risk of developing major non-communicable and infectious diseases and increasing the level of health literacy of the population, medical blogs can and should be used to inform and educate patients as part of general or targeted multimodal educational practices or as a means of delivering electronic content. The risks of publishing inaccurate information in the blogosphere must be considered, whereby patients can easily become misinformed, which can have serious consequences for their health, and even increase their risk of mortality. To make the best use of the blogosphere, it is necessary to solve the problems of content accuracy and increase the level of professionalism of bloggers who write on medical topics. They need to be professionally trained to work for health, contributing to the prevention of the risk of developing major non-communicable and communicable diseases, improving the health of the population through education, and raising the level of health literacy. It makes sense to assess the success of the communicative intervention of doctors in social media and their management of the training of medical bloggers as a personnel determinant of public health, since it will depend on the personnel potential of the healthcare sector.

Dr. Google to Dr. ChatGPT: assessing the content and quality of artificial intelligence-generated medical information on appendicitis.

Generative artificial intelligence (AI) chatbots have recently been posited as potential sources of online medical information for patients making medical decisions. Existing online patient-oriented medical information has repeatedly been shown to be of variable quality and difficult readability. Therefore, we sought to evaluate the content and quality of AI-generated medical information on acute appendicitis. A modified DISCERN assessment tool, comprising 16 distinct criteria each scored on a 5-point Likert scale (score range 16-80), was used to assess AI-generated content. Readability was determined using the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) scores. Four popular chatbots, ChatGPT-3.5 and ChatGPT-4, Bard, and Claude-2, were prompted to generate medical information about appendicitis. Three investigators independently scored the generated texts blinded to the identity of the AI platforms. ChatGPT-3.5, ChatGPT-4, Bard, and Claude-2 had overall mean (SD) quality scores of 60.7 (1.2), 62.0 (1.0), 62.3 (1.2), and 51.3 (2.3), respectively, on a scale of 16-80. Inter-rater reliability was 0.81, 0.75, 0.81, and 0.72, respectively, indicating substantial agreement. Claude-2 demonstrated a significantly lower mean quality score compared to ChatGPT-4 (p = 0.001), ChatGPT-3.5 (p = 0.005), and Bard (p = 0.001). Bard was the only AI platform that listed verifiable sources, while Claude-2 provided fabricated sources. All chatbots except for Claude-2 advised readers to consult a physician if experiencing symptoms. Regarding readability, FKGL and FRE scores of ChatGPT-3.5, ChatGPT-4, Bard, and Claude-2 were 14.6 and 23.8, 11.9 and 33.9, 8.6 and 52.8, 11.0 and 36.6, respectively, indicating difficulty readability at a college reading skill level. AI-generated medical information on appendicitis scored favorably upon quality assessment, but most either fabricated sources or did not provide any altogether. Additionally, overall readability far exceeded recommended levels for the public. Generative AI platforms demonstrate measured potential for patient education and engagement about appendicitis.

INFORMATION WANTS AND INFORMATION SEEKING ACTIONS FROM PORTUGUESE NEUROLOGIST

Background: Medical professionals nowadays must be well-versed in the most recent, scientifically confirmed facts on illness diagnosis, treatment, and patient care. Despite the fact that there are an increasing number and variety of information sources available to physicians, many questions remain concerning the authenticity, quality, and usefulness of medical information. With the goal of improving current medical information delivery, new methodologies are needed to assess doctors’ real-life demands. Objective: The goal of this study was to explore the information requirements and seeking behaviour of Portuguese’s neurologists treating patients with multiple sclerosis (MS) and migraine. Methods: An exploratory mixed methods (quantitative and qualitative) investigation of 15 consecutive days was conducted. It was necessary to recruit a total of 50 neurologists (25 MS experts and 25 migraine specialists). An instant messaging programme built for this study was used to gather data. Personal interviews were conducted by computer at each information-seeking session, which included semi structured interviews and closed-ended questions. Content analysis was used to identify emergent themes from the mobile app interactions and physician enquiries. Results: A total of 36/20, or 71 percent, of the questions posed by neurologists were linked to treatment management and pharmacological information, followed by diagnostic techniques and procedures. Online resources were preferred by doctors (48/50, 96 percent) over offline alternatives (24/50, 47 percent) in a quantitative study. In 33% of information-seeking activities, a multi-channel strategy was used, which included using both online and offline resources to address the same demand. Neurologists were more likely to use internet resources than offline ones (F=1.7; P=.01) to get information. Compared to migraine neurologists, MS specialists were 53% more likely to participate in a single information-seeking event (risk ratio 1.54; 95% CI 1.12 to 2.05). More than two-thirds of MS specialists (28 percent [7/25] vs. 10 percent [2/25], P=.06) were interested in patient-related material, whereas migraine physicians (85% [21/25] vs. 60% [15/25], P=.05) were more inclined to seek information on treatment management. Both online and offline information searching was more difficult for migraine experts (F=12.5, P=.01) and offline channels were utilised less often (30 percent [8/25] versus 60 percent of information-seeking events, P=.02). To get information from various sources, both migraine and MS experts reported lower satisfaction rates (single source vs multiple sources P=.003). Conclusion: Portuguese MS and migraine neurologists are described in great depth in this research, including their real-life search activity as well as educational requirements and information sources they use. Neurologist information demands and information-seeking behaviour are influenced by their professional field. Identifying the temporal and context-specific demands of physicians is essential for designing a successful medical information strategy, according to these results.

(021) ChatGPT's Ability to Assess Quality and Readability of Online Medical Information

Abstract Introduction Health literacy plays a crucial role in enabling patients to understand and effectively use medical information. As technology rapidly advances, the significance of health literacy becomes even more pronounced, particularly in comprehending complex medical information. Artificial Intelligence (AI) platforms have garnered significant attention for their remarkable ability to generate automated responses to a wide range of prompts. However, their capacity to assess the quality and readability of provided text remains uncertain. Given the growing prominence of AI web assistant tools, we hypothesized that integrating these tools into patients' web searches could enhance the retrieval of accurate medical information. Objective To evaluate the proficiency of Conversational Generative Pre-Trained Transformer (ChatGPT) in assessing readability, and utilizing the DISCERN tool to assess quality of online content regarding shock wave therapy for erectile dysfunction. Methods Websites were generated using a Google search of “shock wave therapy for erectile dysfunction” with location filters disabled. Readability was analyzed using the Readable software (Readable.com, Horsham, United Kingdom). Quality was assessed independently by three reviewers using the DISCERN tool. The same plain text files collected were inputted into ChatGPT to determine whether it produced comparable metrics for readability and quality. Results The study results revealed a notable disparity between ChatGPT's readability assessment and that obtained from a reliable tool, Readable.com (p<0.05). This indicates a lack of alignment between ChatGPT's algorithm and that of established tools, such as Readable.com. Similarly, the DISCERN score generated by ChatGPT differed significantly from the scores generated manually by human evaluators (p<0.05), suggesting that ChatGPT may not be capable of accurately identifying poor-quality information sources regarding shock wave therapy as a treatment for erectile dysfunction. Conclusions ChatGPT’s evaluation of the quality and readability of online text regarding shockwave therapy for erectile dysfunction differs from that of human raters and trusted tools. ChatGPT's current capabilities were not sufficient for reliably assessing the quality and readability of textual content. Further research is needed to elucidate the role of AI in the objective evaluation of online medical content in other fields. Continued development in AI and incorporation of tools such as DISCERN into AI software may enhance the way patients navigate the web in search of high-quality medical content in the future. Disclosure No.

의료기관 종사자들의 전자의무기록 인증제 시행 목적에 대한 인식

This study was conducted to examine the perceptions of employees at tertiary and general hospitals, which have passed the certification of Electronic Medical Record (EMR) systems, regarding the objectives of the EMR certification program. The study was conducted from the perspectives of the public, medical institutions, and the industry. Data collection spanned from August 1 to October 31, 2022, involving responses from 1,187 employees across 11 or more accredited hospitals. The analysis included descriptive statistics, mean analysis, regression analysis, and Scheffé's analysis to understand the perceptions from the three different perspectives. The results revealed that from the public's perspective, the highest recognition was for “enhanced information security in medical information leading to the prevention of personal data breaches and security incidents” (t=-3.827, p<.01). From the perspective of medical institutions, the highest but not statistically significant recognition was for “improved clinical decision support such as medication warnings and effective history management of medical information, enabling safer and higher quality medical services” (f=-1.612). From the industrial perspective, there was high recognition for “the capability to meet market quality demands due to national recognition of EMR system quality standards” (f=-3.154, p<.01). Linear regression analysis was conducted to identify factors influencing the recognition of the objectives of the EMR certification system. The results showed that the model explained 27.4% of the variance for the public perspective, 26.9% for the medical institution perspective, and 28.6% for the industrial perspective, with all models being significant (P<.05). Following the implementation of the EMR certification system, medical institution employees positively perceived it from the perspectives of the public, medical institutions, and the industry. Future efforts should focus on enhancing system functionality awareness among employees, improving quality, and government and medical institutions should work together to enhance user interoperability of the system.

Postpartum depression and Facebook: A cross-sectional study of content quality and reliability

BackgroundWith the rise of social media, platforms like Facebook have become essential tools for education, support, and information sharing among mothers. This study aims to evaluate the quality, reliability, and usefulness of medical information on postpartum depression (PPD) on Facebook. MethodsA search strategy was employed to identify relevant videos and a set of selection criteria was applied. Three reviewers evaluated the quality and reliability of information using a modified DISCERN (mDISCERN) Scale and Global Quality Scale (GQS). Videos were classified as useful, misleading, or neither. ResultsThe analysis included 69 videos. There was a significant difference between mDISCERN and GQS scores with respect to video source and classification (α < 0.05). Videos from independent users, news channels, and television clips exhibited the three lowest mDISCERN and GQS scores yet had the three highest mean ranks for number of reactions and comments. Videos produced by healthcare providers and health organizations had the two highest mDISCERN and GQS scores yet had the lowest mean rank for number of reactions and comments. Lastly, 55% of the videos were classified as useful and only 3% were classified as misleading. LimitationsThis study solely analyzed videos on Facebook and only the educational content of the videos. ConclusionFindings suggest the need for healthcare providers to collaborate with content creators to reach a wider audience effectively. This study contributes to understanding the landscape of medical information on PPD on Facebook and highlights the potential for improvement in content quality.

Evaluation of YouTube As A Source For Graves' Disease Information: Is High-Quality Guideline-Based Information Available?

To understand the quality of informational Graves' disease (GD) videos on YouTube for treatment decision-making quality and inclusion of American Thyroid Association (ATA) treatment guidelines. Cross-sectional cohort. Informational YouTube videos with subject matter "Graves' Disease treatment." The top 50 videos based on our query were assessed using the DISCERN instrument. This validated algorithm discretely rates treatment-related information from excellent (≥4.5) to very poor (<1.9). Videos were also screened for ATA guideline inclusion. Descriptive statistics were used for cohort characterization. Univariate and multivariate linear regressions characterized factors associated with DISCERN scores. Significance was set at P < .05. The videos featured 57,513.43 views (SD = 162,579.25), 1054.70 likes (SD = 2329.77), and 168.80 comments (SD = 292.97). Most were patient education (52%) or patient experience (24%). A minority (40%) were made by thyroid specialists (endocrinologists, endocrine surgeons, or otolaryngologists). Under half did not mention all 3 treatment modalities (44%), and 54% did not mention any ATA recommendations. Overall, videos displayed poor reliability (mean = 2.26, SD = 0.67), treatment information quality (mean = 2.29, SD = 0.75), and overall video quality (mean = 2.47, SD = 1.07). Physician videos were associated with lower likes, views, and comments (P < .001) but higher DISCERN reliability (P = .015) and overall score (P = .019). Longer videos (P = .015), patient accounts (P = .013), and patient experience (P = .002) were associated with lower scores. The most available GD treatment content on YouTube varies significantly in the quality of medical information. This may contribute to suboptimal disease understanding, especially for patients highly engaged with online health information sources.

Man Versus Machine: Can Artificial Intelligence Distinguish Between the Quality of Online Health Information on Social Media?

BACKGROUND: ChatGPT is a popular artificial intelligence service offered by OpenAI. It’s utility in assessing quality of online medical information has been investigated by Golan and Ripps et. al. In July 2023, ultimately concluding that ChatGPT was unable to discern poor from high quality information. However, this study focused on information from websites generated from a Google search. To the author’s knowledge, there has been no evaluation of the ability of ChatGPT to stratify the quality of online medical information from social media videos. Two previous studies, by Dhami and Wescott et. al. In May 2023 and Rehman et. al. In 2021, identified differences via manual calculation of DISCERN scores between physician and non-physician Tik-Tok videos on the topic of hidradenitis suppurativa. This study aims to identify whether ChatGPT can distinguish between physician and non-physician health videos pertaining to hidradenitis suppurativa on Tik-Tok via DISCERN score. METHODS: A similar methodology to Dhami and Wescott et. al. was used in an attempt to capture the same population of videos. A new Tik-Tok account was created to eliminate any previous search bias. The term “#hidradenitissuppurativa” was searched in July, 2023. Videos were filtered by “most-liked.” Inclusion criteria consisted of videos posted prior to October 25, 2022, that did not satisfy the exclusion criteria. Exclusion criteria consisted of videos in a language other than English, duplicated videos, videos that did not offer health advice, or videos deemed irrelevant to hidradenitis suppurativa by the author. Videos included were transcribed by the website TokScript (https://script.tokaudit.io) or by the author if transcripts were not available by that method. ChatGPT was prompted to evaluate transcripts based on the DISCERN criteria, which were copied directly from the Discern Instrument website “Rating this question” pop-up. Each of the 16 question criteria were submitted to ChatGPT separately. Transcriptions were provided to ChatGPT and a DISCERN score was generated. A Shapiro-Wilk test was performed on the sample of DISCERN scores to determine normality. A Mann-Whitney U test was performed between physician and non-physician DISCERN scores. RESULTS: Of 377 Tik Tok videos returned with the search, 43 satisfied inclusion criteria (11.4%). Of these, 19 (44.2%) were produced by a physician and 24 (55.8%) by a non-physician. Aggregate DISCERN scores were not normally distributed (Shapiro-Wilk p &lt; 0.0002). Physician produced videos had a mean DISCERN score of 41.87 with an interquartile range of 26-57. Mean DISCERN score for non-physician produced videos was 24.56 with an interquartile range of 16-30.5. Mann-Whitney U test returned a p &lt; 0.0006. CONCLUSION: This study suggests that ChatGPT may have the capacity to distinguish high from low quality social media videos to some extent based on the DISCERN score. More evaluation is necessary to elucidate the boundaries of utilizing ChatGPT in this fashion. This has the potential to suggest that polishing this utilization may automate this process in the future, saving valuable time for human researchers in this field.

The Impact of a Virtual Platform to Reduce Hematology-Oncology Educational Disparities Amongst Healthcare Professionals in Lower Socioeconomic Regions Globally

Introduction: Studies have shown that lower socioeconomic (LS) areas are associated with increased disparities in healthcare access and outcomes leading to increased cancer mortality. While delayed diagnosis, suboptimal access, infrastructural challenges, and finances have contributed to these inequities, access to the latest medical knowledge and advancements has been cited as an equally important contributor responsible for cancer treatment outcomes. In response, MedNews Week (MNW), a free, digital platform was developed to provide global oncology education through its programming featuring Keynote presentations from the field's global leaders. To effectively reach a wider audience of healthcare professionals (HCP), MNW formed a strategic alliance with VuMedi- a global video education platform for physicians. The primary objective of this study was to evaluate the impact of MNW's hematology-oncology programming on VuMedi's educational platform for HCP. Methods: Between September 2022 and July 2023, MNW featured 11 prominent global hematology-oncology leaders in leukemia (mean H-index = 67.82), and their Keynote presentations were prominently presented on VuMedi. This study assessed viewership, global reach, page views, tumor subtype views, unique visits, and audience occupation demographics. In conjunction with VuMedi, a descriptive analysis and mixed-methods approach were utilized to evaluate the level of engagement with the content. Results: Over this 11-month period, MNW demonstrated increased linear growth with programming generating 4,387,213 total page views and 138,288 impressions (Figure 1 a, b). The platform drew 1,750,161 new visitors and 44,982 new registrants during this time ( Figure 1a). The Leukemia Channel and Video pages demonstrated linear growth over this time with 139 channel and 891 total video page views (Figure 2 c-e). The listed videos had 746 starts with a total playing time of 8,610 minutes (Figure 1 a, g). The audience was comprised primarily of surgeons (78.57%) compared to allied healthcare professionals (14.29%) from 22 countries- 12 of which are from LS areas, including Nigeria, Sudan, Georgia, and Bangladesh, and 10 of which are from high socioeconomic areas ( Figure 1 a &amp; h). Conclusions: MNW has demonstrated an ability to attract and retain a diverse and expanding audience of physicians globally, particularly in lower-resource areas. Despite previously identified barriers such as limited internet access and high costs hindering access to quality medical information, MNW has exhibited consistent growth and success, particularly in regions facing these challenges. By hosting Keynote presentations from global leaders to HCP, MNW provides an effective and pragmatic solution to address educational disparities, while impacting hematology-oncology education in leukemia on a global scale.

R3 (Resident Advocacy Project) Evaluation of a Novel COVID-19 Vaccine Consult Service

Abstract Rationale and Objectives Strengthening confidence in COVID-19 vaccination in the paediatric population promotes more children getting vaccinated, leading to fewer COVID-19 illnesses, hospitalizations and deaths. Paediatric providers play an important role in promoting dialogue on the risks and benefits of the COVID-19 vaccines for children. Caregivers who experience hesitancy toward the vaccines may lack access to quality medical information. Developed to close this gap, the SickKids COVID-19 Vaccine Consult Service (VCS) is a nurse-led, telephone consultation service for residents of Ontario with questions/concerns about COVID-19 vaccination for children. The VCS aims to address vaccine hesitancy and enhance vaccine uptake, and ensures that children and caregivers have equitable access to timely and reliable information regarding COVID-19 vaccines in a safe and judgement-free environment. The VCS has escalation pathways in place should further discussion with a physician be required. Appointments can be booked without a referral and discussions are held by telephone. The VCS supports families whose children require specialized accommodation for vaccination with appointments at SickKids and throughout Ontario. Project Description A retrospective review of telephone consults received by the VCS between October 1, 2021- April 1, 2022 was undertaken to evaluate the service utilizing descriptive methodology. Information was extracted from databases stored in RedCap/EPIC and was maintained by staff conducting the calls. Inductive Thematic Analysis was utilized to analyze qualitative data and codes were generated to inform themes. Feedback formation was collected via exit surveys and included details about who was calling, reasons for calling, if concerns had been addressed, and intention to vaccinate. Outcomes Parents were the majority of callers (96%) and 4% were patients themselves, grandparents or other. A total of 99% of the calls included general questions about the vaccine, 36% of calls were questions about children with underlying medical conditions (cardiac conditions being the most common), and 12% were allergy-related questions. 9% of calls required a consult with MD/NP. 83% of questions/concerns were resolved following the call. Further, the majority of those surveyed following the appointment expressed intent for vaccination. Safety and development, side effects, and doses/intervals were the most common reason callers described hesitancy with proceeding with vaccination. Discussion/Future Directions Providing caregivers with timely and reliable information regarding COVID-19 vaccines through the VCS has proven to be effective in reducing vaccine hesitancy and increasing vaccine uptake in the paediatric community. In the future, the VCS model can be utilized to provide education to caregivers on other routine childhood vaccines.

Publisher Accreditation Fails to Reflect Higher Content Quality in Liver Transplant YouTube Videos

IntroductionIn 2021, the National Academy of Medicine began collaborating with YouTube to highlight high-quality channels for patients seeking medical information. This study evaluates whether YouTube videos from accredited publishers are useful for patients searching for information regarding liver transplantation. MethodsAfter searching “Liver Transplant” on YouTube, the first 100 results under 10 min long with English text or audio were transcribed. The Flesch–Kincaid grade level was used to quantify reading grade level. Viewership metrics and the accreditation status of the video publisher were identified. The DISCERN score was used to grade the quality of medical information. We adapted an informed consent curriculum for surgical interns to create an eight-point content metric that we coined the “Anderson-Lau score”. Higher scores indicated higher content quality. Statistical significance was calculated using Wilcoxon rank-sum and chi-squared tests with a significance level of P = 0.05. ResultsOf the 100 videos assessed, 37 met the average American reading level (8th grade) and none met the reading level of the average Medicare patient (5th grade). The median Flesch–Kincaid grade level was 9th grade. The median content (“Anderson-Lau”) and quality (DISCERN) scores were 2/8 and 1/5, respectively. While 56% of videos mentioned operative indications and benefits, under 25% mentioned operative steps, risks, alternatives, and postoperative expectations. A total of 75 videos were from accredited publishers, and there was no significant difference between the quality of videos from accredited and unaccredited publishers. ConclusionsVideos made by accredited sources regarding liver transplantation were not of higher educational quality or content. More informative educational materials are needed to advise patients about liver transplantation, help them understand the procedure, and to supplement discussions with their transplant team.

ChatGPT's Ability to Assess Quality and Readability of Online Medical Information: Evidence From a Cross-Sectional Study.

Introduction Artificial Intelligence (AI) platforms have gained widespread attention for their distinct ability to generate automated responses to various prompts. However, its role in assessing the quality and readability of a provided text remains unclear. Thus, the purpose of this study is to evaluate the proficiency of the conversational generative pre-trained transformer (ChatGPT) in utilizing the DISCERN tool to evaluate the quality of online content regarding shock wave therapy for erectile dysfunction. Methods Websites were generated using a Google search of "shock wave therapy for erectile dysfunction" with location filters disabled. Readability was analyzed using Readable software (Readable.com, Horsham, United Kingdom). Quality was assessed independently by three reviewers using the DISCERN tool. The same plain text files collected were inputted into ChatGPT to determine whether they produced comparable metrics for readability and quality. Results The study results revealed a notable disparity between ChatGPT's readability assessment and that obtained from a reliable tool, Readable.com (p<0.05). This indicates a lack of alignment between ChatGPT's algorithm and that of established tools, such as Readable.com. Similarly, the DISCERN score generated by ChatGPT differed significantly from the scores generated manually by human evaluators (p<0.05), suggesting that ChatGPT may not be capable of accurately identifying poor-quality information sources regarding shock wave therapy as a treatment for erectile dysfunction. Conclusion ChatGPT's evaluation of the quality and readability of online text regarding shockwave therapy for erectile dysfunction differs from that of human raters and trusted tools. Therefore, ChatGPT's current capabilities were not sufficient for reliably assessing the quality and readability of textual content. Further research is needed to elucidate the role of AI in the objective evaluation of online medical content in other fields. Continued development in AI and incorporation of tools such as DISCERN into AI software may enhance the way patients navigate the web in search of high-quality medical content in the future.

Reliability of Medical Information Provided by ChatGPT: Assessment Against Clinical Guidelines and Patient Information Quality Instrument.

ChatGPT-4 is the latest release of a novel artificial intelligence (AI) chatbot able to answer freely formulated and complex questions. In the near future, ChatGPT could become the new standard for health care professionals and patients to access medical information. However, little is known about the quality of medical information provided by the AI. We aimed to assess the reliability of medical information provided by ChatGPT. Medical information provided by ChatGPT-4 on the 5 hepato-pancreatico-biliary (HPB) conditions with the highest global disease burden was measured with the Ensuring Quality Information for Patients (EQIP) tool. The EQIP tool is used to measure the quality of internet-available information and consists of 36 items that are divided into 3 subsections. In addition, 5 guideline recommendations per analyzed condition were rephrased as questions and input to ChatGPT, and agreement between the guidelines and the AI answer was measured by 2 authors independently. All queries were repeated 3 times to measure the internal consistency of ChatGPT. Five conditions were identified (gallstone disease, pancreatitis, liver cirrhosis, pancreatic cancer, and hepatocellular carcinoma). The median EQIP score across all conditions was 16 (IQR 14.5-18) for the total of 36 items. Divided by subsection, median scores for content, identification, and structure data were 10 (IQR 9.5-12.5), 1 (IQR 1-1), and 4 (IQR 4-5), respectively. Agreement between guideline recommendations and answers provided by ChatGPT was 60% (15/25). Interrater agreement as measured by the Fleiss κ was 0.78 (P<.001), indicating substantial agreement. Internal consistency of the answers provided by ChatGPT was 100%. ChatGPT provides medical information of comparable quality to available static internet information. Although currently of limited quality, large language models could become the future standard for patients and health care professionals to gather medical information.

Understanding medical service quality, system quality and information quality of Tele-Health for sustainable development in the Indian context

PurposeThe purpose of the study is to comprehend medical service quality, information quality and system quality toward actual use of Tele-Health in rural India. The study further validates the impact of the actual use of Tele-Health on sustainable development, thus providing implications to improve upon the Tele-Health penetration in India.Design/methodology/approachData was collected from 326 healthcare practitioners practicing Tele-Health in North Indian states and Structural Equation Modeling was applied to validate the conceptual framework.FindingsThe results indicated that medical service quality, information quality and system quality influence Tele-Health behavioral intentions which in turn impact actual use and sustainable development. This research draws upon a conceptual framework to deepen our understanding of Tele-Health by providing an all-inclusive overview.Originality/valueThe massive topography of India with a prime rural populace instills the need for timely healthcare facilities. Tele-Health is a solution to all these problems but is at a nascent stage. Therefore, there is a vital need to study the factors which improve the penetration of Tele-Health in the Indian context. The model that emerged from the study may be validated by other Indian sub-continental countries so that Tele-Health may be implemented hassle-free.

A Web Tool to Help Counter the Spread of Misinformation and Fake News: Pre-Post Study Among Medical Students to Increase Digital Health Literacy

The COVID-19 pandemic was accompanied by the spread of uncontrolled health information and fake news, which also quickly became an infodemic. Emergency communication is a challenge for public health institutions to engage the public during disease outbreaks. Health professionals need a high level of digital health literacy (DHL) to cope with difficulties; therefore, efforts should be made to address this issue starting from undergraduate medical students. The aim of this study was to investigate the DHL skills of Italian medical students and the effectiveness of an informatics course offered by the University of Florence (Italy). This course focuses on assessing the quality of medical information using the "dottoremaeveroche" (DMEVC) web resource offered by the Italian National Federation of Orders of Surgeons and Dentists, and on health information management. A pre-post study was conducted at the University of Florence between November and December 2020. First-year medical students participated in a web-based survey before and after attending the informatics course. The DHL level was self-assessed using the eHealth Literacy Scale for Italy (IT-eHEALS) tool and questions about the features and quality of the resources. All responses were rated on a 5-point Likert scale. Change in the perception of skills was assessed using the Wilcoxon test. A total of 341 students participated in the survey at the beginning of the informatics course (women: n=211, 61.9%; mean age 19.8, SD 2.0) and 217 of them (64.2%) completed the survey at the end of the course. At the first assessment, the DHL level was moderate, with a mean total score of the IT-eHEALS of 2.9 (SD 0.9). Students felt confident about finding health-related information on the internet (mean score of 3.4, SD 1.1), whereas they doubted the usefulness of the information they received (mean score of 2.0, SD 1.0). All scores improved significantly in the second assessment. The overall mean score of the IT-eHEALS significantly increased (P<.001) to 4.2 (SD 0.6). The item with the highest score related to recognizing the quality of health information (mean score of 4.5, SD 0.7), whereas confidence in the practical application of the information received remained the lowest (mean of 3.7, SD 1.1) despite improvement. Almost all students (94.5%) valued the DMEVC as an educational tool. The DMEVC tool was effective in improving medical students' DHL skills. Effective tools and resources such as the DMEVC website should be used in public health communication to facilitate access to validated evidence and understanding of health recommendations.

The Quality of Medical Records and Health Information in Primary Health Care Units in Hadiya Zone, Ethiopia

Health information is an essential factor of public health performance depends on the effectiveness of documentation use for routine and evidence based decisions. Medical record is the chronological, organized and comprehensive documentation of services delivered by service providers to the patients or clients. On the other hand, the quality of medical records (MRs) is rarely evaluated and quality dimensions of medical record are not well assessed particularly in the study area as well as in Ethiopia. The study was assessed the quality of medical record in public health care units of Soro district, Hadiya Zone, Southern Ethiopia. Facility based cross- sectional study design supplemented with qualitative inquiry was conducted among randomly selected four public health care units (PHCU) in Soro district, Hadiya zone. Data on the quality of medical records were collected by reviewing document using checklist, while qualitative data for triangulation were obtained by interviewing key informants from the health center and to describe possible reasons for poor medical record quality. The data were entered in to Epi-data version 3.1, exported, and analyzed by SPSS version 22.0. A total of 384 medical records were reviewed from one-year medical records of four public health care units with 98% retrieval rate. Among the dimensions of MRs quality, none of health centers had enough facility for medical record quality and no auditing of medical record document as of the standard. Majority of medical records had incomplete administrative, clinical and legal components as of health centers standard of the country. The studied PHCUs are not fulfilled the national medical record management requirements to run medical record system of health centers since the Ethiopian PHCUs standard set the medical record personnel to be a health information technician and a minimum of three in numbers. Thus, medical record service providers should be trained and necessary supplies should be equipped in all health center.

TikTok as a source of information regarding premature ejaculation: a qualitative assessment.

Patients are increasingly looking to social media platforms for medical information. In this study we aimed to evaluate the quality of information regarding premature ejaculation (PE) on TikTok. The term "premature ejaculation" was searched on TikTok on a single day in May 2022. Videos were sorted by 3 reviewers as reliable or unreliable based on the accuracy of video content. Relevant user metrics were collected for each video, including the numbers of likes, shares, and followers, and the video length, source of upload, and speaker type. The quality of information was objectified with 2 validated tools, with mean scores obtained from the 3 reviewers, the Patient Education Materials Assessment Tool (PEMAT) and the 5-point modified DISCERN instrument. Outcomes were video reliability categorization, video and user metrics as described above, and video quality as quantified by PEMAT and DISCERN scores. Eight videos were categorized as reliable and 32 videos were categorized as unreliable. The mean number of "likes" per video was higher in the reliable than in the unreliable group (1238 vs 126, P < .018). Accounts posting reliable videos had higher mean numbers of followers than those posting unreliable videos (55 050 vs 12 042, P = .025). The majority of unreliable videos (75%) vs reliable videos (12.5%) were posted by self-identified patients or individual users, whereas 62.5% of reliable videos vs versus 6.3% of unreliable videos were posted by individual physicians or physician groups. Few videos overall mentioned PE definition, indications for PE treatment, types of treatment, or value of psychological intervention (12.5%, 15%, 22.5%, and 5.0% of videos, respectively). Video length and number of shares did not differ between groups. Reliable videos had higher PEMAT (73.0 vs 45.1, P < .001) and DISCERN (2.7 vs 0, P < .001.) scores. There exists a critical need for enhanced quality of medical information on social media platforms in hopes of encouraging patients with impaired sexual function to seek appropriate medical care. Strengths of this study include the objective use of validated quality assessment tools and a focus on TikTok as an emerging social media platform. Limitations include large numbers of excluded videos. The quality of available information regarding PE on TikTok is low, with a significant percentage of videos on this topic fraught with inaccuracies. Given TikTok's prominence as a social media platform primarily geared toward younger audiences, we emphasize the need for improvement in the quality of information available regarding PE and its management.

Google Trends™ and Quality of Information Analyses of Google™ Searches Pertaining to Concussion.

Sports-related concussions occur with high incidence in the United States. Google Trends™ (GT) analyses indicate changes of public interest in a topic over time, and can be correlated with incidence of health events such as concussion. Internet searches represent a primary means of patient education for many health topics, including concussion; however, the quality of medical information yielded by internet searches is variable and frequently of an inappropriate reading level. This study therefore aims to describe GT over time and evaluate the quality and readability of information produced by Google™ searches of the term "concussion." We identified a strong negative correlation from 2009 to 2016 between GT scores and total number of American high school football participants (R 2 = 0.8553) and participants per school (R 2 = 0.9533). Between 2004 and 2020, the monthly GT popularity score were variable (p = 3.193E-08), with September having the greatest scores, correlating with the height of American tackle football season. Applying five validated quality assessment scoring systems at two time points, it was confirmed that different sources yielded varying quality of information. Academic and non-profit healthcare sources demonstrated the highest quality metrics across two time points. There was significant variability of scores among the different scoring systems, however. The majority of searches at both time points yielded information that was rated as "fair" to "poor" in quality. Applying six readability tests, we revealed that only a single commercial website offered information written at or below the American Medical Association- recommended 6th-grade level for healthcare information. In summary, GT data analyses suggest that searches correlate with the American tackle football season and increased between 2009 and 2016, given that public interest in concussion increased and annual participation in football decreased. The quality of information yielded by Google™ searches and readability are inadequate, indicating the need for significant improvement.

Information on Recent Monkeypox Outbreak: Systematic Search and Content Analysis of YouTube Videos

ObjectiveTo examine the availability, characteristics, content, reliability, and quality of YouTube videos on the recent monkeypox outbreak. MethodsA systematic search was carried out on YouTube using the keywords “Monkeypox” on May 31, 2022. Video-related metrics were entered into the EpiCollect5 application simultaneously while watching the videos. A total of 599 videos were taken for final analysis. On the basis of content, the videos were categorized as useful, misleading, and news updates. Standardized tools were utilized to assess the content, quality, and reliability of the useful videos. ResultsVideos attracted cumulative 16,825,261 views and 591,932 likes. A total of 272 (45.4%) videos were categorized as useful, 122 (20.4%) videos as misleading, and 205 (34.2%) videos as news reports, with a 0.96 kappa coefficient of agreement between the 2 accessors. Mass media contributed to approximately 69% of the total videos. The mean Global Quality Scale, medical information and content index, and modified DISCERN scores were 2.8±0.7, 2.2±0.8, and 3.07±0.7 respectively. ConclusionThe present study found that nearly half of the selected videos on recent monkeypox outbreaks contained useful information. Health care professionals and health-related organizations should be encouraged to disseminate scientifically correct information during public health emergencies. One-fifth of the videos contained misleading information, which may be due to a lack of quality assessment and peer review processes. Hence, strict caution and strategies for proper verification procedures should be exercised before uploading any health-related video on this platform.