Incorrect Answers Research Articles

ChatGPT as a prospective undergraduate and medical school student.

This article reports the results of an experiment conducted with ChatGPT to see how its performance compares to human performance on tests that require specific knowledge and skills, such as university admission tests. We chose a general undergraduate admission test and two tests for admission to biomedical programs: the Scholastic Assessment Test (SAT), the Cambridge BioMedical Admission Test (BMAT), and the Italian Medical School Admission Test (IMSAT). In particular, we looked closely at the difference in performance between ChatGPT-4 and its predecessor, ChatGPT-3.5, to assess its evolution. The performance of ChatGPT-4 showed a significant improvement over ChatGPT-3.5 and, compared to real students, was on average within the top 10% in the SAT test, while the score in the IMSAT test granted admission to the two highest ranked Italian medical schools. In addition to the performance analysis, we provide a qualitative analysis of incorrect answers and a classification of three different types of logical and computational errors made by ChatGPT-4, which reveal important weaknesses of the model. This provides insight into the skills needed to use these models effectively despite their weaknesses, and also suggests possible applications of our analysis in the field of education.

Knowledge of newborns’ mothers on breastfeeding during covid-19, hiv and tuberculosis infection attendee in selected hospitals

Background: Two-thirds of the HIV mothers planned to breastfeed their infants while one-third opted for formula feeding; same as during Tuberculosis or COVID-19 infection. What would be the mother do during breastfeeding? Whether the mother could breastfeed their newborn or not. For that it is necessary to assess the level of knowledge of newborns mother on breastfeeding during COVID-19, HIV and Tuberculosis. Methods: It was a descriptive type of cross sectional study was conducted at Maternal and Child Health Training Institute, Azimpur, Dhaka from 1st January to 31st December 2021. Mother of newborns who are willing to participate were conveniently selected. Around 289 sample was selected. Pre tested semi structured questionnaire was used to collect the data. Level of knowledge was measured by knowledge score. Each question of knowledge carries one (01) mark. Each correct answer had 1 mark and incorrect answer had 0 mark. 1-11 score (<60%) was categorized as fair knowledge; 12-15 score (60-79%) was categorized as good knowledge and 16-20% score (80% and above) had very good knowledge. Results: Age and knowledge of whether TB-positive mothers may breastfeed have an association (2=4.488, p = 0.034); age and knowledge of whether HIV-positive mothers can breastfeed have an association (2=9.632, p = 0.002). There is a correlation between occupation and knowledge of whether mothers who have COVID 19 positive status can breastfeed (5.044; p=0.046), as well as a correlation between occupation and knowledge of whether mothers who have TB positive status (9.365; p=0.002). Family income has a correlation with knowledge of whether mothers who are COVID 19 positive, HIV positive, or TB positive can breastfeed (2=17.695, p = 0.0001), COVID 19 positive mothers have a correlation with knowledge of whether they can breastfeed (2=7.511, p = 0.006), and HIV positive mothers have a correlation with knowledge of whether they can breastfeed (2=8.675, p = 0.003). Education level and awareness of whether COVID 19 positive mothers can breastfeed are correlated (2=10.315, p = 0.001), as are knowledge of whether TB positive mothers can breastfeed (2=10.880, p = 0.001). Conclusion: Develop targeted educational programs for mothers based on their demographic characteristics, such as age, occupation, and education level. JOPSOM 2023; 42(1):40-47

Designing Semantic Tutoring Maps for Enhancing Peer Learning through Lesson Preparation

Advancements in Natural Language Processing technology present significant opportunities to enhance student learning through peer tutoring. While there is substantial research on the effectiveness of peer tutoring in learning science, few studies focus on the lesson preparation stage of peer tutoring. This study utilizes Semantic Web technologies to organize tutoring maps, aiming to develop an innovative learning artifact to address this research gap. Adopting a design-based research methodology, this study first provides a literature review regarding the application of semantic technology in education and peer tutoring preparation. Subsequently, a tutoring map application based on a semantic database is developed, encompassing the organization and storage of tutoring lesson plans using Resource Description Framework (RDF) technology, and the querying and feedback on lesson plan quality based on SPARQL Protocol and RDF Query Language (SPARQL) technology. This paper describes the theoretical basis and design rationale by which student tutors’ learning is improved through structured tutoring preparation. This includes enhancing cognitive and meta-cognitive abilities in designing learning activities, raising questions, anticipating answers, and providing guidance responses to incorrect answers. The paper concludes with recommendations for educational researchers and practitioners. Future research will focus on empirical studies of peer tutoring in scientific fields to verify the validity and adaptability of the design, further facilitating peer learning by tutoring and collective knowledge building.

Universal Design Element Utilization and Mathematics Performance: Implications for Diverse Student Populations

This empirical research study investigates the relationship between the utilization of Universal Design (UD) elements and math performance among eighth graders. We analyzed 2017 National Assessment of Educational Progress process data using Poisson Generalized Linear Mixed-Effects Models to examine how the frequency of UD element usage varies across diverse student demographics. Our findings reveal a great divide: lower achievers, students with disabilities, those from lower socioeconomic backgrounds, English Language Learners, and African American students predominantly use text-to-speech, whereas high achievers more frequently employ digital pencils (which allow for manual writing or drawing on the screen) and elimination capacity (which enables students to dismiss incorrect multiple-choice answers). Linear Mixed-Effects Models show that higher levels of utilization of digital pencils and elimination capacity are correlated with improved math performance across various student subgroups. However, frequent changes in color themes, intended to enhance visual differentiation, were associated with lower performance, suggesting that these might serve as distractions rather than aids for some students. We discuss the implications of the findings in terms of the design of UD elements, as well as for equity, accessibility, and math education more broadly.

Musculoskeletal Disorders Among the United Arab Emirates Healthcare Professionals: Ergonomics Knowledge and Practice Study

Background: Healthcare professionals (HCPs) are particularly vulnerable to musculoskeletal disorders (MSDs) due to job demands such as prolonged standing, patient handling, and awkward postures. Objectives: This study aimed to assess the prevalence of MSDs among HCPs, examine the association between ergonomic knowledge and practices with MSDs, and identify predictors of ergonomic knowledge. Methods: A cross-sectional study was conducted among HCPs, including doctors and nurses, working in the United Arab Emirates (UAE). The prevalence of MSDs was determined using the Nordic Questionnaire. Ergonomic knowledge and practice were assessed through a validated questionnaire covering sociodemographic and work-related factors, ten knowledge items, and fifteen practice items. A scoring system was used, where correct answers received a score of one, and incorrect answers received a score of zero. Ergonomic knowledge and practices were considered inadequate when participant scores fell below the median values of six for knowledge and eight for practice. The calculated sample size was 398, with a response rate of 96%, resulting in 380 participants. The chi-square test, binary, and multivariable logistic regression analyses were performed. Results: The twelve-month prevalence of MSDs was 90.4%, with the highest percentage reported in the lower back (56%), followed by the upper back (43.4%), neck (39.2%), shoulder (38.9%), and elbow (21.3%). Inadequate ergonomic knowledge and incorrect ergonomic practices were found in 61.8% and 36.3% of HCPs, respectively. A significant association was observed between inadequate ergonomic knowledge (P < 0.001) and practice (P = 0.035) and the presence of MSDs. Significant predictors of inadequate ergonomic knowledge included the number of years worked (OR: 1.9; 95%CI: 1.00 - 3.5, P = 0.049), the provision of ergonomics training and awareness at the workplace (OR: 3.7; 95%CI: 1.9 - 7.2, P < 0.001), and inadequate ergonomic practices (OR: 1.9; 95%CI: 1.1 - 3.0, P = 0.014). Conclusions: A very high prevalence of MSDs was found among HCPs, with the lower back being the most commonly affected site. Significant predictors of inadequate ergonomic knowledge included the number of years worked, the availability of ergonomics training at the workplace, and inadequate ergonomic practices. This study underscores the need for ergonomic training programs and ergonomically friendly workplaces to equip HCPs with the knowledge and practices necessary to prevent MSDs.

B-learning in human anatomy: Comparative analysis of academic achievement between face-to-face and e-learning modalities

IntroductionIn recent years, modern technologies have become established in most educational fields. Thus, e-learning tends to be an integral part of the learner-centered learning process, with the teacher acting as a facilitator. However, the methodologies used to study the impact of e-learning have been varied and imprecise, making comparison and meta-analysis difficult. This study attempts to overcome these obstacles with a large and homogeneous sample to compare (1) the academic outcomes obtained with face-to-face and e-learning in a blended module of human anatomy and (2) the response attempts (response index) of each student in answering questions specific to each learning modality. Material and methodsThe results of the multiple-choice exams under study were collected. All students (n=1160) were from four consecutive academic years following the same teaching program with a b-learning method: 13 topics were presented face to face by the same lecturers, and six via e-learning with the same online resources. Two variables were compared: (1) the academic grade, based on the score for correct answers and the penalty for incorrect ones, and (2) the response index, based on the number of correct and incorrect answers. Results(1) 73.45 % of the examinees passed the test. In the sample as a whole, results were better in face-to-face than in e-learning. In the quartiles ordered by overall academic performance, this superiority was limited to the top half of the higher-performing students. In contrast, lower-scoring students performed better in e-learning. However, these differences were modest (≤ 0.54 points). (2) In proportion, the questions on topics learned face-to-face were the most frequently answered. A strong correlation was observed between the variables in the whole sample and the students with the highest academic scores (first quartile) on the global exam and the questions on topics learned in each modality. In the remaining quartiles, the correlation was also strong in the e-learning content. Conclusions(1) Both modalities included in b-learning are academically effective. (2) Proportionally, students take more risks when answering content questions learned in face-to-face classes, and there is a strong correlation between response attempts and academic grades, especially, on the brightest exams and e-learning content.

Medical residents' knowledge, attitudes and practices regarding antibiotics, antimicrobial stewardship and multidrug-resistant bacteria: a cross-sectional study in a major university in Iran.

Antimicrobial resistance (AMR) is one of the biggest threats to global public health systems. This study aimed to assess the knowledge, attitudes and practice about AMR, antimicrobial stewardship programs (ASPs) and multidrug-resistant (MDR) bacteria. A web-based questionnaire survey was conducted among the residents of Isfahan University of Medical Sciences from May to November 2023. Data analysis was done using SPSS version 24.0 software. Overall, 400 out of 450 medical residents responded to the questionnaire, giving a response rate of 88.9%. The participants' ages ranged from 26 to 54 years, and the majority were female (227/400 56.8%). Average scores for knowledge, attitudes, and practices were 53.70 ± 15.88, 36.97 ± 5.89 and 24.69 ± 4.24, respectively. In terms of knowledge, only 26.8% had heard the term "ASPs" and knew what it was. Most incorrect answers appeared to the treatment of infection caused by MDR bacteria including ESBL-producing Escherichia coli (27.8%) and carbapenem-resistant Klebsiella pneumoniae (30.8%), as well as the atypical bacteria (45.5%). Approximately, 50 and 71.7% said they had received no specific training in the fields of microbiological sampling methods and the appropriate time to prescribe antibiotics, respectively. Surprisingly, regarding practice, 81.8% of the respondents stated that antibiotics are used to treat flu or the common cold. Residents considered their training on important issues including ASPs, MDR bacteria and the spectrum of antibiotics insufficient. This result highlights the need for targeted training interventions about antibiotic prescription in the curriculum at the university with more emphasis on ASPs to limit the development of resistance.

Exploring Middle School Students' Challenges in Mathematical Literacy: A Study on AKM Problem-Solving

Mathematical literacy is the ability to formulate, apply, and interpret everyday problems. The Minimum Competency Assessment (AKM) program in Indonesia is designed to evaluate these essential skills, similar to the PISA framework. Assessing mathematical literacy through the AKM ensures that students meet basic competency standards, helping to improve overall educational quality and better preparing students for global competition. This research reveals the characteristics of students' mathematical literacy when solving AKM problems. The study was conducted at a junior high school in Surakarta, Indonesia. The research stages included giving AKM problems, observing students' problem-solving processes through think-aloud methods, conducting interviews for confirmation, reducing data, coding data, analyzing data, and drawing conclusions. The data used included students' answers, observation sheets, and interview transcripts. When formulating problems, students often struggle to accurately understand the issues at hand. This leads to incomplete data during the problem-solving process, resulting in calculation errors or mistakes in applying formulas. These errors create a domino effect, leading to incorrect answers. Students' difficulties in formulating mathematical problems can potentially cause calculation errors and improper use of formulas, ultimately leading to incorrect solutions. This negatively impacts their conceptual understanding, learning motivation, and readiness to face educational challenges or enter the workforce. To address these difficulties, teachers can enhance students' conceptual understanding, promote critical thinking, implement problem-based learning, use visual aids, provide targeted support, and focus on developing mathematical literacy from an early age.

AWARENESS OF USAGE THE ARTIFICIAL SWEETENERS IN FOOD BY DIFFERENT POPULATION IN NORTH MACEDONIA

The artificial sweeteners (AS) are popular low-calorie substances used as sugar substitutes, providing strong sweetening effects without adding sugar and energy to the diet. The most popular AS are aspartame, saccharin, and acesulfame K. The aim of this study is to investigate the awareness and knowledge on the usage and health effects of the artificial sweeteners in food products in North Macedonia. Therefore, an online survey was conducted, distributed through Forms media platform on different age population, completed by 392 participants aged 12 years and above, where 69 % were at the age of 30 - 64 old. According to the obtained results, 6 % of respondents do not know what AS are, while 23 % are partially familiar with them. Moreover, 19 % of the respondents do not know how to recognize the additives on the product declarations at all. On the question if the respondents could list which artificial sweeteners they know, 43 % did not know how to answer, and 28 % gave a partially correct or incorrect answer. Also, 35 % of respondents answered that they use artificial sweeteners in order to reduce the calorie intake. Regarding the risk of consuming large amounts of AS in the diet, 15 % of respondents are not aware, and 36 % are partially aware. The results suggest that it is necessary to develop effective strategies for greater education, information and awareness of consumers, about the usage of AS in their diet, as well as about the advantages and risks of their intake.

Performance of generative pre-trained transformer-4 on the certification test for mental health management: A factorial design

This study aimed to investigate the performance of generative pre-trained transformer-4 (GPT-4) on the Certification Test for Mental Health Management and whether tuned prompts could improve its performance. This study used a 3 × 2 factorial design to examine the performance according to test difficulty (courses) and prompt conditions. We prepared 200 multiple-choice questions (600 questions overall) for each course using the Certification Test for Mental Health Management (levels I-III) and essay questions from the level I test for the previous four examinations. Two conditions were used: a simple prompt condition using the questions as prompts and tuned prompt condition using techniques to obtain better answers. GPT-4 (gpt-4-0613) was adopted and implemented using the OpenAI API. The simple prompt condition scores were 74.5, 71.5, and 64.0 for levels III, II, and I, respectively. The tuned and simple prompt condition scores had no significant differences (OR = 1.03, 95% CI; 0.65-1.62, p = 0.908). Incorrect answers were observed in the simple prompt condition because of the inability to make choices, whereas no incorrect answers were observed in the tuned prompt condition. The average score for the essay questions under the simple prompt condition was 22.5 out of 50 points (45.0%). GPT-4 had a sufficient knowledge network for occupational mental health, surpassing the criteria for levels II and III tests. For the level I test, which required the ability to describe more advanced knowledge accurately, GPT-4 did not meet the criteria. External information may be needed when using GPT-4 at this level. Although the tuned prompts did not significantly improve the performance, they were promising in avoiding unintended outputs and organizing output formats. UMIN trial registration: UMIN-CTR ID = UMIN000053582.

Implementation and Preliminary Evaluation of an Entrepreneurship, Biomedical Innovation, and Design Pathway in a School of Medicine Curriculum.

New educational curricula are emerging to train physicians to practice healthcare in the 21st century. The University of Massachusetts Chan Medical School T.H. Chan School of Medicine (UMass Chan) implemented an MD curriculum redesign in the fall of 2022 that included seven educational pathways, including Entrepreneurship, Biomedical Innovation and Design. This pathway is modeled after the I-Corps curriculum with added material regarding engineering design. This manuscript describes this pathway curriculum and provides preliminary evaluation data and learning outcomes. First-year (Class of 2027) and second-year (Class of 2026) pathway students were invited to participate in online surveys evaluating course material and their knowledge of course content. Course evaluations and self-assessments were performed on a 4 or 5 point Likert scale. The material assessment comprised of multiple-choice questions; some had four options while others had five. Simple means were calculated for each question of the self-assessment, and as an aggregate. A two-sample t-test was performed using those means to assess statistical significance. A distribution of correct and incorrect answers was generated between the pre and post survey results, and a chi-squared analysis was used to determine whether the two correct/incorrect distributions were significantly different. Initial results show that the program was well received, with 15/20 (75%) of first year students rating the experience as good or excellent and 8/10 (80%) of second year students rating the experience as good or excellent. Three lectures were provided during the Fall 2023 semester to 11 second-year students. Results of self-assessment of student comfort and understanding of engineering content significantly improved after delivery of these lectures. Objective student knowledge also significantly improved. This new pathway curriculum at UMass Chan is designed to introduce students to the principles of innovation, entrepreneurship, and technology commercialization. An element of this pathway focused on basic engineering principles provided students with baseline understandings of biomedical design, human factors, and risk/hazard analysis. Despite small sample sizes, the results show improvements in student comfort with the material and knowledge. Novel curricula have the potential to transform medical education and prepare future physicians to practice healthcare in the 21st Century.

GPT-4o vs. Human Candidates: Performance Analysis in the Polish Final Dentistry Examination.

Background This study aims to evaluate the performance of OpenAI's GPT-4o in the Polish Final Dentistry Examination (LDEK) and compare it with human candidates' results. The LDEK is a standardized test essential for dental graduates in Poland to obtain their professional license. With artificial intelligence (AI) becoming increasingly integrated into medical and dental education, it is important to assess AI's capabilities in such high-stakes examinations. Materials and methods The study was conducted from August 1 to August 15, 2024, using the Spring 2023 LDEK exam. The exam comprised 200 multiple-choice questions, each with one correct answer among five options. Questions spanned various dental disciplines, including Conservative Dentistry with Endodontics, Pediatric Dentistry, Dental Surgery, Prosthetic Dentistry, Periodontology, Orthodontics, Emergency Medicine, Bioethics and Medical Law, Medical Certification, and Public Health. The exam organizers withdrew one question. GPT-4o was tested on these questions without access to the publicly available question bank. The AI model's responses were recorded, and each answer's confidence level was assessed. Correct answers were determined based on the official key provided by the Center for Medical Education (CEM) in Łódź, Poland. Statistical analyses, including Pearson's chi-square test and the Mann-Whitney U test, were performed to evaluate the accuracy and confidence of ChatGPT's answers across different dental fields. Results GPT-4o correctly answered 141 out of 199 valid questions (70.85%) and incorrectly answered 58 (29.15%). The AI performed better in fields like Conservative Dentistry with Endodontics (71.74%) and Prosthetic Dentistry (80%) but showed lower accuracy in Pediatric Dentistry (62.07%) and Orthodontics (52.63%). A statistically significant difference was observed between ChatGPT's performance on clinical case-based questions (36.36% accuracy) and other factual questions (72.87% accuracy), with a p-value of 0.025. Confidence levels also varied significantly between correct and incorrect answers, with a p-value of 0.0208. Conclusions GPT-4o's performance in the LDEK suggests it has potential as a supplementary educational tool in dentistry. However, the AI's limited clinical reasoning abilities, especially in complex scenarios, reveal a substantial gap between AI and human expertise. While ChatGPT demonstrates strong performance in factual recall, it cannot yet match the critical thinking and clinical judgment exhibited by human candidates.

Community Awareness and Knowledge About Precocious Puberty in Riyadh City, Saudi Arabia.

Background Puberty is the process of physical maturation where an adolescent reaches sexual maturity and becomes capable of reproduction. On average, puberty typically begins between 8 and 13 years old in females and 9 and 14 years oldin males. The traditional definition of precocious puberty is the development of secondary sexual characteristics before eight years of age in girls and nine years in boys. In this study, we aimed to assess the knowledge and awareness of Riyadh residents in Saudi Arabia about precocious puberty and its complications. Methodology A cross-sectional study was conducted to estimate the awareness and knowledge among Riyadh residents about precocious puberty and its complications; 426 participants were included in this study. Participants completed a pretested self-administered questionnaire that included sociodemographic data, knowledge about precocious puberty definition, risk factors, complications, and treatment. Results This study enrolled 426 participants. Overall, 10.1% were correct about the precocious puberty age for girls, and 8.5% knew about the precocious puberty age for boys. The overall mean knowledge score among the study sample was 5.56 (SD = 2.79), with poor knowledge being prominent and constituting 68.8%, while fair and good knowledge were 30% and 1.2%, respectively. Working in a non-medical field was the sole independent significant predictor of incorrect answers of knowledge about precocious puberty. Conclusions Awareness strategies are needed to increase awareness among the community to encourage prevention and treatment for precocious puberty as the awareness and knowledge of Riyadh residents in Saudi Arabia about precocious puberty and its complications are low.

Optimizing biomedical information retrieval with a keyword frequency-driven prompt enhancement strategy

BackgroundMining the vast pool of biomedical literature to extract accurate responses and relevant references is challenging due to the domain's interdisciplinary nature, specialized jargon, and continuous evolution. Early natural language processing (NLP) approaches often led to incorrect answers as they failed to comprehend the nuances of natural language. However, transformer models have significantly advanced the field by enabling the creation of large language models (LLMs), enhancing question-answering (QA) tasks. Despite these advances, current LLM-based solutions for specialized domains like biology and biomedicine still struggle to generate up-to-date responses while avoiding “hallucination” or generating plausible but factually incorrect responses.ResultsOur work focuses on enhancing prompts using a retrieval-augmented architecture to guide LLMs in generating meaningful responses for biomedical QA tasks. We evaluated two approaches: one relying on text embedding and vector similarity in a high-dimensional space, and our proposed method, which uses explicit signals in user queries to extract meaningful contexts. For robust evaluation, we tested these methods on 50 specific and challenging questions from diverse biomedical topics, comparing their performance against a baseline model, BM25. Retrieval performance of our method was significantly better than others, achieving a median Precision@10 of 0.95, which indicates the fraction of the top 10 retrieved chunks that are relevant. We used GPT-4, OpenAI's most advanced LLM to maximize the answer quality and manually accessed LLM-generated responses. Our method achieved a median answer quality score of 2.5, surpassing both the baseline model and the text embedding-based approach. We developed a QA bot, WeiseEule (https://github.com/wasimaftab/WeiseEule-LocalHost), which utilizes these methods for comparative analysis and also offers advanced features for review writing and identifying relevant articles for citation.ConclusionsOur findings highlight the importance of prompt enhancement methods that utilize explicit signals in user queries over traditional text embedding-based approaches to improve LLM-generated responses for specialized queries in specialized domains such as biology and biomedicine. By providing users complete control over the information fed into the LLM, our approach addresses some of the major drawbacks of existing web-based chatbots and LLM-based QA systems, including hallucinations and the generation of irrelevant or outdated responses.

Use of Language By generative AI Tools in Mathematical Problem Solving: The Case of ChatGPT

Texts generated by artificial intelligence agents have been suggested as tools supporting students’ learning. The present research analyses the language of texts generated by ChatGPT when solving mathematical problems related to the quadratic equation. We use the functional grammar theoretical framework that includes three meta-functions: the ideational meta-function, the interpersonal meta-function and the textual meta-function. The results indicated that in at least one of six problem-solving tasks ChatGPT provided a mathematically incorrect answer. The processes appearing in ChatGPT texts, aiming at developing students’ understanding of mathematical concepts, included verbal, mental, existential, relational and behavioural processes but no material processes. Specifically, ChatGPT performed a mathematically incorrect existential process. ChatGPT generally used the first plural pronoun ‘we’ when describing the processes of solving mathematical problems, while it generally used the first-person singular pronoun when taking responsibility for a specific mistake or when expressing happiness for the actions of the user. Moreover, generally the text of the solution did not include direct imperatives but used ‘let us do’. The advancement of the ChatGPT textual solution was made usually through steps like ‘first’, ‘second’, etc. The research results indicated that the way ChatGPT responded to the mathematical problems would be useful in supporting learners’ understanding of ways to solve quadratic equations, but only if the teacher critically accompanies the student in the problem-solving process. Self-study with ChatGPT could lead to or confirm students’ mathematical misconceptions.

Assessment Study of ChatGPT-3.5's Performance on the Final Polish Medical Examination: Accuracy in Answering 980 Questions.

The use of artificial intelligence (AI) in education is dynamically growing, and models such as ChatGPT show potential in enhancing medical education. In Poland, to obtain a medical diploma, candidates must pass the Medical Final Examination, which consists of 200 questions with one correct answer per question, is administered in Polish, and assesses students' comprehensive medical knowledge and readiness for clinical practice. The aim of this study was to determine how ChatGPT-3.5 handles questions included in this exam. This study considered 980 questions from five examination sessions of the Medical Final Examination conducted by the Medical Examination Center in the years 2022-2024. The analysis included the field of medicine, the difficulty index of the questions, and their type, namely theoretical versus case-study questions. The average correct answer rate achieved by ChatGPT for the five examination sessions hovered around 60% and was lower (p < 0.001) than the average score achieved by the examinees. The lowest percentage of correct answers was in hematology (42.1%), while the highest was in endocrinology (78.6%). The difficulty index of the questions showed a statistically significant correlation with the correctness of the answers (p = 0.04). Questions for which ChatGPT-3.5 provided incorrect answers had a lower (p < 0.001) percentage of correct responses. The type of questions analyzed did not significantly affect the correctness of the answers (p = 0.46). This study indicates that ChatGPT-3.5 can be an effective tool for assisting in passing the final medical exam, but the results should be interpreted cautiously. It is recommended to further verify the correctness of the answers using various AI tools.

Evaluation of the Quality of ChatGPT’s Responses to Top 20 Questions about Robotic Hip and Knee Arthroplasty: Findings, Perspectives and Critical Remarks on Healthcare Education

Robotic-assisted hip and knee arthroplasty represents significant advancements in orthopedic surgery. Artificial intelligence (AI)-driven chatbots, such as ChatGPT, could play a significant role in healthcare education. This study aims to evaluate the quality of responses provided by ChatGPT to the top 20 questions concerning robotic-assisted hip and knee arthroplasty. We have asked ChatGPT to select the top 20 questions on Google concerning robotic hip and knee arthroplasty and to provide a detailed answer to each of them. The accuracy and completeness of the information provided were examined by three orthopedic surgeons with scientific and clinical experience in hip- and knee-replacement surgery. The accuracy was assessed through a 5-point Likert scale (from 1—completely incorrect to 5—correct); the completeness through a 4-point Likert scale (from 0—comprehensiveness not assessable for completely incorrect answers to 3—exhaustive information) on two different occasions to ensure the consistency of the assessment. Our analysis reveals that ChatGPT provides a relatively high degree of accuracy; moreover, the explanations can be considered satisfying, especially for factual questions. The findings suggest that ChatGPT can serve as a valuable initial resource for general information on robotic hip and knee arthroplasty but the integration with human expertise remains essential.

Knowledge and practices of neonatal intensive care unit nurses concerning hypothermia in preterm infants: Adescriptive cross-sectional study.

To investigate the knowledge level and clinical practice of neonatal intensive care unit nurses on the whole process of hypothermia prevention in preterm infants. A polycentric descriptive cross-sectional study was conducted in 23 hospitals in Shandong province involving 254 neonatal intensive care unit nurses. An evidence-based knowledge and clinical practice questionnaire on hypothermia prevention in preterm infants and a general information questionnaire were used to collect data. SPSS and EXCEL 24.0 database were used for data statistics and analysis. Nearly one-third of nurses were unclear about the definition of mild hypothermia and heat preservation measures during labour management and resuscitation. Knowledge about the correct rate of amniotic fluid evaporation was low. The higher the hospital level and nurses' education level, professional title, work experience, and position, the higher the knowledge level. The clinical practice of nurses differed in terms of body temperature assessment tools, measurement sites, and measurement frequency for premature infants. Nurses in neonatal intensive care units need to apply evidence-based knowledge and carry out practice interventions to ensure preterm infants' safe transition from the delivery room to the neonatal intensive care unit. There have been few studies evaluating neonatal nurses' knowledge and practices with regards to the preventing hypothermia of premature infants from delivery room to neonatal intensive care unit. The study identifies the deficiencies and problems in temperature management that can be attributed to the knowledge level of nurses. The findings will help improve the current curriculum, teaching strategies, and the nurses' knowledge levels, preventing premature hypothermia in infants. This study adhered to the STROBE statement for observational studies and obtained approval (KYLL-2023LW045) from the ethics committee of The Second Hospital of Shandong University. The other 22 third- and second-level hospitals, as our alliance members, recognized our chairperson status within the alliance and were willing to join our academic activities. Mutual responsibility for ethical filing between alliance units. 254 nurses were recruited from 23 tertiary and secondary public hospitals in Shandong province. No public or patient involvement. Members of the research group used the Questionnaire Star platform to prepare the electronic questionnaire, including obtaining informed consent, taking precautions for questionnaire completion, and using the aforementioned measurement tools. The project research team contacted the head nurses of 23 neonatal intensive care units in Shandong Province. After obtaining consent from the head nurses, they were asked to send the requirements and link of the electronic questionnaire to their WeChat management group. The nurses go through each question and make a choice which has two options of 'agree' or 'disagree.' Based on their answers, nurses were evaluated as having 'mastered' or 'not mastered' each item. Each 'mastered' item (correct answer) was given a score of 4; an item 'not mastered' (incorrect answer) was given a score of 0. The total score ranged from 0 to 100.

Dealing with “Do Not Know” Responses in the Assessment of Financial Literacy: The Use of a Sample Selection Model

Financial literacy assessments typically rely on sample surveys containing sets of questions designed to gauge respondents’ comprehension of fundamental financial concepts necessary for making informed decisions. The answers to such questions, either categorical or continuous in nature, generally include a “Do not know” option. If those who choose the “Do not know” option are not a random sample of the population but exhibit peculiar characteristics, treating these observations as either incorrect responses or as missing data may distort the results regarding the determinants of financial literacy. A noteworthy case lies in the observation from survey studies that women tend to choose the “Do not know” option more frequently than men. In similar cases, treating the “Do not know” responses as incorrect answers increases the gender gap in financial literacy while treating them as missing values reduces the gap. We propose using a model with sample selection, which enables us to disentangle the inclination to answer “Do not know” from actual responses. By applying this model to a representative sample of the UK population, we do not find any systematic gender gap in financial knowledge. The study’s novel treatment of “Do not know” responses contributes valuable insights to the broader discourse on the determinants of financial literacy and the related gender-based differences.

Evaluation of Current Artificial Intelligence Programs on the Knowledge of Glaucoma.

To measure the success of three different artificial intelligence chatbots, ChatGPT, Bard, and Bing, in correctly answering questions about glaucoma types and treatment modalities and to examine their superiority over each other. Thirty-two questions about glaucoma types and treatment modalities were asked using the ChatGPT, Bard, and Bing chatbots. The correct and incorrect answers were also provided. Accuracy rates were compared. Questions asked: ChatGPT answered 56.3%, Bard 78.1%, and Bing 59.4% correctly. There was no statistically significant difference between the three artificial intelligence chatbots in the rate of correct and incorrect answers to the questions asked (p = 0.195). Artificial intelligence chatbots can be used as a tool to access accurate information regarding glaucoma types and treatment modalities. However, the information obtained is not always accurate, and care should be taken when using this information.