Examination Questions Research Articles

Performance Assessment of GPT 4.0 on the Japanese Medical Licensing Examination.

To evaluate the accuracy and parsing ability of GPT 4.0 for Japanese medical practitioner qualification examinations in a multidimensional way to investigate its response accuracy and comprehensiveness to medical knowledge. We evaluated the performance of the GPT 4.0 on Japanese Medical Licensing Examination (JMLE) questions (2021-2023). Questions are categorized by difficulty and type, with distinctions between general and clinical parts, as well as between single-choice (MCQ1) and multiple-choice (MCQ2) questions. Difficulty levels were determined on the basis of correct rates provided by the JMLE Preparatory School. The accuracy and quality of the GPT 4.0 responses were analyzed via an improved Global Qualily Scale (GQS) scores, considering both the chosen options and the accompanying analysis. Descriptive statistics and Pearson Chi-square tests were used to examine performance across exam years, question difficulty, type, and choice. GPT 4.0 ability was evaluated via the GQS, with comparisons made via the Mann-Whitney U or Kruskal-Wallis test. The correct response rate and parsing ability of the GPT4.0 to the JMLE questions reached the qualification level (80.4%). In terms of the accuracy of the GPT4.0 response to the JMLE, we found significant differences in accuracy across both difficulty levels and option types. According to the GQS scores for the GPT 4.0 responses to all the JMLE questions, the performance of the questionnaire varied according to year and choice type. GTP4.0 performs well in providing basic support in medical education and medical research, but it also needs to input a large amount of medical-related data to train its model and improve the accuracy of its medical knowledge output. Further integration of ChatGPT with the medical field could open new opportunities for medicine.

Using Generative Artificial Intelligence in Exams: A Research on KPSS with ChatGPT

ChatGPT is a language model developed by OpenAI that uses generative artificial intelligence (GenAI) algorithms for natural language processing (NLP) tasks. It is a popular Chatbot due to its impressive ability to respond consistently to users' various prompts. In this study, the Public Personnel Selection Examination (KPSS) questions, organized by ÖSYM (Turkish Student Selection and Placement Center) every year, were asked to ChatGPT, and its performance was evaluated. The evaluation was made based on fundamental text questions asked in the last three years (2021-2023) in Biology, History, Geography, Turkish Language Literature, and Social Studies in KPSS. This study also aims to investigate the accuracy and reliability of the questions asked to ChatGPT in the field of KPSS educational sciences. According to the research results, ChatGPT's KPSS knowledge and interpretation skills are significantly lower than the university graduate candidates who took the exam. Some suggestions are included for better Chatbot performance, more effective prompt engineering and ethical use of these tools.

Comparative Analysis of the Response Accuracies of Large Language Models in the Korean National Dental Hygienist Examination Across Korean and English Questions.

Large language models such as Gemini, GPT-3.5, and GPT-4 have demonstrated significant potential in the medical field. Their performance in medical licensing examinations globally has highlighted their capabilities in understanding and processing specialized medical knowledge. This study aimed to evaluate and compare the performance of Gemini, GPT-3.5, and GPT-4 in the Korean National Dental Hygienist Examination. The accuracy of answering the examination questions in both Korean and English was assessed. This study used a dataset comprising questions from the Korean National Dental Hygienist Examination over 5 years (2019-2023). A two-way analysis of variance (ANOVA) test was employed to investigate the impacts of model type and language on the accuracy of the responses. Questions were input into each model under standardized conditions, and responses were classified as correct or incorrect based on predefined criteria. GPT-4 consistently outperformed the other models, achieving the highest accuracy rates across both language versions annually. In particular, it showed superior performance in English, suggesting advancements in its training algorithms for language processing. However, all models demonstrated variable accuracies in subjects with localized characteristics, such as health and medical law. These findings indicate that GPT-4 holds significant promise for application in medical education and standardized testing, especially in English. However, the variability in performance across different subjects and languages underscores the need for ongoing improvements and the inclusion of more diverse and localized training datasets to enhance the models' effectiveness in multilingual and multicultural contexts.

Testing and Evaluation of Health Care Applications of Large Language Models

Large language models (LLMs) can assist in various health care activities, but current evaluation approaches may not adequately identify the most useful application areas. To summarize existing evaluations of LLMs in health care in terms of 5 components: (1) evaluation data type, (2) health care task, (3) natural language processing (NLP) and natural language understanding (NLU) tasks, (4) dimension of evaluation, and (5) medical specialty. A systematic search of PubMed and Web of Science was performed for studies published between January 1, 2022, and February 19, 2024. Studies evaluating 1 or more LLMs in health care. Three independent reviewers categorized studies via keyword searches based on the data used, the health care tasks, the NLP and NLU tasks, the dimensions of evaluation, and the medical specialty. Of 519 studies reviewed, published between January 1, 2022, and February 19, 2024, only 5% used real patient care data for LLM evaluation. The most common health care tasks were assessing medical knowledge such as answering medical licensing examination questions (44.5%) and making diagnoses (19.5%). Administrative tasks such as assigning billing codes (0.2%) and writing prescriptions (0.2%) were less studied. For NLP and NLU tasks, most studies focused on question answering (84.2%), while tasks such as summarization (8.9%) and conversational dialogue (3.3%) were infrequent. Almost all studies (95.4%) used accuracy as the primary dimension of evaluation; fairness, bias, and toxicity (15.8%), deployment considerations (4.6%), and calibration and uncertainty (1.2%) were infrequently measured. Finally, in terms of medical specialty area, most studies were in generic health care applications (25.6%), internal medicine (16.4%), surgery (11.4%), and ophthalmology (6.9%), with nuclear medicine (0.6%), physical medicine (0.4%), and medical genetics (0.2%) being the least represented. Existing evaluations of LLMs mostly focus on accuracy of question answering for medical examinations, without consideration of real patient care data. Dimensions such as fairness, bias, and toxicity and deployment considerations received limited attention. Future evaluations should adopt standardized applications and metrics, use clinical data, and broaden focus to include a wider range of tasks and specialties.

Exploring preservice teachers’ pedagogical content knowledge for teaching analytical geometry in multilingual classrooms

The narrative permeating higher education institutions globally is the intention to advance the quality of preservice mathematics teachers to gain better performance than mathematics school learners. In this article we report on a qualitative case study which explored the pedagogical content knowledge (PCK) of preservice teachers (PST) (n = 20) at a South African university of technology. These preservice teachers were provided with activity sheets with past examination questions in analytical geometry. Their written responses were analysed, and verification interviews were conducted. This was followed by an analysis of the preservice teachers’ lesson plans for teaching the topic to Grade 11 learners. The domains and subdomains of the Shulman’s PCK derived Mathematics Teacher’s Specialised Knowledge (MTSK) model were used as the theoretical framework. The study established that the PSTs had limited content knowledge, and knowledge for teaching analytical geometry at Grade 11 as their responses to the activities confirmed that they mostly had procedural understanding. The study was conducted in a context with multilingual PSTs who are training in multilingual classrooms.Contribution: The study will highlight the misconceptions and existing gaps in the PSTs’ knowledge for teaching analytical geometry in multilingual classrooms.

Experience of caring for patients with COVID-19 and educational achievement among Japanese resident physicians: a nationwide survey with general medicine in-training examination

BackgroundAs healthcare professional trainees, resident physicians are expected to help with COVID-19 care in various ways. Many resident physicians worldwide have cared for COVID-19 patients despite the increased risk of burnout. However, few studies have examined the experience with COVID-19 care among resident physicians and its effects on competency achievement regarding clinical basics and COVID-19 patient care.MethodThis nationwide, cross-sectional Japanese study used a clinical training environment questionnaire for resident physicians (PGY-1 and − 2) in 593 teaching hospitals during the General Medicine In-Training Examination in January 2021. The General Medicine In-Training Examination questions comprised four categories (medical interviews and professionalism; symptomatology and clinical reasoning; physical examination and clinical procedures; and disease knowledge) and a COVID-19-related question. We examined the COVID-19 care experience and its relationship with the General Medicine In-Training Examination score, adjusting for resident and hospital variables.ResultsOf the 6,049 resident physicians, 2,841 (47.0%) had no experience caring for patients with COVID-19 during 2020. Total and categorical General Medicine In-Training Examination scores were not different irrespective of the experience with COVID-19 patient care. For the COVID-19-related question, residents with experience in COVID-19 care showed a significant increase in correct response by 2.6% (95% confidence interval, 0.3–4.9%; p = 0.028).ConclusionsThe resident physicians’ COVID-19 care experience was associated with better achievement of COVID-19-related competency without reducing clinical basics. However, approximately half of the residents missed the critical experience of caring for patients during this unparalleled pandemic in Japan.

Assessment of ChatGPT-4 in Family Medicine Board Examinations Using Advanced AI Learning and Analytical Methods: Observational Study.

This research explores the capabilities of ChatGPT-4 in passing the American Board of Family Medicine (ABFM) Certification Examination. Addressing a gap in existing literature, where earlier artificial intelligence (AI) models showed limitations in medical board examinations, this study evaluates the enhanced features and potential of ChatGPT-4, especially in document analysis and information synthesis. The primary goal is to assess whether ChatGPT-4, when provided with extensive preparation resources and when using sophisticated data analysis, can achieve a score equal to or above the passing threshold for the Family Medicine Board Examinations. In this study, ChatGPT-4 was embedded in a specialized subenvironment, "AI Family Medicine Board Exam Taker," designed to closely mimic the conditions of the ABFM Certification Examination. This subenvironment enabled the AI to access and analyze a range of relevant study materials, including a primary medical textbook and supplementary web-based resources. The AI was presented with a series of ABFM-type examination questions, reflecting the breadth and complexity typical of the examination. Emphasis was placed on assessing the AI's ability to interpret and respond to these questions accurately, leveraging its advanced data processing and analysis capabilities within this controlled subenvironment. In our study, ChatGPT-4's performance was quantitatively assessed on 300 practice ABFM examination questions. The AI achieved a correct response rate of 88.67% (95% CI 85.08%-92.25%) for the Custom Robot version and 87.33% (95% CI 83.57%-91.10%) for the Regular version. Statistical analysis, including the McNemar test (P=.45), indicated no significant difference in accuracy between the 2 versions. In addition, the chi-square test for error-type distribution (P=.32) revealed no significant variation in the pattern of errors across versions. These results highlight ChatGPT-4's capacity for high-level performance and consistency in responding to complex medical examination questions under controlled conditions. The study demonstrates that ChatGPT-4, particularly when equipped with specialized preparation and when operating in a tailored subenvironment, shows promising potential in handling the intricacies of medical board examinations. While its performance is comparable with the expected standards for passing the ABFM Certification Examination, further enhancements in AI technology and tailored training methods could push these capabilities to new heights. This exploration opens avenues for integrating AI tools such as ChatGPT-4 in medical education and assessment, emphasizing the importance of continuous advancement and specialized training in medical applications of AI.

Performance of ChatGPT on Hand Surgery Board-Style Examination Questions

Background The performance of large-language models, such as ChatGPT, on medical and sub-specialty examinations has been preliminarily explored in fields such as radiology, obstetrics and gynecology, and orthopedic surgery. However, no literature assessing ChatGPT’s ability to answer hand surgery exam questions exists. This study’s purpose was to evaluate ChatGPT’s performance on hand surgery board-style examination questions. Methods All questions from the American Society for Surgery of the Hand (ASSH) Hand 100 Exam, Beginner, and Intermediate Assessment tools were entered into ChatGPT-3.5. Responses were regenerated two times to identify inconsistencies. Duplicate questions, questions with figures and/or videos, and questions that ChatGPT refused to provide a response to were excluded. ChatGPT’s correct response rate, answer modifications, and human accuracy were recorded. Results 117 questions from the 3 assessment tools were analyzed: 49 from the ASSH Hand 100, 32 from the Beginner, and 36 from the Intermediate Assessment tools. On ChatGPT’s initial attempt, 40.82% (20/49), 50.0% (16/32), 38.89% (14/36) of questions were correctly answered, respectively. Overall, ChatGPT correctly answered 50/117 (42.7%) of questions on the first try. ChatGPT excelled in topics (>60% correct) of mass/tumor, nerve, wrist, and performed poorly (<40% correct) on topics regarding anatomy/basic science/imaging, brachial plexus, congenital, elbow, tendon, and vascular disorders, trauma. On the Beginner and Intermediate Exams, humans correctly answered 56.64% and 62.73% of questions, respectively. Conclusions ChatGPT can correctly answer simpler hand surgery questions but performed poorly when compared to humans on higher-difficulty questions.

Ethnic bias and the hidden curriculum: The impact of routine inclusion of ethnicity in medical education assessment

Introduction: In 2012, the University of Otago Medical School in New Zealand amended high stakes examination questions to routinely include ethnicity. This policy change was prompted by the assessment committee, including the Māori subcommittee. This study aimed to examine the portrayal of Māori patients and to explore the impact of routine inclusion of ethnicity. Methods: A quantitative description of demographic variables of examination questions and responses (role-model answers, multiple-response preferred answer, guidelines for markers) from a pre policy (2009–2011), post policy (2012–2013) and follow-up period (2018–2019) was undertaken. A qualitative content analysis of Māori examination questions was conducted with consideration of trends over time. Results: The majority (98%) of examination questions did not report ethnicity prior to the 2012 policy, whereas around 80% of cases did post policy implementation. Three themes were identified: the non-adherent Māori patient, determinants of health and the patient assigned Māori ethnicity. Examination questions and responses that portrayed Māori as being non-adherent were more common prior to the implementation of the policy. Post policy, many questions did not require students to consider the relevance of ethnicity but presented an increase in social and cultural contextual information of the patient over time. Conclusion: Policy implementation by the assessment committee led to less racist stereotyping and “othering” of Māori patients and prompted further refinement of social and cultural determinants of health over time. Future recommendations include increasing representation of Māori as a reflection of the increased burden of disease alongside representing intersectionality using other socio-cultural indicators.

Performance of Publicly Available Large Language Models on Internal Medicine Board-style Questions.

Ongoing research attempts to benchmark large language models (LLM) against physicians' fund of knowledge by assessing LLM performance on medical examinations. No prior study has assessed LLM performance on internal medicine (IM) board examination questions. Limited data exists on how knowledge supplied to the models, derived from medical texts improves LLM performance. The performance of GPT-3.5, GPT-4.0, LaMDA and Llama 2, with and without additional model input augmentation, was assessed on 240 randomly selected IM board-style questions. Questions were sourced from the Medical Knowledge Self-Assessment Program released by the American College of Physicians with each question serving as part of the LLM prompt. When available, LLMs were accessed both through their application programming interface (API) and their corresponding chatbot. Mode inputs were augmented with Harrison's Principles of Internal Medicine using the method of Retrieval Augmented Generation. LLM-generated explanations to 25 correctly answered questions were presented in a blinded fashion alongside the MKSAP explanation to an IM board-certified physician tasked with selecting the human generated response. GPT-4.0, accessed either through Bing Chat or its API, scored 77.5-80.7% outperforming GPT-3.5, human respondents, LaMDA and Llama 2 in that order. GPT-4.0 outperformed human MKSAP users on every tested IM subject with its highest and lowest percentile scores in Infectious Disease (80th) and Rheumatology (99.7th), respectively. There is a 3.2-5.3% decrease in performance of both GPT-3.5 and GPT-4.0 when accessing the LLM through its API instead of its online chatbot. There is 4.5-7.5% increase in performance of both GPT-3.5 and GPT-4.0 accessed through their APIs after additional input augmentation. The blinded reviewer correctly identified the human generated MKSAP response in 72% of the 25-question sample set. GPT-4.0 performed best on IM board-style questions outperforming human respondents. Augmenting with domain-specific information improved performance rendering Retrieval Augmented Generation a possible technique for improving accuracy in medical examination LLM responses.

New Metrics for Analysing Multiple-choice Questions

While the ideal of constructive alignment in curriculum is well established, and the importance of evaluating learning and teaching is well known, evaluating assessments remains a complex task and gaps can arise between learning outcomes, learning activities and assessments. Our study outlines an innovative way of analysing multiple-choice question (MCQ) examinations, which reveals possible weaknesses in the examination design and gaps in the alignment of curriculum. Individual examination questions are analysed through traditional metrics of the Discrimination Index (DI) and Difficulty Index (DIFF), combined with two novel metrics called the Grade Inversion Score (GIS) and Association with Total Score (ATS). Focusing on examination marks from a data science examination from The University of Sydney, we perform two investigations. First, we identify poorly designed questions using DI, DIFF, GIS and ATS. Second, as multiple questions arise as deficient in these metrics, we explore specific areas in the curriculum where there may be misalignment of course material with the learning outcomes. Our analysis provides a simple visual way for examiners to inspect the validity of an examination design and encourages an evidence-based approach to exploring curriculum alignment using multiple-choice assessments.

Performance of ChatGPT in the In-Training Examination for Anesthesiology and Pain Medicine Residents in South Korea: Observational Study.

ChatGPT has been tested in health care, including the US Medical Licensing Examination and specialty exams, showing near-passing results. Its performance in the field of anesthesiology has been assessed using English board examination questions; however, its effectiveness in Korea remains unexplored. This study investigated the problem-solving performance of ChatGPT in the fields of anesthesiology and pain medicine in the Korean language context, highlighted advancements in artificial intelligence (AI), and explored its potential applications in medical education. We investigated the performance (number of correct answers/number of questions) of GPT-4, GPT-3.5, and CLOVA X in the fields of anesthesiology and pain medicine, using in-training examinations that have been administered to Korean anesthesiology residents over the past 5 years, with an annual composition of 100 questions. Questions containing images, diagrams, or photographs were excluded from the analysis. Furthermore, to assess the performance differences of the GPT across different languages, we conducted a comparative analysis of the GPT-4's problem-solving proficiency using both the original Korean texts and their English translations. A total of 398 questions were analyzed. GPT-4 (67.8%) demonstrated a significantly better overall performance than GPT-3.5 (37.2%) and CLOVA-X (36.7%). However, GPT-3.5 and CLOVA X did not show significant differences in their overall performance. Additionally, the GPT-4 showed superior performance on questions translated into English, indicating a language processing discrepancy (English: 75.4% vs Korean: 67.8%; difference 7.5%; 95% CI 3.1%-11.9%; P=.001). This study underscores the potential of AI tools, such as ChatGPT, in medical education and practice but emphasizes the need for cautious application and further refinement, especially in non-English medical contexts. The findings suggest that although AI advancements are promising, they require careful evaluation and development to ensure acceptable performance across diverse linguistic and professional settings.

A Global Social History of Princes, Courts and Elites

AbstractRoyals abound in global history. Kings were served at court by domestic, administrative, and military elites who connected the centre to the provinces. Contemporary observers as well as modern historians have often stressed the contrast between oriental despots and limited monarchs in the West, downplaying structural resemblances. This article moves beyond clichés commonly ascribed to East and West, and asks to what extent social practices of court life were shared across early modern Eurasia. Then it reviews the profound changes in European court life during the long eighteenth century. Can parallel reform movements be found in other parts of Eurasia? Finally, it moves from comparisons to connections, by tracing fundamental shifts in the relationships between European royals and royals across the globe from the sixteenth century to the nineteenth. This longue durée examination questions common views about European exceptionalism and corrects persistent clichés about rising middle classes and declining nobilities.

Performance of GPT-4 with Vision on Text- and Image-based ACR Diagnostic Radiology In-Training Examination Questions.

Background Recent advancements, including image processing capabilities, present new potential applications of large language models such as ChatGPT (OpenAI), a generative pretrained transformer, in radiology. However, baseline performance of ChatGPT in radiology-related tasks is understudied. Purpose To evaluate the performance of GPT-4 with vision (GPT-4V) on radiology in-training examination questions, including those with images, to gauge the model's baseline knowledge in radiology. Materials and Methods In this prospective study, conducted between September 2023 and March 2024, the September 2023 release of GPT-4V was assessed using 386 retired questions (189 image-based and 197 text-only questions) from the American College of Radiology Diagnostic Radiology In-Training Examinations. Nine question pairs were identified as duplicates; only the first instance of each duplicate was considered in ChatGPT's assessment. A subanalysis assessed the impact of different zero-shot prompts on performance. Statistical analysis included χ2 tests of independence to ascertain whether the performance of GPT-4V varied between question types or subspecialty. The McNemar test was used to evaluate performance differences between the prompts, with Benjamin-Hochberg adjustment of the P values conducted to control the false discovery rate (FDR). A P value threshold of less than.05 denoted statistical significance. Results GPT-4V correctly answered 246 (65.3%) of the 377 unique questions, with significantly higher accuracy on text-only questions (81.5%, 159 of 195) than on image-based questions (47.8%, 87 of 182) (χ2 test, P < .001). Subanalysis revealed differences between prompts on text-based questions, where chain-of-thought prompting outperformed long instruction by 6.1% (McNemar, P = .02; FDR = 0.063), basic prompting by 6.8% (P = .009, FDR = 0.044), and the original prompting style by 8.9% (P = .001, FDR = 0.014). No differences were observed between prompts on image-based questions with P values of .27 to >.99. Conclusion While GPT-4V demonstrated a level of competence in text-based questions, it showed deficits interpreting radiologic images. © RSNA, 2024 See also the editorial by Deng in this issue.

Can a large language model create acceptable dental board-style examination questions? A cross-sectional prospective study

Background/purposeNumerous studies have shown that large language models (LLMs) can score above the passing grade on various board examinations. Therefore, this study aimed to evaluate national dental board-style examination questions created by an LLM versus those created by human experts using item analysis. Materials and methodsThis study was conducted in June 2024 and included senior dental students (n = 30) who participated voluntarily. An LLM, ChatGPT 4o, was used to generate 44 national dental board-style examination questions based on textbook content. Twenty questions for the LLM set were randomly selected after removing false questions. Two experts created another set of 20 questions based on the same content and in the same style as the LLM. Participating students simultaneously answered a total of 40 questions divided into two sets using Google Forms in the classroom. The responses were analyzed to assess difficulty, discrimination index, and distractor efficiency. Statistical comparisons were performed using the Wilcoxon signed rank test or linear-by-linear association test, with a confidence level of 95%. ResultsThe response rate was 100%. The median difficulty indices of the LLM and human set were 55.00% and 50.00%, both within the range of “excellent” range. The median discrimination indices were 0.29 for the LLM set and 0.14 for the human set. Both sets had a median distractor efficiency of 80.00%. The differences in all criteria were not statistically significant (P > 0.050). ConclusionThe LLM can create national board-style examination questions of equivalent quality to those created by human experts.

Performance of trauma-trained large language models on surgical assessment questions: A new approach in resource identification

BackgroundLarge language models have successfully navigated simulated medical board examination questions. However, whether and how language models can be used in surgical education is less understood. Our study evaluates the efficacy of domain-specific large language models in curating study materials for surgical board style questions. MethodsWe developed EAST-GPT and ACS-GPT, custom large language models with domain-specific knowledge from published guidelines from the Eastern Association of the Surgery of Trauma and the American College of Surgeons Trauma Quality Programs. EAST-GPT, ACS-GPT, and an untrained GPT-4 performance were assessed trauma-related questions from Surgical Education and Self-Assessment Program (18th edition). Large language models were asked to choose answers and provide answer rationales. Rationales were assessed against an educational framework with 5 domains: accuracy, relevance, comprehensiveness, evidence-base, and clarity. ResultsNinety guidelines trained EAST-GPT and 10 trained ACS-GPT. All large language models were tested on 62 trauma questions. EAST-GPT correctly answered 76%, whereas ACS-GPT answered 68% correctly. Both models outperformed ChatGPT-4 (P < .05), which answered 45% correctly. For reasoning, EAST-GPT achieved the gratest mean scores across all 5 educational framework metrics. ACS-GPT scored lower than ChatGPT-4 in comprehensiveness and evidence-base; however, these differences were not statistically significant. ConclusionOur study presents a novel methodology in identifying test-preparation resources by training a large language model to answer board-style multiple choice questions. Both trained models outperformed ChatGPT-4, demonstrating its answers were accurate, relevant, and evidence-based. Potential implications of such AI integration into surgical education must be explored.

Performance Evaluation of the Generative Pre-trained Transformer (GPT-4) on the Family Medicine In-Training Examination.

In this study, we sought to comprehensively evaluate GPT-4 (Generative Pre-trained Transformer)'s performance on the 2022 American Board of Family Medicine's (ABFM) In-Training Examination (ITE), compared with its predecessor, GPT-3.5, and the national family residents' performance on the same examination. We utilized both quantitative and qualitative analyses. First, a quantitative analysis was employed to evaluate the model's performance metrics using zero-shot prompt (where only examination questions were provided without any additional information). After this, qualitative analysis was executed to understand the nature of the model's responses, the depth of its medical knowledge, and its ability to comprehend contextual or new information through chain-of-thoughts prompts (interactive conversation) with the model. This study demonstrated that GPT-4 made significant improvement in accuracy compared with GPT-3.5 over a 4-month interval between their respective release dates. The correct percentage with zero-shot prompt increased from 56% to 84%, which translates to a scaled score growth from 280 to 690, a 410-point increase. Most notably, further chain-of-thought investigation revealed GPT-4's ability to integrate new information and make self-correction when needed. In this study, GPT-4 has demonstrated notably high accuracy, as well as rapid reading and learning capabilities. These results are consistent with previous research indicating GPT-4's significant potential to assist in clinical decision making. Furthermore, the study highlights the essential role of physicians' critical thinking and lifelong learning skills, particularly evident through the analysis of GPT-4's incorrect responses. This emphasizes the indispensable human element in effectively implementing and using AI technologies in medical settings.

Increasing Sexual Violence Reporting and Disclosure in Higher Education Institutions: A Proposed Approach to Critically Analyze the Internal Organizational Context.

This article explores the underreporting of sexual violence (SV) in higher education, highlighting serious implications for survivors who may silently cope with its aftermath instead of accessing crucial resources. We utilize Bolman and Deal's four-frame model for organizational change to assess how internal factors within organizations may influence reporting of SV. The four frames-symbolic, structural, human resources, and political-offer a systematic analysis of the internal organizational context in higher education institutions concerning SV reporting. Our suggested approach offers concrete dimensions and probing questions for examination. Derived from a qualitative study, our recommendations align with Bolman and Deal's four-frame model, aiding in assessing the organizational environment. This approach assists stakeholders in identifying barriers/facilitators in the internal organizational context of higher education institutions, enabling effective planning for improved SV reporting/disclosure. A thorough analysis is essential for understanding factors influencing campus SV reporting. Our proposed critical analysis and recommendations serve as a starting point to identify organizational barriers/facilitators, informing the revision of SV policies and processes, including reporting. Enhanced reporting of sexual assault in higher education benefits forensic nurses and allies, like student affairs, advocacy groups, unions, SV coordinators, health centers, equity departments, human rights officers, and administration. Improved analysis of institutional and cultural contexts allows for tailored services to better meet survivors' needs. Increased reporting should lead institutions to higher service utilization, requiring careful planning for resource allocation.

An Examination of the Relationships between Pre-service Teachers’ Test Anxiety, Use of Social Networking Sites, and Cumulative Grade Point Average

This study aimed to determine situations or conditions that promote test anxiety, identify the frequent uses of social networking sites, and examine the relationship between social media use, test anxiety, and Cumulative Grade Point Average (CGPA). A test anxiety and social networking questionnaire which consisted of 49 items was administered to randomly selected pre-service teachers in a college of education at the end of the second-semester examination. A multi-stage sampling technique was employed to select 106 pre-service teachers. The data collected from the respondents was analyzed using Pearson’s Product Moment Correlation Co-efficient, mean, and frequency. In this study, it was found that taking an unannounced examination, feeling one should have done better than he/she did after an examination, waiting to see the results of an examination published, discussing with the tutor the answer one believed to be right, however, that was marked wrong and reading through the examination questions and feeling that one does not know any of the answers promote test anxiety among the pre-service teachers. The pre-service teachers used social networking sites predominantly to read news, get information regarding current social events, conduct research, share new ideas, relieve academic stress, share pictures, and view funny videos. It was found that there is a significant positive moderate correlation between test anxiety and the use of social networking sites among pre-service teachers. It was also found that, statistically, there was no relationship between test anxiety and CGPA, and CGPA and use of social networking sites.

The Accuracy of Artificial Intelligence ChatGPT in Oncology Examination Questions

The aim of this study is to assess the accuracy of Chat Generative Pretrained Transformer (ChatGPT) in response to oncology examination questions in the setting of one-shot learning. Consecutive national radiation oncology in-service multiple-choice examinations were collected and inputted into ChatGPT 4o and ChatGPT 3.5 to determine ChatGPT’s answers. ChatGPT’s answers were then compared with the answer keys to determine whether ChatGPT correctly or incorrectly answered each question and to determine if improvements in responses were seen with the newer ChatGPT version. A total of 600 consecutive questions were inputted into ChatGPT. ChatGPT 4o answered 72.2% questions correctly, whereas 3.5 answered 53.8% questions correctly. There was a significant difference in performance by question category (P < .01). ChatGPT performed poorer with respect to knowledge of landmark studies and treatment recommendations and planning. ChatGPT is a promising technology, with the latest version showing marked improvement. Although it still has limitations, with further evolution, it may be considered a reliable resource for medical training and decision making in the oncology space.