Board Examinations Research Articles

Can a large language model create acceptable dental board-style examination questions? A cross-sectional prospective study

Background/purposeNumerous studies have shown that large language models (LLMs) can score above the passing grade on various board examinations. Therefore, this study aimed to evaluate national dental board-style examination questions created by an LLM versus those created by human experts using item analysis. Materials and methodsThis study was conducted in June 2024 and included senior dental students (n = 30) who participated voluntarily. An LLM, ChatGPT 4o, was used to generate 44 national dental board-style examination questions based on textbook content. Twenty questions for the LLM set were randomly selected after removing false questions. Two experts created another set of 20 questions based on the same content and in the same style as the LLM. Participating students simultaneously answered a total of 40 questions divided into two sets using Google Forms in the classroom. The responses were analyzed to assess difficulty, discrimination index, and distractor efficiency. Statistical comparisons were performed using the Wilcoxon signed rank test or linear-by-linear association test, with a confidence level of 95%. ResultsThe response rate was 100%. The median difficulty indices of the LLM and human set were 55.00% and 50.00%, both within the range of “excellent” range. The median discrimination indices were 0.29 for the LLM set and 0.14 for the human set. Both sets had a median distractor efficiency of 80.00%. The differences in all criteria were not statistically significant (P > 0.050). ConclusionThe LLM can create national board-style examination questions of equivalent quality to those created by human experts.

Using resident performance on Step 2 to predict surgical residency success

ObjectiveTo determine whether performance on Step 2 CK can be used to predict adverse academic outcomes in surgical residency. MethodsSurgical resident data from a single institution was used. Step 2 scores of each resident were normalized against the average Step 2 CK score for each cohort's application cycle (ie, resident who scored 246 on a year with a national average of 246 would have a normed score of 1; resident with a score >246 would have a normed score >1 etc). All adverse events were classified, and logistic regression analysis was used to determine the effects of the Step 2 scores. Adverse academic outcomes were defined. ResultsNinety-two surgical residents were evaluated with 7,182 faculty evaluations reviewed. Lower Step 2 CK scores demonstrated an increased likelihood of lower American Board of Surgery In-Training Examination performance <25% (P = .011). The remaining adverse events, including passing the Qualifying Examination (P = .998), passing the Certifying Examination (P = .778), early withdrawal from the surgical residency program (P = .565), failure on mock orals during postgraduate years 4 and 5 (P = .731; P = .600), as well as selection of administrative chiefs (P = .565), all demonstrated no correlation. ConclusionEmphasis has historically been placed on board examinations like Step 2 CK to predict overall success in residency and is used as an important metric during the interview process. However, these findings demonstrate that although lower scores on Step 2 CK correlate with a lower quartile on the American Board of Surgery In-Training Examination during general surgery residency, the examination cannot predict success in all areas of surgical residency.

Scheduling surgical success: Early minimally invasive surgery rotations as a predictor of ABSITE performance

IntroductionThe American Board of Surgery In-Training Examination (ABSITE®) serves as a benchmark for assessing resident knowledge and comprehension in surgery training programs. While previous studies have examined factors such as USMLE Step 1 and Step 2 performance in ABSITE® preparation, the impact of rotational schedule on ABSITE® performance remains underexplored. MethodsWe retrospectively investigated the independent influence of rotational schedule on ABSITE® performance among interns at a single academic institution over an 11 year period. ResultsEarly exposure to minimally invasive surgery rotations and specialty rotations such as surgical oncology and vascular surgery during the winter or just before ABSITE® was associated with higher ABSITE® percentiles when controlled for USMLE Step 2 performance. ConclusionThis study underscores the significance of rotational scheduling in optimizing ABSITE® performance and suggests potential strategies for enhancing resident preparation and success on this examination.

Performance of trauma-trained large language models on surgical assessment questions: A new approach in resource identification

BackgroundLarge language models have successfully navigated simulated medical board examination questions. However, whether and how language models can be used in surgical education is less understood. Our study evaluates the efficacy of domain-specific large language models in curating study materials for surgical board style questions. MethodsWe developed EAST-GPT and ACS-GPT, custom large language models with domain-specific knowledge from published guidelines from the Eastern Association of the Surgery of Trauma and the American College of Surgeons Trauma Quality Programs. EAST-GPT, ACS-GPT, and an untrained GPT-4 performance were assessed trauma-related questions from Surgical Education and Self-Assessment Program (18th edition). Large language models were asked to choose answers and provide answer rationales. Rationales were assessed against an educational framework with 5 domains: accuracy, relevance, comprehensiveness, evidence-base, and clarity. ResultsNinety guidelines trained EAST-GPT and 10 trained ACS-GPT. All large language models were tested on 62 trauma questions. EAST-GPT correctly answered 76%, whereas ACS-GPT answered 68% correctly. Both models outperformed ChatGPT-4 (P < .05), which answered 45% correctly. For reasoning, EAST-GPT achieved the gratest mean scores across all 5 educational framework metrics. ACS-GPT scored lower than ChatGPT-4 in comprehensiveness and evidence-base; however, these differences were not statistically significant. ConclusionOur study presents a novel methodology in identifying test-preparation resources by training a large language model to answer board-style multiple choice questions. Both trained models outperformed ChatGPT-4, demonstrating its answers were accurate, relevant, and evidence-based. Potential implications of such AI integration into surgical education must be explored.

Proposed Alternative Models for Licensing Filipino Seafarers

Licensing is evidence that one has spent time and money to gain access to a class of professionals that influence the safety, health, and welfare of the public and is charged to conform to codes of ethical conduct and standards. Licensure elevates an individual above other graduates and bestows permission to be legally responsible for activities within his specialization. Licensure will likewise enhance an individual’s personal worth and value, provide a broader set of occupational options, and earn high respect from the public (Berson, 2012). A state’s law governs the merchant marine profession. To practice the profession, one must complete an academic degree in marine courses, such as a BS in Marine Transportation or a BS in Marine Engineering. Seafarers must pass the required state board-written examinations. The passing grade is determined by a body of examiners, usually called the Board of Examiners or, as in. the Philippine setting, the Professional Regulation Commission (PRC), which is designated by the law to perform the function of regulation (Geguiento, 2010, as cited by Palmejar, 2010). In the attempt to develop alternative models for acquiring professional licenses for Filipino seafarers, a descriptive exploratory study was conducted that involved seafarers, maritime education students, and graduates of maritime education programs. Schools offering maritime education in the National Capital Region, officially listed in the Commission of Higher Education- Maritime Higher Education Institution (CHED-MHEI), as well as Maritime Training Centers and Agencies, registered in the Department of Labor-Maritime Training Council (DOLE-MTC), comprised the research population from which respondents were selected employing non-probability, purposive sampling techniques, particularly maximum variation sampling. The assessment of the acceptability of the current licensing scheme ranged from neutral to slightly acceptable. Students (with CAR and on-board training and junior maritime students) generally favor Alternative Model B, as reflected in the high mean acceptability scores, while unlicensed seafarers showed interest in Alternative Models C and D, as shown in the high mean acceptability scores. Keywords: Licensing Filipino Seafarers, licensing model, maritime licensing, licensure of maritime professionals

Evaluating Artificial Intelligence Competency in Education: Performance of ChatGPT-4 in the American Registry of Radiologic Technologists (ARRT) Radiography Certification Exam

Rationale and ObjectivesThe American Registry of Radiologic Technologists (ARRT) leads the certification process with an exam comprising 200 multiple-choice questions. This study aims to evaluate ChatGPT-4's performance in responding to practice questions similar to those found in the ARRT board examination. Materials and MethodsWe used a dataset of 200 practice multiple-choice questions for the ARRT certification exam from BoardVitals. Each question was fed to ChatGPT-4 fifteen times, resulting in 3,000 observations to account for response variability. ResultsChatGPT's overall performance was 80.56%, with higher accuracy on text-based questions (86.3%) compared to image-based questions (45.6%). Response times were longer for image-based questions (18.01 seconds) than for text-based questions (13.27 seconds). Performance varied by domain: 72.6% for Safety, 70.6% for Image Production, 67.3% for Patient Care, and 53.4% for Procedures. As anticipated, performance was best on on easy questions (78.5%). ConclusionChatGPT demonstrated effective performance on the BoardVitals question bank for ARRT certification. Future studies could benefit from analyzing the correlation between BoardVitals scores and actual exam outcomes. Further development in AI, particularly in image processing and interpretation, is necessary to enhance its utility in educational settings.

Retrospective Analysis of Patients Applying to The Pediatric Neurology Outpatient Clinic For a Health Board Report

Introduction: Disability is any physical or mental impairment that makes it difficult for a person to perform certain activities and interact with the world around them.Individuals with disabilities and their families have primarily economic, psychological, and social problems. Patients apply to the health board to have easier access to other socioeconomic rights and services such as health, education, and transportation. This study aimed to examine the purposes of application, causes of childhood disability, and age distribution of pediatric neurology patients applying to the health board; and to raise awareness about the problems and delays encountered in reporting in the light of the relevant legislation. Results: Our study included 1506 patients who applied to the pediatric neurology outpatient clinic for a medical board report. The age of the patients was 8.5±4.5 years. 894(59.4%) were boys and 612(40.4%) were girls. The most common diagnoses at the outpatient clinic were retardation in cognitive development 406(27%), cerebral palsy 126(8.4%), autism 120(8%), speech delay 110(7.3%), and developmental delayn99(6.6%). In a pediatric neurology department, 487(32.3%) patients were scored in the nervous system and movement domain. 445(91.4%) patients were scored in the movement domain and 42(8.6%) patients in the nervous system domain. 237(48.7%) patients were given a "special condition requirement" score. Conclusion: Health board examinations constitute the patient groups that pediatric neurologists frequently encounter in daily practice. Approximately one-third of the patients admitted to the health board have a disability related to pediatric neurology. Therefore, all healthcare personnel, especially pediatric neurologists, have important duties.

No Strings Attached: An Examination of Board Designated Endowments in Nonprofit Organizations

No Strings Attached: An Examination of Board Designated Endowments in Nonprofit Organizations

NATIONAL EDUCATION POLICY 2020: PRACTICAL IMPEDIMENTS AND CHALLENGES

Quality education has been crucial and in demand in a rapidly changing global scenario. With the advent of new theories like constructivism in education, the academicians and the theorists believe that learners develop their understanding through experience rather than by stimulus and response which were the core concepts in the traditional behaviorist theories. National Education Policy 2020 (NEP 2020) is a document that seeks to make education equitable, inclusive and sustainable. The Policy advocates flexibility for the learners, to opt for the learning trajectories and programs, no hard separation between different domains of studies and multidisciplinary education. The policy has proposed to bring structural changes to the education system. The present paper is an attempt to foreground the challenges and hindrances to implementing and practicing a few of the structural changes and aspects. NEP 2020 states that the students will be able to choose many of the subjects and the level in which they take the Board examination- Medium level and Advanced level. The student will select different subjects for study in class 9th. This may invite more challenges relating to infrastructure, conducting examinations, recruiting teachers, etc. Emphasis on the mother tongue as a medium of instruction may be a problem for the children of people in transferable jobs, this means the private English Medium Schools will keep on flourishing. As per the guidelines the Foreign Colleges will be permitted to open campuses in India, however, there is less possibility to get admission in these colleges for the students whose medium of instruction will be regional language. This may again lead to widening the rift between the economically poor and the rich and may invite unequal learning patterns.

Pattern of occupational lung disease among industrial workers attending a medical college of Eastern India.

Occupational lung diseases (OLDs) contribute a significant proportion to the global burden of pulmonary morbidities but are grossly misdiagnosed due to the relative lack of attribution given to occupational exposures. Obstructive lung diseases are known to be associated with long-lasting disability and loss of earning capacity (LOEC) among workers in industrial setups, thus reducing nationwide productivity. In this context, the study aimed to find out the pattern of OLD and factors associated with the severity of it among patients in a tertiary care hospital. The study was a record-based secondary data analysis conducted in the Medical Records Department of a Medical College in Kolkata. A computerized database of patients attending Special Medical Board (SMB) examinations from the Department of Medical Records was utilized for data collection. A data abstraction format was constructed to collect information on pulmonary morbidity, occupational exposure, and sociodemographic and behavioral variables. Extracted data were analyzed in Microsoft Excel and Statistical Package for Social Sciences (SPSS) software. After a review of records, it was shown that 62.3% (66 out of 106 people) of the study subjects had an obstructive type of OLD, the most common being Jute Byssinosis. A negative correlation (Spearman's ρ = -0.136) was found between pulmonary function (FEV1/FVC) and LOEC (%) in the study subjects. In the multivariable logistic regression, exposure to organic dust was found to be significantly associated with worsened lung function {adjusted-Odd's Ratio (95% Confidence Interval) =3.11 (1.1-8.8), P value = 0.03}. OLD is an understated health issue, especially in an industrial diaspora of developing countries, like India. Healthcare facilities should utilize their resources properly for the advancement of medical surveillance in industries where organic dust is produced. Health education of the stakeholders regarding the consequences of OLDs and the benefits of preventive primary approaches will go a long way in alleviating the burden of disease.

Mock Board and Customs Broker Licensure Examination Performance of a Select Higher Education Institution: A Predictive-Correlational Approach

Mock Board and Customs Broker Licensure Examination Performance of a Select Higher Education Institution: A Predictive-Correlational Approach

Perception on hematology/oncology pharmacy practice and education needs in Nigeria: A national survey.

Nigeria is a low-middle income country located in sub-Saharan Africa. Although cancer burden has continued to increase over time, there is a lack of pharmacists who specialize in oncology providing pharmaceutical care. The objectives of this study were to understand the perception of hematology/oncology pharmacy practice in Nigeria as well as the unmet education needs. This is a cross-sectional survey conducted in December 2023. Pharmacists who were active members of various pharmacy organizations in Nigeria were invited to complete the survey. The survey focused on the perceptions of training needs in oncology pharmacy, current challenges and opportunities, as well as resources and support needed. Descriptive statistics were utilized for the data analysis. Of the 263 responders, the largest group fell between 25 and 34 years old (35%), over half received a Bachelor of Pharmacy as their highest degree (53%), and majority did not receive any oncology pharmacy training as part of their education (73%). Many respondents strongly agreed that pharmacists in Nigeria need specialized training to better serve cancer patients (79%). Cancer therapeutics and chemotherapy pharmacology were more prioritized compared to other training needs (mean (SD): 2.63(1.95) and 2.69(1.64), respectively). The main challenge that respondents faced was a lack of relevant skills and knowledge in oncology pharmacy (mean (SD): 2.23(1.53)). More than a fourth of the respondents were aware about the board certification process (28.8%), citing overcoming the cost barrier as the main obstacle for taking the board examination (mean (SD): 1.92 (1.38)). Respondents preferred online courses and onsite specialty training (75%) as the mode of training to address unmet needs. Despite the challenges and unmet education needs in hematology/oncology clinical practice, Nigerian pharmacists are very willing to expand their knowledge and improve their skills in patient care. The findings of this study can inform policy makers, healthcare administrators, and pharmacy educators in developing global partnerships among pharmacy organizations which employ targeted strategies like sponsorships to address such unmet needs.

Diagnostic accuracy of vision-language models on Japanese diagnostic radiology, nuclear medicine, and interventional radiology specialty board examinations.

The performance of vision-language models (VLMs) with image interpretation capabilities, such as GPT-4 omni (GPT-4o), GPT-4 vision (GPT-4V), and Claude-3, has not been compared and remains unexplored in specialized radiological fields, including nuclear medicine and interventional radiology. This study aimed to evaluate and compare the diagnostic accuracy of various VLMs, including GPT-4 + GPT-4V, GPT-4o, Claude-3 Sonnet, and Claude-3 Opus, using Japanese diagnostic radiology, nuclear medicine, and interventional radiology (JDR, JNM, and JIR, respectively) board certification tests. In total, 383 questions from the JDR test (358 images), 300 from the JNM test (92 images), and 322 from the JIR test (96 images) from 2019 to 2023 were consecutively collected. The accuracy rates of the GPT-4 + GPT-4V, GPT-4o, Claude-3 Sonnet, and Claude-3 Opus were calculated for all questions or questions with images. The accuracy rates of the VLMs were compared using McNemar's test. GPT-4o demonstrated the highest accuracy rates across all evaluations with the JDR (all questions, 49%; questions with images, 48%), JNM (all questions, 64%; questions with images, 59%), and JIR tests (all questions, 43%; questions with images, 34%), followed by Claude-3 Opus with the JDR (all questions, 40%; questions with images, 38%), JNM (all questions, 42%; questions with images, 43%), and JIR tests (all questions, 40%; questions with images, 30%). For all questions, McNemar's test showed that GPT-4o significantly outperformed the other VLMs (all P < 0.007), except for Claude-3 Opus in the JIR test. For questions with images, GPT-4o outperformed the other VLMs in the JDR and JNM tests (all P < 0.001), except Claude-3 Opus in the JNM test. The GPT-4o had the highest success rates for questions with images and all questions from the JDR, JNM, and JIR board certification tests.

Pregnancy and Parenthood Among US Surgical Residents

The ability to pursue family planning goals is integral to gender equity in any field. Procedural specialties pose occupational risks to pregnancy. As the largest procedural specialty, general surgery provides an opportunity to understand family planning, workplace support for parenthood, obstetric outcomes, and the impact of these factors on workforce well-being, gender equity, and attrition. To examine pregnancy and parenthood experiences, including mistreatment and obstetric outcomes, among a cohort of US general surgical residents. This cohort study involved a cross-sectional national survey of general surgery residents in all programs accredited by the Accreditation Council for Graduate Medical Education after the 2021 American Board of Surgery In-Training Examination. Female respondents who reported a pregnancy and male respondents whose partners were pregnant during clinical training were queried about pregnancy- and parenthood-based mistreatment, obstetric outcomes, and current well-being (burnout, thoughts of attrition, suicidality). Primary outcomes included obstetric complications and postpartum depression compared between female residents and partners of male residents. Secondary outcomes included perceptions about support for family planning, pregnancy, or parenthood; assisted reproductive technology use; pregnancy/parenthood-based mistreatment; neonatal complications; and well-being, compared between female and male residents. A total of 5692 residents from 325 US general surgery programs participated (81.2% response rate). Among them, 957 residents (16.8%) reported a pregnancy during clinical training (692/3097 [22.3%] male vs 265/2595 [10.2%] female; P < .001). Compared with male residents, female residents more frequently delayed having children because of training (1201/2568 [46.8%] females vs 1006/3072 [32.7%] males; P < .001) and experienced pregnancy/parenthood-based mistreatment (132 [58.1%] females vs 179 [30.5%] males; P < .001). Compared with partners of male residents, female residents were more likely to experience obstetric complications (odds ratio [OR], 1.42; 95% CI, 1.04-1.96) and postpartum depression (OR, 1.63; 95% CI, 1.11-2.40). Pregnancy/parenthood-based mistreatment was associated with increased burnout (OR, 2.03; 95% CI, 1.48-2.78) and thoughts of attrition (OR, 2.50; 95% CI, 1.61-3.88). Postpartum depression, whether in female residents or partners of male residents, was associated with resident burnout (OR, 1.93; 95% CI, 1.27-2.92), thoughts of attrition (OR, 2.32; 95% CI, 1.36-3.96), and suicidality (OR, 5.58; 95% CI, 2.59-11.99). This study found that pregnancy/parenthood-based mistreatment, obstetric complications, and postpartum depression were associated with female gender, likely driving gendered attrition. Systematic change is needed to protect maternal-fetal health and advance gender equity in procedural fields.

Analysis of Oncology and Radiation Therapy Representation on the National Board of Medical Examiners Official Practice Material for the United States National Standardized Medical Board Examinations.

Radiation therapy (RT) is a critical component of multidisciplinary cancer care, but has inconsistent curricular exposure. We characterize the radiation oncology (RO) content on the standardized undergraduate medical examinations by comparing its context and prevalence with other domains in oncology. National Board of Medical Examiners (NBME) self-assessments and sample questions for the United States Medical Licensing Exam (USMLE) Steps 1-3 and NBME clinical science shelf examinations were accessed (n = 3878). Questions were inductively analyzed for content pertaining to oncology and treatment modalities of RT, systemic therapy (ST), and surgical intervention (SI). Questions were coded using USMLE Physician Tasks/Competencies and thematic analysis. Descriptive statistics and analyses using the Kruskal-Wallis test are reported. A total of 337 questions (8.6%) within the USMLE and shelf exams included oncology content, with 101 questions (2.6%) referencing at least one cancer treatment modality (n = 35 RT, 45 ST, 57 SI). Treatment questions were more common on USMLE Step 2 CK (n = 35/101, 32%) compared to Step 1 (n = 23/101, 23%) and Step 3 (n = 8/101, 8%) (p < 0.001). RT was significantly less likely to be the correct answer (2/35, 6%) compared to ST (4/45, 9%) and SI (18/57, 32%) (p = 0.003). Therapeutic oncology questions are uncommon on the examination material, with an under-representation of radiation-related content, and contextual bias favoring surgical approaches. We advocate for greater RO involvement in the content creation of such examinations to help trainees better understand multidisciplinary cancer care.

Gender Disparities in the Academic Performance of Neurology Residents in Portugal.

Implicit bias has been linked to gender disparities in medical careers, impacting not only access to leadership positions but also early career opportunities. We aimed to evaluate if there were differences in the assessment of Neurology residents according to gender. We collected publicly available grades and rankings of two major evaluations that residents are submitted to, one at the beginning and another at the end of the neurology residency program, the National Board Exam and neurology examination, respectively. The National Board Examination is a multiple-choice gender-blinded evaluation, while the neurology examination is an oral gender-unblinded evaluation. We found that 36.5% of women and 21.6% of men were in the first quartile of the National Board Examination ranking, which reflects a similar representation among top classifications when assessed through a gender-blinded examination. On the other hand, the percentage of men who were in the top classification of NE, a gender-unblinded evaluation, was more than twice as high compared to women (37.8% vs 18.3%, p < 0.05). The findings of the present study may imply that there could be a disparity in women's career progression among neurology residents in Portugal, although the recruitment seems balanced between genders.

Performance of GPT-3.5 and GPT-4 on standardized urology knowledge assessment items in the United States: a descriptive study.

This study aimed to evaluate the performance of Chat Generative Pre-Trained Transformer (ChatGPT) with respect to standardized urology multiple-choice items in the United States. In total, 700 multiple-choice urology board exam-style items were submitted to GPT-3.5 and GPT-4, and responses were recorded. Items were categorized based on topic and question complexity (recall, interpretation, and problem-solving). The accuracy of GPT-3.5 and GPT-4 was compared across item types in February 2024. GPT-4 answered 44.4% of items correctly compared to 30.9% for GPT-3.5 (P>0.0001). GPT-4 (vs. GPT-3.5) had higher accuracy with urologic oncology (43.8% vs. 33.9%, P=0.03), sexual medicine (44.3% vs. 27.8%, P=0.046), and pediatric urology (47.1% vs. 27.1%, P=0.012) items. Endourology (38.0% vs. 25.7%, P=0.15), reconstruction and trauma (29.0% vs. 21.0%, P=0.41), and neurourology (49.0% vs. 33.3%, P=0.11) items did not show significant differences in performance across versions. GPT-4 also outperformed GPT-3.5 with respect to recall (45.9% vs. 27.4%, P<0.00001), interpretation (45.6% vs. 31.5%, P=0.0005), and problem-solving (41.8% vs. 34.5%, P=0.56) type items. This difference was not significant for the higher-complexity items. s: ChatGPT performs relatively poorly on standardized multiple-choice urology board exam-style items, with GPT-4 outperforming GPT-3.5. The accuracy was below the proposed minimum passing standards for the American Board of Urology's Continuing Urologic Certification knowledge reinforcement activity (60%). As artificial intelligence progresses in complexity, ChatGPT may become more capable and accurate with respect to board examination items. For now, its responses should be scrutinized.

P-735 Is Artificial Intelligence (AI) currently able to provide evidence-based scientific responses on methods that can improve the outcomes of embryo transfers?

Abstract Study question Could chatbots potentially be used as tools for both patients’ and physicians’ education in an infertility journey? Summary answer Artificial Intelligence (AI) is not yet in a position to give clear, evidence-based recommendations in the field of fertility, particularly concerning embryo transfer. What is known already The rapid development of AI has raised questions about its potential uses in different sectors of everyday life. Individuals from diverse fields have tried to incorporate its usage in their personal but also in their professional lives, with varying levels of success. Specifically in medicine, ChatGPT has been studied intensely with more than 400 related articles having been indexed in PubMed until May 2023, while AI has reportedly also succeeded in several standardized medical tests and passed several Medical Board examinations. Hence the question arises whether chatbots could be used as tools for clinical decision-making or patients’ and physicians’ education. Study design, size, duration We used nine of the most popular free AI chatbots available (ChatGPT, Bard, Writesonic, You, Perplexity, Learnt, Bing, Magickpen, and Rytr) and entered the following command: “Write me a 300-word scientific essay about evidence-based methods that can improve the outcomes of embryo transfer” in May 2023. We collected the responses and extracted the methods each chatbot suggested. When sufficient similarity among answers was present we categorized the answers under one category to facilitate the study. Participants/materials, setting, methods We calculated descriptive statistics and the prevalence of each response. We used as a comparator for widely acceptable practices that are proven to improve embryo transfer outcomes the 2017 ASRM guideline on performing embryo transfer taking also into consideration more current literature. Data was compared using chi-squared tests and a level of p &amp;lt; 0.05 was used as the threshold of statistical significance. Main results and the role of chance The range of the recommendations was from one to nine per chatbot (median = 4, IQR = 2) with an average of 4.78 suggestions per chatbot. Out of a total of 43 recommendations, which could be grouped into 19 similar categories, only 3/19 (15.8%) were evidence-based practices, those being “ultrasound-guided embryo transfer”, which was also the most commonly appearing response, appearing in 7/9 (77.8%) chatbots, “single embryo transfer” appearing in 4/9 (44.4%) chatbots and “use of a soft catheter” in 2/9 (22.2%) chatbots. Some controversial responses appeared even more often than the two latter evidence-based ones with “preimplantation genetic testing (PGT)” being the second most common response with 6/9 chatbots suggesting it (66.7%) and the vague suggestion of “optimal endometrium preparation” being in the third place with 5/9 chatbots (55.6%), both non-evidence-based practices to improve embryo transfer. The majority of the recommendations were unique, with 10 answers appearing only once (1/9; 11.1%), those being “endometrial scratching”, “natural cycle embryo transfer”, “use of GnRH antagonist protocol”, “blastocyst transfer”, “use of specialized catheter”, “preimplantation genetic screening (PGS)”, “mock embryo transfer”, “uninterrupted embryo culture”, “minimizing transfer time”, and “maintaining temperature and pH of the culture media”. Limitations, reasons for caution Firstly, we only used some of the most popular chatbots and not all existing ones. Additionally, chatbots tend to give different answers when repeatedly asked the same questions. However, we believe that our study effectively captures the prevailing practices of chatbot users, who typically pose a question only once. Wider implications of the findings Both patients and physicians should be wary of guiding care based on chatbot recommendations in infertility since the majority of responses consists of scientifically unsupported recommendations. Chatbot results might improve with time especially if trained to obtain information from validated medical databases, however, this will have to be verified scientifically. Trial registration number N/A

Standards as a social contract in curriculum-based qualifications: stakeholder views in Scotland

ABSTRACT The theoretical literature on the meaning of qualification standards depicts a variety of definitions. Some definitions describe properties of examinees, whilst others rely on cohort-level or system-level characteristics. Different definitions can be compatible or contradictory. In this study, stakeholders’ views of the meaning of qualification standards in Scotland were collected, using focus groups (82 participants) and a questionnaire (918 participants). Almost 60% of questionnaire participants responded that standards tell us about performances on the assessment (criterion-referencing) and approximately 40% responded that they tell you about an underlying ability (construct-referencing). Few participants considered that maintaining statistical grade distributions every year were important. Discrepancies in views raise questions regarding how an examination board manages the political and technical process of maintaining public confidence in standards. Based upon this Scottish case, the authors argue that social settlements regarding qualification standards are a social contract, and a solely technocratic view of standards is conceptually inadequate.

ChatGPT and the German board examination for ophthalmology: an evaluation

In recent years artificial intelligence (AI), as anew segment of computer science, has also become increasingly more important in medicine. The aim of this project was to investigate whether the current version of ChatGPT (ChatGPT4.0) is able to answer open questions that could be asked in the context of aGerman board examination in ophthalmology. After excluding image-based questions, 10questions from 15different chapters/topics were selected from the textbook 1000 questions in ophthalmology (1000 Fragen Augenheilkunde 2ndedition, 2014). ChatGPT was instructed by means of aso-called prompt to assume the role of a board certified ophthalmologist and to concentrate on the essentials when answering. Ahuman expert with considerable expertise in the respective topic, evaluated the answers regarding their correctness, relevance and internal coherence. Additionally, the overall performance was rated by school grades and assessed whether the answers would have been sufficient to pass the ophthalmology board examination. The ChatGPT would have passed the board examination in 12out of 15topics. The overall performance, however, was limited with only 53.3% completely correct answers. While the correctness of the results in the different topics was highly variable (uveitis and lens/cataract 100%; optics and refraction 20%), the answers always had ahigh thematic fit (70%) and internal coherence (71%). The fact that ChatGPT4.0 would have passed the specialist examination in 12out of 15topics is remarkable considering the fact that this AI was not specifically trained for medical questions; however, there is aconsiderable performance variability between the topics, with some serious shortcomings that currently rule out its safe use in clinical practice.