Singapore’s Accreditation Council for Graduate Medical Education-International (ACGME-I) Otorhinolaryngology residency programme: Otorhinolaryngology Training Exam (OTE) scores as a predictor of performance at the Otorhinolaryngology final exit MCQ (Multiple-Choice-Question) examinations

The Sun Never Sets on Graduate Medical Education-International Peer Review of ACGME-I-Accredited Programs and Institutions.

RadioGraphics Content Curation: A Comprehensive Curriculum for Radiology Trainees.

Development and validation of a multiple choice examination assessing cognitive and behavioural knowledge of pediatric resuscitation: A report from the EXPRESS pediatric research collaborative

Clinical-vignette multiple choice question (MCQ) examinations are used widely in medical education. Standardised MCQ examinations are used by licensure and certification bodies to award credentials that are meant to assure stakeholders as to the quality of physicians. Such uses are based on the interpretation of MCQ examination performance as giving meaningful information about the quality of clinical reasoning. There are several assumptions foundational to these interpretations and uses of standardised MCQ examinations. This study explores the implicit assumption that cognitive processes elicited by clinical-vignette MCQ items are like the processes thought to occur with 'real-world' clinical reasoning as theorised by dual-process theory. Fourteen participants (three medical students, five residents and six staff physicians) completed three sets of five timed MCQ items (total 15) from the Medical Knowledge Self-Assessment Program (MKSAP). Upon answering a set of MCQs, each participant completed a retrospective think aloud (TA) protocol. Using constant comparative analysis (CCA) methods sensitised by dual-process theory, we performed a qualitative thematic analysis. Examinee behaviours fell into three categories: clinical reasoning behaviours, test-taking behaviours and reactions to the MCQ. Consistent with dual-process theory, statements about clinical reasoning behaviours were divided into two sub-categories: analytical reasoning and non-analytical reasoning. Each of these categories included several themes. Our study provides some validity evidence that test-takers' descriptions of their cognitive processes during completion of high-quality clinical-vignette MCQs align with processes expected in real-world clinical reasoning. This supports one of the assumptions important for interpretations of MCQ examination scores as meaningful measures of clinical reasoning. Our observations also suggest that MCQs elicit other cognitive processes, including certain test-taking behaviours, that seem 'inauthentic' to real-world clinical reasoning. Further research is needed to explore if similar themes arise in other contexts (e.g. simulated patient encounters) and how observed behaviours relate to performance on MCQ-based assessments.

We observed an unprecedented and consistent low performance of senior residents as compared with juniors in monthly examinations. This compelled us to evaluate systematically and compare the scores of senior...

In-training examinations (ITEs), arranged during residency training, evaluate the residents’ performances periodically. There is limited literature focusing on the effectiveness of resident ITEs in the format of simulation-based examinations, as compared to traditional oral or written tests. Our primary objective is to investigate the effectiveness and discriminative ability of high-fidelity simulation compared with other measurement formats in emergency medicine (EM) residency training program.This is a retrospective cohort study. During the 5-year study period, 8 ITEs were administered to 68 EM residents, and 253 ITE measurements were collected. Different ITE scores were calculated and presented as mean and standard deviation. The ITEs were categorized into written, oral, or high-fidelity simulation test forms. Discrimination of ITE scores between different training years of residency was examined using a one-way analysis of variance test.The high-fidelity simulation scores correlated to the progression of EM training, and residents in their fourth training year (R4) had the highest scores consistently, followed by R3, R2, and then R1. The oral test scores had similar results but not as consistent as the high-fidelity simulation tests. The written test scores distribution failed to discriminate the residents’ seniority. The high-fidelity simulation test had the best discriminative ability and better correlation between different EM residency training years comparing to other forms.High-fidelity simulation tests had the good discriminative ability and were well correlated to the EM training year. We suggest high-fidelity simulation should be a part of ITE in training programs associated with critical or emergency patient cares.

Recently, the clinical practice examination (CPX) using standardized patients has been introduced into several specialty certifying examinations in Korea. The purpose of this paper was to determine the correlation of a resident's performance on the CPX with the comprehensive written multiple-choice question (MCQ) examination on the certifying examination for family medicine. The subjects of this study were 1,023 residents who completed the 1st and 2nd certifying examination for family medicine between 2009 and 2011. We determined the correlation between the total scores and 4 domain scores (history taking, physical examination, patient education, and patient-physician interaction) on the CPX with the MCQ scores of the 1st written test and 2nd slide examination and the correlation between the total CPX score and scores on the CPX domains. The correlation between CPX score with each MCQ examination (0.21~0.45 with 1st written MCQ, 0.15~0.33 with 2nd slide MCQ) was lower than that between each MCQ examination (0.46~0.59). The CPX score on patient education did not correlate with the 1st written and 2nd MCQ scores. The CPX scores on history taking and physical examination correlated slightly with the 1st written MCQ scores. The global ratings of preceptor examiners had the highest correlation (r=0.68~0.82) with the total CPX scores. Considering the mild correlation of CPX scores with each MCQ examination, the CPX is more likely to measure other qualities, such as critical thinking and communication skills.

Anxiety is thought to affect test performance. Studies have shown that students with low levels of test anxiety achieve higher scores on multiple choice question (MCQ) examinations than those with high anxiety levels. Female students have been shown to have higher test anxiety levels than male students. Standardized patient (SP) examinations are being used in medical schools and for licensing purposes. As SP exams are relatively new, there are few studies measuring anxiety levels for the SP test. The purpose of this study was to measure and compare medicine clerkship student SP versus MCQ examination anxiety levels and to determine if level affected test performance. The Spielberger test attitude inventory was used to measure anxiety in 150 students rotating through the clerkship. Students completed questionnaires after the MCQ and SP examinations. Mean examination scores and anxiety levels were compared. Based on questionnaire scores, students were divided into 3 groups: low, moderate, and high anxiety. The MCQ and SP examination scores were analyzed to determine if male/female anxiety-level affected test performance. There were no meaningful anxiety-level differences between the SP and MCQ examinations. An inverse relationship between anxiety level and test scores was not identified. Female students had higher anxiety levels but sex differences did not influence examination performance. Medicine clerkship student test performance is not affected by anxiety level. Implications of the findings for incorporating stress management training in medical school curricula and suggestions for future research are discussed.

CONTEXTHistorically, the assessment of medical resident knowledge and skill has occurred through annual in-training examinations and faculty observation during real patient encounters. To improve patient care and the learning environment, medical educators have begun creating simulation experiences for medical residents to perform procedures without using real patients. However, simulation curricula and accompanying assessment techniques have not been standardized. Using a longitudinal record review, in-training examination scores were compared to annualized simulation assessment scores to see if there is any relationship between the assessment types.METHODSThis project was a retrospective eight-year study from a single residency program. The scores were collected from 102 resident academic records from 2007 to 2015 for the annual American Board of Emergency Medicine in-training examination and the resident’s annual simulation assessment. Complete data sets were analyzed to determine if a relationship exists between the assessment methods. Then the averages were compared for only the first three years for all students and for all four years for osteopathic residents as they have an additional fourth year of training.RESULTSThis study showed a lack of relationship between the two assessment types when reviewing three years of data. When the fourth year data is considered, there was a significant relationship between the assessment types.CONCLUSIONSThe performance scores for both types of assessment provide independent information on the resident progress in training. Therefore, they should both be reviewed and considered to appropriately measure the resident’s performance. The significance of the fourth year of training for osteopathic residents requires further study.

Fifth year of residency versus fourth year of medical school and residency

Background: Script concordance (SC) questions, in which a learner is given a brief clinical scenario then asked if additional information makes one hypothesis more or less likely, with answers compared to a panel of experts, are designed to reflect a learner's clinical reasoning. Purpose: The purpose is to compare reliability, validity, and learner satisfaction between a three-option modified SC examination to a multiple-choice question (MCQ) examination among medical students during a 3rd-year internal medicine clerkship, to compare reliability and learner satisfaction of SC between medical students and a convenience sample of house staff, and to compare learner satisfaction with SC between 1st- and 4th-quarter medical students. Methods: Using a prospective cohort design, we compared the reliability of 20-item SC and MCQ examinations, sequentially administered on the same day. To measure validity, scores were compared to scores on the National Board of Medical Examiners (NBME) subject examination in medicine and to a clinical performance measure. SC and MCQ were also administered to a convenience sample of internal medicine house staff. Medical student and house staff were anonymously surveyed regarding satisfaction with the examinations. Results: There were 163 students who completed the examinations. With students, the initial reliability of the SC was half that of MCQ (KR20 0.19 vs. 0.41), but with house staff (n = 15), reliability was the same (KR20 = 0.52 for both examinations). SC performance correlated with student clinical performance, whereas MCQ did not (r = .22, p = .005 vs. .11, p = .159). Students reported that SC questions were no more difficult and were answered more quickly than MCQ questions. Both exams were considered easier than NBME, and all 3 were considered equally fair. More students preferred MCQ over SC (55.8% vs. 18.0%), whereas house staff preferred SC (46% vs. 23%; p = .03). Conclusions: This SC examination was feasible and was more valid than the MCQ examination because of better correlation with clinical performance, despite being initially less reliable and less preferred by students. SC was more reliable and preferred when administered to house staff.

The Introduction of Single Best Answer Questions as a Test of Knowledge in the Final Examination for the Fellowship of the Royal College of Radiologists in Clinical Oncology

A Model for Wilderness Medicine Education in a Family Practice Residency

OPINION article Front. Pediatr., 24 April 2014Sec. Neonatology Volume 2 - 2014 | https://doi.org/10.3389/fped.2014.00020

To examine whether resident communication skills evaluated through patient satisfaction surveys demonstrate evidence of decline through the 3years of internal medicine residency. Data for this study were collected retrospectively from a database of patient satisfaction surveys completed for internal medicine residents at different levels of training. Patient satisfaction was measured with the Aggregated EVGFP (excellent, very good, good, fair, or poor) questionnaire recommended by the American Board of Internal Medicine. Over a span of 5years (2005-2009), a total of 768 patient rating forms were completed for 67 residents during their 3years of residency training. In postgraduate year (PGY)-1, the residents had a mean satisfaction rating of 4.33 ± 0.48 compared to a mean rating of 4.37 ± 0.45 in their PGY-3year. Analysis of variance indicated no significant difference by PGY level. Our findings demonstrate that resident communication skills and patient satisfaction do not decline during the 3 years of residency. This is contrary to our hypothesis that patient satisfaction would worsen as residents progressed through training.