Certifying Examination Pass Rates Research Articles

Quantifying For-Profit Outcomes in GME: A Multispecialty Analysis of Board Certifying Examination Pass Rates in For-Profit Affiliated Residency Programs.

The number of for-profit hospitals has increased in the United States, but their role in and outcomes for graduate medical education (GME) are unclear. To describe for-profit involvement in internal medicine (IM), general surgery (GS), and pediatrics GME by quantifying change in for-profit affiliated residency programs and comparing for-profit and nonprofit affiliated program board certifying examination pass rates. We used Accreditation Council for Graduate Medical Education and Medicare data to quantify for-profit prevalence in IM, GS, and pediatrics GME from 2001 to 2021. We used public pass rate data from the American Board of Surgeons (2017-2019; n=242 programs; 6562 examinees), American Board of Internal Medicine (2018-2020; n=465 programs; 23 922 examinees), and American Board of Pediatrics (2018-2020; n=202 programs; 9819 examinees) to model the relationship between profit status and pass rate within each specialty and across specialties combined using linear regression. The proportion of for-profit affiliated residency programs increased 400.0% in IM, 334.4% in GS, and 23.2% in pediatrics from 2001 to 2021. Bivariate linear regression revealed significantly lower pass rate in for-profit affiliated programs in IM β =-7.73, P<.001), pediatrics (β =-14.6, P<.001), and the 3 specialties combined (β =-5.45, P<.001). Upon multiple regression with addition of program characteristic covariates, this relationship remained significant in pediatrics (β =-10.04, P=.006). The proportion of for-profit affiliated residency programs has increased in IM, GS, and pediatrics from 2001 to 2021. After controlling for covariates, for-profit affiliated programs were associated with lower board examination pass rates in pediatrics with no association in IM, GS, or the combined measure.

Predictive Factors of First Time Pass Rate on the American Board of Surgery Certification in General Surgery Exams: A Systematic Review

General Surgery residency programs are evaluated on their American Board of Surgery (ABS) Qualifying examination (QE) and Certifying examination (CE) pass rates. This systematic review aims to evaluate predictive factors of ABS QE and CE first time pass rates. Using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines, the following electronic databases were searched: PubMed, Embase, JAMA Network, and Google Scholar. Studies available in the English language published between January 2000 and July 2020 were deemed eligible. Articles that did not assess either of the ABS board examinations performance and pass-rates as outcomes were excluded. The Oxford Centre for Evidence-Based Medicine was used to determine the quality and risk of bias of each study. A total of 31 publications were included for analysis. Undergraduate medical education variables associated with first-time pass rates on the QE and CE include USMLE score, AOA membership, and class rank. Program factors affecting pass rates include program size, geographic location, and Program Director turnover. There is strong correlation between ABSITE and QE. Evidence supports the utility of mock oral examinations (MOEs) in predicting CE success. ABSITE scores demonstrated higher correlation with QE pass rate than CE pass rate. MOEs have a positive association with first-time CE pass rates. Nonmodifiable factors such as race/ethnicity, marital status, and geographic location were also found to be predictors. Delaying board certification examination beyond 1 year after residency graduation significantly reduces first-time pass rate.

Cardiothoracic surgery mock oral examinations: A single institution's 5-year experience

ObjectivesAlthough in-training examinations provide surrogate data on qualifying exam readiness, use of mock oral examinations (MOEs) in cardiothoracic surgery training before the American Board of Thoracic Surgery certifying oral exam is not uniform. Although MOEs are prioritized by some institutions, development and execution of these labor-intensive, time-consuming exams may be a barrier to others. Therefore, we aimed to develop an MOE program and to assess its educational value. MethodsWe developed an institutional MOE program that mimicked the certification examination and was serially administered to 10 cardiothoracic surgery trainees from 2014 to 2018. Biannual MOE scores were reviewed, along with certifying examination pass rates. MOE data were available for curriculum development and trainee performance evaluations. ResultsMOEs were conducted twice each academic year, with 4 exams administered during each individual's training. MOE program development required significant up-front time commitment, and thereafter each MOE required approximately 24 total faculty hours and 4 administrator hours. The pass rates for sequential MOEs demonstrated gradual improvement, and the corresponding certifying exam pass rate was 100% for these same individuals. MOE data were routinely used for curriculum refinement, as well as individual trainee feedback. ConclusionsStandardized MOEs are useful educational adjuncts to assess trainees' knowledge and readiness for certification exams, but require significant coordination and time to develop an accurate, rigorous simulation mechanism. Although we recognize that improvement in serial MOEs is likely related to exposure as well as expanding funds of knowledge, we believe these results justify use of this assessment tool in training.

Effects of gender in resident evaluations and certifying examination pass rates

BackgroundThough the proportion of female Internal Medicine (IM) residents and faculty has increased, there is minimal large scale modern data comparing resident performance by gender. This study sought to examine the effects of resident and faculty gender on resident evaluations.MethodsRetrospective observational study over 5 years in a single IM program. IM certifying examination pass rates were obtained from the American Board of IM.ResultsFour hundred eighty-eight residents (195 women, 293 men), evaluated by 430 attending physicians (163 women, 270 men) were included. Twelve thousand six hundred eighty-one evaluations between 2007 and 2012 were analyzed. Female residents scored higher in two domains (Medical Interviewing, and Interpersonal and Communication Skills) (p < 0.01 for each), with no significant difference between genders for the other domains (Medical Knowledge, Overall Patient Care, Physical Examination, Procedural Skills, Professionalism, Practice Based Learning and Improvement, System Based Practices and Overall score). There were no differences in scoring between female and male attending physicians. There were no differences in certifying examination scores between women and men among graduating residents. National pass rates for women were not statistically different to pass rates for men from 1987 to 2015.ConclusionsData from one large academic medical center demonstrate higher ratings for female residents on performance domains reflecting bedside care and interpersonal skills, with similar scores for medical knowledge and remaining domains. No significant difference was seen locally in certifying examination scores, nor in recent national pass rates, an objective measure of medical knowledge. Despite imbalanced female representation in areas of medicine, our data suggest that gender-based disparities in Internal Medicine resident medical knowledge and physician competency are no longer present.

Utility of intermittent online quizzes as an early warning for residents at risk of failing the pediatric board certification examination

BackgroundTraditionally, quizzes have been applied as a tool for summative assessment, though literature suggests their use as a formative assessment can improve motivation and content retention. With this premise, we implemented a series of intermittent, online quizzes known as the Board Examination Simulation Exercise (BESE). We sought to demonstrate an association between BESE participation and scores and performance on the American Board of Pediatrics (ABP) Certifying Examination (CE).MethodsResidents were assigned online quizzes on a single topic at 2 week intervals that consisted of 20 multiple choice questions written by the study authors. This analysis includes graduates of 3 Pediatric and Internal Medicine-Pediatrics residency programs.ResultsData were available for 329 residents. The overall BESE score weakly correlated with ABP CE score (n = 287; r = 0.39, p < 0.0001). ABP CE pass rates increased from 2009 to 2016 at all programs combined (p = 0.0001). A composite BESE score ≤ 11 had sensitivity of 54% and specificity of 80% for predicting ABP CE failure on the first attempt. There was no difference in ABP CE failure rates or scores by number of completed quizzes.ConclusionIntermittent online quizzes implemented at three pediatric residency programs were associated with overall increasing ABP CE pass rates. BESE increased program emphasis on board preparation. Residents with lower BESE scores more often failed ABP CE. Though additional data are needed, BESE is a promising tool for pediatric resident learning and board preparation. It may also aid in earlier identification of residents at higher risk of failing the ABP CE and facilitate targeted interventions.

Key Factors Leading to Program Selection: A Survey of Physician Assistant Program Interviewees.

The purpose of this study was to determine which factors had the greatest influence on physician assistant (PA) interviewees' decision to choose a PA program to attend. The information in this article may assist PA programs in making their program more attractive to potential applicants and also may help applicants identify programs that will best fit their needs. Applicants who interviewed with a PA program were asked to rate 33 different influential factors when choosing a program to attend. Respondents most highly endorsed quality of faculty and staff, first-time Physician Assistant National Certifying Examination pass rates, and morale of faculty and staff. Results varied by demographics, including marital status, age, and sex of respondent. Results also varied from pre-PA students. Although there are numerous factors involved in program selection, PA programs may want to focus on the quality and morale of their faculty and staff to help improve the likelihood of attracting and retaining the highest quality applicants.

Association of General Surgery Resident Remediation and Program Director Attitudes With Resident Attrition

Previous studies of resident attrition have variably included preliminary residents and likely overestimated categorical resident attrition. Whether program director attitudes affect attrition has been unclear. To determine whether program director attitudes are associated with resident attrition and to measure the categorical resident attrition rate. This multicenter study surveyed 21 US program directors in general surgery about their opinions regarding resident education and attrition. Data on total resident complement, demographic information, and annual attrition were collected from the program directors for the study period of July 1, 2010, to June 30, 2015. The general surgery programs were chosen on the basis of their geographic location, previous collaboration with some coauthors, prior work in surgical education and research, or a program director willing to participate. Only categorical surgical residents were included in the study; thus, program directors were specifically instructed to exclude any preliminary residents in their responses. Five-year attrition rates (2010-2011 to 2014-2015 academic years) as well as first-time pass rates on the General Surgery Qualifying Examination and General Surgery Certifying Examination of the American Board of Surgery (ABS) were collected. High- and low-attrition programs were compared. The 21 programs represented different geographic locations and 12 university-based, 3 university-affiliated, and 6 independent program types. Programs had a median (interquartile range [IQR]) number of 30 (20-48) categorical residents, and few of those residents were women (median [IQR], 12 [5-17]). Overall, 85 of 966 residents (8.8%) left training during the study period: 15 (17.6%) left after postgraduate year 1, 34 (40.0%) after postgraduate year 2, and 36 (42.4%) after postgraduate year 3 or later. Forty-four residents (51.8%) left general surgery for another surgical discipline, 21 (24.7%) transferred to a different surgery program, and 18 (21.2%) exited graduate medical education altogether. Each program had an annual attrition rate ranging from 0.73% to 6.0% (median [IQR], 2.5% [1.5%-3.4%]). Low-attrition programs were more likely than high-attrition programs to use resident remediation (21.0% vs 6.8%; P < .001). Median (IQR) Qualifying Examination pass rates (93% [90%-98%] vs 92% [86%-100%]; P = .92) and Certifying Examination pass rates (83% [68%-84%] vs 81% [71%-86%]; P = .47) were similar. Program directors at high-attrition programs were more likely than their counterparts at low-attrition programs to agree with this statement: "I feel that it is my responsibility as a program director to redirect residents who should not be surgeons." The overall 5-year attrition rate of 8.8% was significantly lower than previously reported. Program directors at low-attrition programs were more likely to use resident remediation. Variations in attrition may be explained by program director attitudes, although larger studies are needed to further define program factors affecting attrition.

Relationship Between Physician Assistant Program Length and Physician Assistant National Certifying Examination Pass Rates.

This study was conducted to examine the relationship between physician assistant (PA) educational program length and PA programs' 5-year average Physician Assistant National Certifying Examination (PANCE) first-time pass rates. This was a retrospective correlational study that analyzed previously collected data from a nonprobability purposive sample of accredited PA program Web sites. Master's level PA programs (n = 108) in the United States with published average PANCE scores for 5 consecutive classes were included. Provisional and probationary programs were excluded (n = 4). Study data were not normally distributed per the Kolmogorov-Smirnov test, P = .00. There was no relationship between program length and PANCE pass rates, ρ (108) = -0.04, P = .68. Further analyses examining a possible relationship between program phase length (didactic and clinical) and PANCE pass rates also demonstrated no differences (ρ [107] = -0.05, P = .60 and ρ [107] = 0.02, P = .80, respectively). The results of this study suggest that shorter length PA programs perform similarly to longer programs in preparing students to pass the PANCE. In light of rapid expansion of PA educational programs, educators may want to consider these findings when planning the length of study for new and established programs.

The impact of an advanced certifying examination simulation program on the American Board of Surgery Certifying Examination passage rates.

The national pass rate for the American Board of Surgery Certifying Examination has decreased in the past 5 years. An individual's ability to pass might be as dependent on his or her handling of the psychology of the examination-the "examanship" -because it is about clinical knowledge and judgment. To assess this, we implemented the Advanced Certifying Examination Simulation (ACES) program. The ACES was created as a novel method to simulate the stress of the Certifying Examination and focuses on the examanship of the test. We compared the outcomes of the ACES program with its predecessor, a conventional mock oral program, as measured by residents' first-time pass rates on the Certifying Examination. First-time Certifying Examination pass rates of 26 residents who went through the ACES program were compared with 30 residents who completed the conventional mock oral program. There was a significant increase in passage rates for residents taking part in the ACES program (100%) compared with residents taking part in the conventional mock oral group (83.3%). The groups were equivalent based on previously determined predictive factors of Certifying Examination success, such as in-training and licensing examination scores. The ACES program provides feedback on the qualities of examanship: controlling anxiety, expressing a positive attitude, and maintaining a strong and confident voice. By providing a structured, simulated venue where residents can safely gain experience, we believe that ACES might lead to increased first-time passage rates on the American Board of Surgery Certifying Examination.

Predicting Performance on the American Board of Surgery Qualifying and Certifying Examinations

We sought to determine whether US Medical Licensing Examination (USMLE) Step 1 score, American Board of Surgery (ABS) In-Training Examination (ABSITE) score, and other variables are associated with failing the ABS qualifying and certifying examinations. Identifying such factors may assist in the early implementation of an academic intervention for at-risk residents. Retrospective review. Seventeen general surgery training programs in the western United States. Six hundred seven residents who graduated in 2000-2007. First-time pass rates on the qualifying and certifying examinations, US vs non-US medical school graduation, USMLE Steps 1 and 2 scores, ABSITE scores, operative case volume, fellowship training, residency program type, and mandatory research. The first-time qualifying and certifying examination pass rates for the 607 graduating residents were 78% and 74%, respectively. On multivariable analysis, scoring below the 35th percentile on the ABSITE at any time during residency was associated with an increased risk of failing both examinations (odds ratio, 0.23 [95% confidence interval, 0.08-0.68] for the qualifying examination and 0.35 [0.20-0.61] for the certifying examination), as was scoring less than 200 on the USMLE Step 1 (0.36 [0.21-0.62] for the qualifying examination and 0.62 [0.42-0.93] for the certifying examination). A mandatory research year was associated with an increased likelihood of passing the certifying examination (odds ratio, 3.3 [95% confidence interval, 1.6-6.8]). Residents who are more likely to fail the ABS qualifying and certifying examinations can be identified by a low USMLE Step 1 score and by poor performance on the ABSITE at any time during residency. These findings support the use of the USMLE Step 1 score in the surgical residency selection process and a formal academic intervention for residents who perform poorly on the ABSITE.

Benefits of Mock Oral Examinations in a Multi-institutional Consortium for Board Certification in General Surgery Training

The Southeast Michigan Center for Medical Education (SEMCME) is a consortium of teaching hospitals in the Greater Detroit metropolitan area. SEMCME pools its resources for several educational means, including mock oral board examinations. The educational and cost benefits to mock oral examinations on a multi-institutional basis in preparation for the American Board of Surgery (ABS) certifying examination were analyzed. Ten-year multi-institution data from the mock oral examinations were correlated with ABS certifying examination pass rates. Mock oral examination scores were available for 107 of 147 graduates, which included 12 candidates who failed their certifying examination on the first attempt (pass rate = 89%). Four of 31 examinees who had a low score (4.9 or less) in their mock oral exams failed their certifying examination in their first attempt. The cost of running the mock examination was low (approximately $35/resident for 50 residents). When graduates from the last 10 years were surveyed, the majority of respondents believed that the mock oral examination helped in their success and with their preparation for the certifying examination. Thus, the many benefits of administering the examination with the resources of a consortium of hospitals result in the accurate reproduction of real-life testing conditions with reasonable overall costs per resident.