Surgery Qualifying Examination Research Articles

Relationships between study habits, burnout, and general surgery resident performance on the American Board of Surgery In-Training Examination

BackgroundThe American Board of Surgery In-Training Examination (ABSITE) is used by programs to evaluate the knowledge and readiness of trainees to sit for the general surgery qualifying examination. It is often used as a tool for resident promotion and may be used by fellowship programs to evaluate candidates. Burnout has been associated with job performance and satisfaction; however, its presence and effects on surgical trainees' performance are not well studied. We sought to understand factors including burnout and study habits that may contribute to performance on the ABSITE examination. MethodsAnonymous electronic surveys were distributed to all residents at 10 surgical residency programs (n = 326). Questions included demographics as well as study habits, career interests, residency characteristics, and burnout scores using the Oldenburg Burnout Inventory, which assesses burnout because of both exhaustion and disengagement. These surveys were then linked to the individual's 2016 ABSITE and United States Medical Licensing Examination (USMLE) step 1 and 2 scores provided by the programs to determine factors associated with successful ABSITE performance. ResultsIn total, 48% (n = 157) of the residents completed the survey. Of those completing the survey, 48 (31%) scored in the highest ABSITE quartile (≥75th percentile) and 109 (69%) scored less than the 75th percentile. In univariate analyses, those in the highest ABSITE quartile had significantly higher USMLE step 1 and step 2 scores (P < 0.001), significantly lower burnout scores (disengagement, P < 0.01; exhaustion, P < 0.04), and held opinions that the ABSITE was important for improving their surgical knowledge (P < 0.01). They also read more frequently to prepare for the ABSITE (P < 0.001), had more disciplined study habits (P < 0.001), were more likely to study at the hospital or other public settings (e.g., library, coffee shop compared with at home; P < 0.04), and used active rather than passive study strategies (P < 0.04). Gender, marital status, having children, and debt burden had no correlation with examination success. Backward stepwise multiple regression analysis identified the following independent predictors of ABSITE scores: study location (P < 0.0001), frequency of reading (P = 0.0001), Oldenburg Burnout Inventory exhaustion (P = 0.02), and USMLE step 1 and 2 scores (P = 0.007 and 0.0001, respectively). ConclusionsResidents who perform higher on the ABSITE have a regular study schedule throughout the year, report less burnout because of exhaustion, study away from home, and have shown success in prior standardized tests. Further study is needed to determine the effects of burnout on clinical duties, career advancement, and satisfaction.

General surgery resident rotations in surgical critical care, trauma, and burns: what is optimal for residency training?

BackgroundThere are no specific Accreditation Council for Graduate Medical Education General Surgery Residency Program Requirements for rotations in surgical critical care (SCC), trauma, and burn. We sought to determine the experience of general surgery residents in SCC, trauma, and burn rotations. MethodsData analysis of surgical rotations of American Board of Surgery general surgery resident applicants (n = 7,299) for the last 8 years (2006 to 2013, inclusive) was performed through electronic applications to the American Board of Surgery Qualifying Examination. Duration (months) spent in SCC, trauma, and burn rotations, and postgraduate year (PGY) level were examined. ResultsThe total months in SCC, trauma and burn rotations was mean 10.2 and median 10.0 (SD 3.9 months), representing approximately 16.7% (10 of 60 months) of a general surgery resident's training. However, there was great variability (range 0 to 29 months). SCC rotation duration was mean 3.1 and median 3.0 months (SD 2, min to max: 0 to 15), trauma rotation duration was mean 6.3 and median 6.0 months (SD 3.5, min to max: 0 to 24), and burn rotation duration was mean 0.8 and median 1.0 months (SD 1.0, min to max: 0 to 6). Of the total mean 10.2 months duration, the longest exposure was 2 months as PGY-1, 3.4 months as PGY-2, 1.9 months as PGY-3, 2.2 months as PGY-4 and 1.1 months as PGY-5. PGY-5 residents spent a mean of 1 month in SCC, trauma, and burn rotations. PGY-4/5 residents spent the majority of this total time in trauma rotations, whereas junior residents (PGY-1 to 3) in SCC and trauma rotations. ConclusionsThere is significant variability in total duration of SCC, trauma, and burn rotations and PGY level in US general surgery residency programs, which may result in significant variability in the fund of knowledge and clinical experience of the trainee completing general surgery residency training. As acute care surgery programs have begun to integrate emergency general surgery with SCC, trauma, and burn rotations, it is an ideal time to determine the optimal curriculum and duration of these important rotations for general surgery residency training.

A Sensitivity Analysis and Opportunity Cost Evaluation of the Surgical Council on Resident Education Curriculum

The study purpose is to evaluate the Surgical Council on Resident Education (SCORE) Curriculum regarding American Board of Surgery Qualifying Examination (ABS QE) outcomes. The goal is to perform effect size analyses, sensitivity analyses, and sample size analyses with opportunity cost estimates required to favor the SCORE Curriculum subscription regarding ABS QE outcomes. Published data were used to construct 2 × 2 matrices regarding ABS QE outcome (pass/fail) and SCORE subscription status (subscriber/nonsubscriber). Post hoc analyses of effect sizes and sample sizes, with opportunity cost estimates, were performed to evaluate ABS QE outcomes favoring SCORE subscription (2-tailed and 1-tailed tests) using an α = 0.05. The absolute risk increase of SCORE subscription on ABS QE outcome was 1.6% (number needed to treat = 63). Sensitivity analyses showed that a pass rate difference of 4.9% to 7.5% was required to favor SCORE subscription (all p < 0.05). Sample size analyses required an 8- to 18-fold increase to favor SCORE subscription to achieve statistical significance with an opportunity cost of $6.0 to $13.5 million ($30,000-$67,000/program), not adjusting for inflation. The number needed to treat and pass rate differences required to favor SCORE subscription are large. The opportunity costs of SCORE subscription are substantial. Residency programs with more limited resources should determine if the subscription costs are financially sound.

Using the American Board of Surgery In-Training Examination to Predict Board Certification: A Cautionary Study

Although designed as a low-stakes formative examination, the American Board of Surgery In-Training Examination (ABSITE) is often used in high-stakes decisions such as promotion, remediation, and retention owing to its perceived ability to predict the outcome of board certification. Because of the discrepancy between intent and use, the ability of ABSITE scores to predict passing the American Board of Surgery certification examinations was analyzed. All first-time American Board of Surgery qualifying examination (QE) examinees between 2006 and 2012 were reviewed. Examinees' postgraduate year (PGY) 1 and PGY5 ABSITE standard scores were linked to QE scores and pass/fail outcomes (n = 6912 and 6846, respectively) as well as first-time certifying examination (CE) pass/fail results (n = 1329). Linear and logistic regression analyses were performed to evaluate the utility of ABSITE scores to predict board certification scores and pass/fail outcomes. PGY1 ABSITE scores accounted for 22% of the variance in QE scores (p < 0.001). PGY5 scores were a slightly better predictor, accounting for 30% of QE score variance (p < 0.001). Analyses showed that selecting a PGY5 ABSITE score that maximized overall decision accuracy for predicting QE pass/fail outcomes (86% accuracy) resulted in 98% sensitivity, 13% specificity, a positive predictive value of 87%, and a negative predictive value of 57%. ABSITE scores were not predictive of success on the CE. ABSITE scores are a useful predictor of QE scores and outcomes but do not predict passing the CE. Although scoring well on the ABSITE is highly predictive of QE success, using low ABSITE scores to predict QE failure results in frequent decision errors. Program directors and other evaluators should use additional sources of information when making high-stakes decisions about resident performance.

Subscription to the Surgical Council on Resident Education Web Portal and Qualifying Examination Performance

The Surgical Council on Resident Education (SCORE) curriculum for general surgery was developed to guide surgery residents in the acquisition of knowledge for patient care. We hypothesized that residents in programs that subscribed to the SCORE web portal would perform better on the American Board of Surgery (ABS) Surgery Qualifying Examination (QE). Scaled scores and the percent passing the 2011 ABS Surgery QE for individual residents and programs were compared between programs that subscribed to the SCORE portal in 2010 to 2011 and those that did not subscribe. Regression analyses were performed to control for program QE percent passing from 2004 to 2008 (baseline performance), as well as demographic factors known to affect examination results. There were 200 programs and 893 residents that subscribed to the SCORE web portal and 33 programs with 139 residents that did not subscribe. Regression analysis comparing predicted 2011 mean program QE scores based on 2004 to 2008 results showed that subscribing programs had a substantial increase in mean scaled scores of 1.4 points (adjusted means of 81.5 and 80.1, respectively), controlling for the percentage of international medical graduates and program size (p = 0.048). Residents from SCORE portal subscribing programs had a QE percent passing that was 1.6% higher than nonsubscribing residents, and the mean percent passing was higher for subscribing programs (86.4% vs 82.7%), but neither difference was statistically significant. The SCORE subscription status did not correlate with program size, percent of international medical graduates, or baseline scale scores. There was a considerable improvement in mean QE scaled scores for residents in programs that initially subscribed to the SCORE web portal. The percent passing the QE showed a trend toward improvement for subscribing programs and their residents. This association is promising and deserves additional investigation.

Delay in taking the American Board of Surgery qualifying examination affects examination performance

The American Board of Surgery (ABS) Qualifying Examination (QE) represents an important step along the pathway to board certification. We investigated whether candidates who delayed taking the QE had worse performance on the examination. QE pass rates and equated scaled scores for all first-time examinees from 2006 to 2010 (n = 5,193) were reviewed. After eliminating examinees who could not be matched to final ABS In-Training Examination (ABSITE) scores, the remaining cohort (n = 4,909) was analyzed by comparing those who took the exam immediately after residency (Immediate, n = 4,488) to those who delayed for 1 or more years (Delay, n = 421). The Immediate group had a mean first-time QE pass rate of 87% compared to 57% for those who delayed 1 year and 48% for those who delayed 2 or more years (P < .001). Regression analysis demonstrated that delay in taking the QE remained a significant determinant of exam failure after controlling for ABSITE scores (odds ratio = 0.35; 95% CI, 0.29-0.43; P = .001). Undergraduate medical education and postresidency training did not affect the results. The Delay group had lower equated scaled scores, a greater ultimate failure rate on the QE, and was more likely to fail the ABS Certifying Examination on the first attempt. These results demonstrate that candidates who delayed taking the QE immediately are at extremely high risk for exam failure and failure to achieve board certification. These findings presumably are due to deterioration of knowledge over time, but they also may represent characteristics of the Delay group that are currently undefined.

The state of performance on the American Board of Surgery Qualifying Examination and Certifying Examination and the effect of residency program size on program pass rates

The state of performance on the American Board of Surgery Qualifying Examination and Certifying Examination and the effect of residency program size on program pass rates

Feast or Famine?

Nearly 80% of general surgery residents (GSR) pursue Fellowship training. We hypothesized that fellowships coexisting with general surgery residencies do not negatively impact GSR case volumes and that fellowship-bound residents (FBR) preferentially seek out cases in their chosen specialty ("early tracking"). To test our hypotheses, we analyzed the Accreditation Council for Graduate Medical Education Surgical Operative Log data from 2009 American Board of Surgery qualifying examination applicants (N = 976). General surgery programs coexisted with 35 colorectal (CR), 97 vascular (Vasc), 80 minimally invasive (MIS), and 12 Endocrine (Endo) fellowships. We analyzed (1) operative cases for general surgery residency programs with and without coexisting Fellowships, comparing caseloads for FBR and all GSR and (2) operative cases of FBR in their chosen specialties compared to all other GSR. Group means were compared using ANOVA with significance set at P < 0.01. Coexisting fellowships had minimal impact on GSR caseloads. Endocrine fellowships actually enhanced case volumes for all residents. CR impact was neutral while MIS and vascular fellowships resulted in small declines. Endo, CR, and Vasc but not MIS FBR performed significantly more cases in their future specialties than their GSR counterparts, consistent with self-directed, prefellowship tracking. Tracking seems to be additive and FBR do not sacrifice other GSR cases. Our data establish that the impact of Fellowships on GSR caseloads is minimal. Our data confirm that FBR seek out cases in their future specialties ("early tracking").

How Important Are American Board of Surgery In-Training Examination Scores When Applying for Fellowships?

The American Board of Surgery In-Training examination (ABSITE) first was administered in 1975 to evaluate a resident's general knowledge as well as the deficiencies within the resident and surgical program. The added importance of this examination in recent years stems from a correlation between ABSITE performance and performance on the American Board of Surgery qualifying examination. However, data are lacking in regard to how fellowship programs view ABSITE scores when considering applicants. Thus, this study was initiated to determine the importance of the ABSITE for surgical residents applying to fellowships. Program coordinators and directors of various surgical fellowships were sent a short survey in regard to the ABSITE. The data then were analyzed. One hundred forty-eight surveys were completed, with 74.8% of the programs ranking the importance of ABSITE scores as 3 or 4 (on a scale of 1 through 5). Most programs (78.9%) reported no minimum percentile requirement. Those that did required a mean percentile of 54.4. Of the programs, 57.8% placed a greater emphasis on the senior examination versus the junior examination (p = 0.06). When compared with other application factors, the ABSITE score ranked 3rd behind letters of recommendation and a candidate's residency program. Colon and rectal surgery placed the highest importance on ABSITE scores, whereas transplant surgery placed the lowest importance. The ABSITE score is an important factor for residents applying to surgical fellowship; however, more weight is given to candidates' letters of recommendation and his or her residency program. Applicants should aim to score above the 50th percentile to be competitive for most fellowship programs.

The Impact of Work Hour Restrictions on Surgical Resident Education

Resident work-hour restrictions were instituted in July 2003 based on ACGME mandates. The American Board of Surgery In-Training Examination (ABSITE), American Board of Surgery Qualifying Examination (ABSQE), and operative volume traditionally have been measures of surgical resident education and competency. The objective of this study was to determine the effect of reduced work hours on resident standardized test scores and operative volume at our institution. We reviewed ABSITE scores, ABSQE scores, and operative logs from 1997 to 2005 of all general surgery residents. Linear mixed-effects models were fitted for each component ABSITE score (total, basic science, and clinical management), and they were compared using a chi-squared likelihood ratio. Operative logs of graduating residents were compared before and after the work restrictions and were evaluated for association with ABSITE score. p-values less than 0.05 were considered significant. The program was compliant with ACGME mandates within 6 months of institution. ABSITE scores improved significantly after the restriction of work hours in both basic science (p = 0.003) and total score (p = 0.008). Clinical management scores were not affected. The number of major cases recorded by graduating residents did not change. A positive correlation was found between number of cases performed during residency and clinical management ABSITE scores (p = 0.045). ABSQE scores were not impacted by operative volume during residency. ABSITE scores improved significantly after the restriction of resident work hours. Resident operative experience was not affected. An unexpected consequence of work-hour restrictions may be an improvement in surgical resident education.

Reactions of surgical program directors to a web-based interactive educational program focusing on cognitive skills

There are no specific Accreditation Council for Graduate Medical Education General Surgery Residency Program Requirements for rotations in surgical critical care (SCC), trauma, and burn. We sought to determine the experience of general surgery residents in SCC, trauma, and burn rotations.Data analysis of surgical rotations of American Board of Surgery general surgery resident applicants (n = 7,299) for the last 8 years (2006 to 2013, inclusive) was performed through electronic applications to the American Board of Surgery Qualifying Examination. Duration (months) spent in SCC, trauma, and burn rotations, and postgraduate year (PGY) level were examined.The total months in SCC, trauma and burn rotations was mean 10.2 and median 10.0 (SD 3.9 months), representing approximately 16.7% (10 of 60 months) of a general surgery resident's training. However, there was great variability (range 0 to 29 months). SCC rotation duration was mean 3.1 and median 3.0 months (SD 2, min to max: 0 to 15), trauma rotation duration was mean 6.3 and median 6.0 months (SD 3.5, min to max: 0 to 24), and burn rotation duration was mean 0.8 and median 1.0 months (SD 1.0, min to max: 0 to 6). Of the total mean 10.2 months duration, the longest exposure was 2 months as PGY-1, 3.4 months as PGY-2, 1.9 months as PGY-3, 2.2 months as PGY-4 and 1.1 months as PGY-5. PGY-5 residents spent a mean of 1 month in SCC, trauma, and burn rotations. PGY-4/5 residents spent the majority of this total time in trauma rotations, whereas junior residents (PGY-1 to 3) in SCC and trauma rotations.There is significant variability in total duration of SCC, trauma, and burn rotations and PGY level in US general surgery residency programs, which may result in significant variability in the fund of knowledge and clinical experience of the trainee completing general surgery residency training. As acute care surgery programs have begun to integrate emergency general surgery with SCC, trauma, and burn rotations, it is an ideal time to determine the optimal curriculum and duration of these important rotations for general surgery residency training.

The impact of work hour restrictions on surgical resident education

Objective Resident work-hour restrictions were instituted in July 2003 based on ACGME mandates. The American Board of Surgery In-Training Examination (ABSITE), American Board of Surgery Qualifying Examination (ABSQE), and operative volume traditionally have been measures of surgical resident education and competency. The objective of this study was to determine the effect of reduced work hours on resident standardized test scores and operative volume at our institution. Design We reviewed ABSITE scores, ABSQE scores, and operative logs from 1997 to 2005 of all general surgery residents. Linear mixed-effects models were fitted for each component ABSITE score (total, basic science, and clinical management), and they were compared using a chi-squared likelihood ratio. Operative logs of graduating residents were compared before and after the work restrictions and were evaluated for association with ABSITE score. p-values less than 0.05 were considered significant. Results The program was compliant with ACGME mandates within 6 months of institution. ABSITE scores improved significantly after the restriction of work hours in both basic science (p = 0.003) and total score (p = 0.008). Clinical management scores were not affected. The number of major cases recorded by graduating residents did not change. A positive correlation was found between number of cases performed during residency and clinical management ABSITE scores (p = 0.045). ABSQE scores were not impacted by operative volume during residency. Conclusions ABSITE scores improved significantly after the restriction of resident work hours. Resident operative experience was not affected. An unexpected consequence of work-hour restrictions may be an improvement in surgical resident education.

Weekly Reading Assignments and Examinations Result in Sustained Improvement in American Board of Surgery In-Training Examination (ABSITE) Scores

The American Board of Surgery In-Training Examination (ABSITE) score is used by general surgery training programs as a measure of the medical knowledge component of the Accreditation Council of Graduate Medical Education competencies. Poor performance on ABSITE (< 35%) has correlated with high failure rates on the American Board of Surgery Qualifying Examination. We previously demonstrated an improvement in ABSITE scores over a 1-year period through the initiation of weekly reading assignments and review examinations. We sought to determine whether this educational endeavor would result in a sustained improvement in ABSITE scores over several years. ABSITE scores from two successive, 3-year time periods (1999-2001, Group 1, 2002-2004, Group 2), were compared. For Group 1, no formal reading assignments or preparatory exams were given. For Group 2, weekly reading assignments and examinations were administered by the faculty. For Group 1, the mean ABSITE score was 60 per cent, versus 75 per cent (P < 0.01) for Group 2. In Group 1, 13 out of 76 scores (17%) were below 35 per cent, whereas in Group 2, only 4 out of 74 scores (5%) were below 35 per cent (P = 0.03). Sustained improvement in ABSITE scores and fewer scores < 35 per cent were achieved over a 3-year period with an educational program of weekly topic-specific reading assignments and written review examinations.

Basic science curriculum in vascular surgery residency

Recognizing the importance of basic science teaching in surgical education, the leadership of the Association of Program Directors in Vascular Surgery (APDVS) appointed a panel to gather information and to present its findings at the 1999 annual fall meeting of the Apdvs. A questionnaire was distributed to the program directors present. In addition, information was gathered from the American Board of Surgery regarding the basic science content in the vascular surgery item pool on the vascular surgery qualifying examination (VQE). The vascular surgery unit of the surgical resident curriculum was also analyzed. Fifty-three program directors (64%) completed the questionnaire. Although only two program directors felt that their residents were better prepared to answer basic science questions, the results of the Vqe showed that the examinees do not, as a group, perform differently on basic science items than on clinical management questions. In addition, only a minority of program directors (15%) use a specific method to monitor the learning process of their residents. The majority of the program directors responding (75%) felt that they were capable of teaching basic science to residents. Interestingly, almost half the 53 respondents (47%) said that a basic science curriculum should be comprehensive, not exclusively relevant to the clinical setting. Vqe content outline and the vascular surgery unit of the surgical resident curriculum revealed great emphasis on clinically relevant basic science information. The Apdvs panel recommends that a basic science curriculum should be comprehensive, yet clinically pertinent, and completely integrated with the clinical curriculum. In terms of how to teach basic science in vascular residencies, the panel supports teaching conferences that are problem-based with a faculty member acting as the “resource person” and with specific goals set for the conferences. The panel also suggested establishing a Web site that provides a series of questions, the answers of which could be readily available to trainees and program directors. such immediate feedback could be of great help to program directors to focus the learning process of their residents and monitor its progress. (J Vasc Surg 2001;33:854-60.)

Colon and rectal surgery: a true subspecialty.

The study contained herein used the database of the American Board of Colon and Rectal Surgery to demonstrate trends in colorectal practice from 1989 to 1996 and to compare the one-year technical experience of a colorectal resident with the five-year totals of a general surgery resident. Complete case lists from applicants for the American Board of Colon and Rectal Surgery's qualifying examination have been entered into a database. Similar data have been compiled from the Residency Review Committee for Surgery. From 1989 through 1996, 446,082 procedures have been listed by 417 colorectal residents, an average of 1,060 cases per resident. When contrasted with the operative experience of a general surgery resident, the colorectal resident performs substantially more anorectal operations, more endoscopic procedures, and more index abdominal operations in one year than the average general surgery resident performs in five years. When added to the required general surgery experience, one year of training in colorectal surgery trains a true subspecialist with unique expertise in the treatment of disorders of the colon, rectum, and anus.