American Board Of Surgery In-Training Examination Scores Research Articles

Flipping the Classroom in Residency Didactics Leads to Higher American Board of Surgery in-Training Examination Scores

Didactic education in General Surgery (GS) residency typically follows a nationally standardized curriculum; however, instructional format varies by institution. In recent years, GS residents at our institution expressed discontentment with weekly didactics and were not meeting their goals on the American Board of Surgery In-Training Examination (ABSITE). We sought to develop improvements in our didactic curriculum to increase resident satisfaction and ABSITE scores of GS junior residents (Jrs). In a quality improvement project, we changed the weekly didactic curriculum format from hour-long lectures in the 2018 to 2019 academic year (AY) to a partially-flipped classroom in the 2019 to 2020 AY, involving a 30-minute faculty-led presentation followed by 30 minutes of resident-led practice questions. The outcomes measured were ABSITE scores taken in 2019 and 2020 and resident opinions via an anonymous survey. This study was conducted at the University of Minnesota (Minneapolis, MN). The cohort for this study included all GS Jrs in our GS residency program, including postgraduate year (PGY) 1 nondesignated preliminary, PGY1 to 3 categorical GS residents, and residents in their lab time. Senior residents attended a separate didactics session. After curriculum changes, the ABSITE percentile scores for GS Jrs rose from 52% ± 5% to 66% ± 4% (p = 0.03). No categorical GS Jr scored <30% in 2020, compared to 20% (6/30) of categorical General Surgery residents in 2019. All residents preferred the new format overall and reported greater engagement in and preparation for didactics. After changing didactic education from hour-long lectures in the 2018 to 2019 AY to a flipped classroom model in the 2019 to 2020 AY including 30 minutes of faculty-led lecture followed by 30 minutes of resident-led practice questions, ABSITE scores and resident satisfaction at the University of Minnesota General Surgery Program improved.

SCORE-Based Simulated ABSITE Exam Performance as a Predictor of Performance on the ABSITE

The American Board of Surgery In-Training Examination (ABSITE) is a crucial, objective assessment of surgical knowledge during training. In 2014, the American Board of Surgery (ABS) announced the alignment of the ABSITE to the SCORE® (Surgical Council on Resident Education) Curriculum Outline for General Surgery Residency. We hypothesized that implementing a pre-ABSITE SCORE-based exam would help identify underperforming residents and provide early guidance to improve performance on the ABSITE. In October 2014, our university-based surgical residency program began administering a yearly comprehensive pre-ABSITE SCORE-based exam consisting of 225 to 250 multiple-choice questions selected from the SCORE question bank to all our general surgery residents, preliminary and categorical. The 4-hour exam addresses both clinical management (80%) and applied sciences (20%). Residents receive reports with their scores (percentage correct). Residents performing at less than 60% meet with the Program Director for discussion and formulation of a study plan. Correlational analysis was performed between resident ABSITE scores, pre-ABSITE SCORE-based exam scores, gender, resident status (preliminary vs. categorical), postgraduate year (PGY), and the United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores. A total of 244 exam scores (122 pre-ABSITE SCORE-based exams and 122 matched ABSITE) were completed by 51 residents at different PGY levels (32 PGY1, 32 PGY2, 20 PGY3, 19 PGY4, and 18 PGY5). Fifty-seven percent were males, 62% were categorical residents, and 38% were preliminary residents. October pre-ABSITE SCORE-based exam scores were compared to the subsequent January ABSITE scores. Categorical residents completed 101 (83%) of the January exams, while preliminary residents completed 21 (17%) of these paired exams. We found strong correlations between the correct percentage on ABSITE and pre-ABSITE SCORE-based scores (r = 0.637, p < 0.001), between the correct percentage on ABSITE and PGY (r = 0.688, p < 0.001), and between ABSITE and resident status (r = 0.462, p < 0.001). Additionally, there was a weak to negligible correlation between the correct percentage on ABSITE and resident gender (r = 0.274, p = 0.001), USMLE-2 (r = 0.12, p = 0.16), and USMLE-1 (r = 0.04, p = 0.653). Multiple regression analysis, with all predictors, was performed to predict the percentage score on the ABSITE and produced R2 0.58, with an adjusted R2 of 0.57, with a large size effect, p < 0.001. After controlling for the other variables, three factors reached statistical significance (p < 0.05): pre-ABSITE SCORE-based exam scores, PGY, and resident gender. We found a strong correlation between performance on the pre-ABSITE SCORE-based exam and performance on the ABSITE exam. Surgery residents are encouraged to start studying earlier and to utilize SCORE contents as outlined by the ABS in their study plan.

USMLE Step 1 Is Pass/Fail – Should the ABSITE Follow Suit?

The United States Medical Licensing Examination (USMLE) Step 1 was recently made pass/fail. This decision was controversial largely because of the reliance on USMLE Step 1 scores in resident selection. However, these scores do not correlate with resident ability. In this manuscript, we consider if the American Board of Surgery In-Training Examination (ABSITE) should be pass/fail as well. The ABSITE has been used for "high-stakes" purposes, such as preliminary resident advancement and prospective fellow evaluation, for which it was not intended. Moreover, similar to the USMLE Step 1 exam, ABSITE scores have demonstrated no correlation with clinical ability. A pass/fail ABSITE would return the exam to its original purpose and minimize an over-reliance on scores. Moving forward, new objective measures will need to be developed to assess surgical trainees in a more holistic manner.

Study Strategies for General Surgery Residents Preparing for the American Board of Surgery In-Training Examination: What to Keep, Discard, and Adopt

Undergraduate and graduate education research has stratified study strategies from low-utility to high-utility with respect to durable learning. The purpose of this study was to determine the prevalence of these evidence-based learning strategies among surgery residents in preparation for the American Board of Surgery In-Training Examination (ABSITE). A 23-item survey was administered during individual interviews. It assessed whether they had a study plan, and the average length and frequency of their independent study, both during the year and the month prior to the ABSITE. Data were also collected on their primary resources and study strategies. Residents rated their usage of those strategies based on a 5-point Likert scale. Maine Medical Center, an academic tertiary care center located in Portland, ME. All residents in the Department of Surgery. Residents (n= 23) intensified their preparation for the ABSITE in the month prior to the exam compared to the remainder of the year, adopting study plans (87% vs 61%, p = 0.53) and increasing the time spent studying (median, 420 vs 120 minutes per week, p < 0.001). Primary resources used were textbooks (65%), ABSITE review books (26%) and online question banks (9%). All residents (100%) often or always used testing, but fewer residents often or always used spacing (24%), both considered high-utility strategies. Most residents (60%) often or always used highlighting, considered a low-utility strategy. There were no relationships between study strategies and ABSITE scores. All residents use self-testing as a study strategy. Most underuse spacing and overuse highlighting. Further research is needed to establish the relationship between these study strategies and ABSITE scores.

USMLE Scores and Clinical Rotation Role in Predicting ABSITE Performance Among Surgery Interns

BackgroundThe American Board of Surgery In-training Examination (ABSITE) is an important marker of medical knowledge. It is unclear what factors predict or improve these scores. We evaluated demographics, United States Medical Licensing Examination (USMLE) step 1 and 2 scores, and surgical rotations during the intern year to determine if there were any correlations with the ABSITE performance. MethodsThis was a multicenter retrospective review during a 6-y period, investigating the correlation and association of demographics, USMLE scores, and types of rotations on the ABSITE percentile score of interns. Demographics included age, gender, race/ethnicity, U.S. versus international/foreign medical graduate for stratified analyses. Descriptive analysis was performed with ANOVA, correlation was evaluated with 95% confidence interval, and significance was defined as P < 0.05. ResultsComplete records obtained on 89 interns from six different general surgery programs over 6 y revealed that there was a significant correlation between USMLE 1 and 2 with the ABSITE. USMLE 2 correlation was the strongest (r = 0.44, 95% confidence interval = [0.25-0.60], P < 0.05). There was a significant difference in ABSITE performance (mean score difference of 17.3 percentile, P = 0.01) of interns who had an intensive care unit rotation before examination. Other surgical rotations were not associated with an ABSITE difference. Demographic factors such as age, gender, race/ethnicity, or medical graduate background status were not associated with ABSITE scores. ConclusionsUSMLE step 2 scores had a higher correlation with intern ABSITE performance. An intensive care unit rotation before taking the ABSITE was associated with a significant difference in their percentile scores. Demographic factors were not correlated with ABSITE performance.

Mid-Year Medical Knowledge Milestones and ABSITE Scores in First-Year Surgery Residents

Accreditation Council for Graduate Medical Education (ACGME) Surgery milestone ratings in the "Knowledge of Diseases and Conditions" (MK1) sub competency have been shown to correlate with American Board of Surgery In Training Examination (ABSITE) scores, and hypothesized to predict them. To better assess the predictive value of the MK1 milestone and avoid the potential bias caused by previous years' ABSITE scores, we designed a study including only first-year (PGY-1) residents and analyzed the correlation between their mid-year MK1 ratings and their scores in the ABSITE they took approximately a month later. De-identified United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores, mid-year MK1 milestone ratings and the subsequent ABSITE standard scores for the five academic years from 2014-2015 to 2018-2019 were collected and tabulated for 247 PGY-1 preliminary- and categorical-track residents from ten ACGME-accredited surgery residency programs. The mid-year rating of PGY-1 residents' MK1 was predictive of their subsequent first ABSITE score for the entire cohort and for the categorical residents' subset. Notably, controlling for all other independent predictors, each half-point increase in MK1 rating was associated with a 25-point increase in ABSITE score. Preliminary residents performed significantly worse on the ABSITE, and their scores did not correlate significantly with their MK1 ratings. The mid-year rating of PGY-1 residents' MK1 was predictive of their subsequent first ABSITE score for the entire cohort and for the categorical but not the preliminary residents. This finding suggests that evaluators correctly rated MK1 higher in the categorical residents who did perform better on the subsequent ABSITE.

Practice Makes Perfect: Introducing a Question Bank for ABSITE Preparation Improves Program Performance

The American Board of Surgery In-Training Examination (ABSITE) is an important predictor of passing the Qualifying Examination and a determinant of fellowship competitiveness. Study the impact of providing program-wide access to a commercially available question bank for ABSITE preparation. The surgery residency program purchased access to the TrueLearn question bank in 2018 A paired sample t test analysis compared the 2018 ABSITE percentage and percentile scores, prior to practice question bank access to 2019 ABSITE percentile scores. A simple linear regression analysis was calculated to predict improvement in percentage scores from 2018 to 2019 based on total number of practice questions as well as number of correct practice questions completed. Data were analyzed using SPSS. Among the residents utilizing practice questions with serial exam scores, the individual resident ABSITE percentage of correct questions showed a statistically significant improvement after introduction of the question bank from 2018 (mean = 68.7, standard deviation = 7.3) to 2019 (mean = 72.2, standard deviation = 7.2; t(35) = -4.529, p < 0.001). A statistically significant regression equation both linear (F(1,33) = 6.274, p = 0.017) and logarithmic (F(1,33) = 7.405, p =0.01) was found with an R2 of 0.160 and 0.183, respectively, for total number of practice questions completed, signifying that more completed practice questions correlated with a higher improvement in ABSITE percentage score. The improvement in residents' ABSITE percentage score increased by 3 ± 1 percentage point for each 100 practice questions completed from 2018 to 2019 (Figure 1). A significant regression equation was also found for improvement in percentage score among all residents (F (1,33) = 8.211, p = 0.007) with an R2 of 0.199 for the number of correct practice questions completed. Use of a commercial question bank improved overall ABSITE scores. More questions answered translated into improved performance. Percent correct on the practice questions also correlated strongly with performance. Programs seeking to improve scores may wish to provide access to a question bank.

Addition of Advanced Practice Providers May Improve ABSITE Scores Through Enhanced Resident Education

BackgroundResident work hour restrictions and required protected didactic time limit their ability to perform clinical duties and participate in structured education. Advanced practice providers (APPs) have previoulsy been shown to positively impact patients' outcomes and overall hospital costs. We describe a model in which nurse practitioners (NPs) improve resident education and American Board of Surgery In Training Examination (ABSITE) scores by providing support to our trauma and acute care surgery (ACS) service thereby protecting resident didactic time. Materials and methodsA new educational model aimed to improve ABSITE scores was created, increasing protected resident didactic time. The addition of three full-time NPs to the ACS service allowed implementation of this redesigned academic curriculum to be put into effect without neglecting patient or service-related responsibilities that were previously fulfilled by resident staff. Resident ABSITE results including standard score, percent correct, and percentile were compared before and after the educational changes were instituted. ResultsEleven residents' scores were included. For each ABSITE score, we used a mixed model with time and postgraduate year (PGY) level as fixed effects and subject ID as a random effect. The interaction term between PGY level and time was not significant and removed from the model. A significant main effect of PGY level and of time was then observed. A statistically significant improvement in ABSITE scores after intervention was observed across all the PGY levels. Standard score increased 77.3 points (P-value = 0.001), percent correct increased 5.9% (P-value = 0.002), and percentile increased 23.8 (P-value = 0.02). Following the educational reform, no residents scored below the 35th percentile. ConclusionsUtilization of NPs on our ACS service provided adequate service coverage, allowing the implementation of an educational reform increasing protected resident education time and improved ABSITE scores.

Beyond 250: A Comprehensive Strategy to Maximize the Operative Experience for Junior Residents

Among surgical educators, duty-hour restrictions led to concern regarding the adequacy of operative experience during residency, especially for junior residents. The American Board of Surgery recently instituted guidelines mandating "a minimum of 250 operations by the end of the PGY-2 year". A series of programmatic and institutional changes were implemented at our institution to augment the junior resident operative experience and to exceed compliance with this mandate. Operative data from Accreditation Council for Graduate Medical Education case logs for categorical and nondesignated preliminary interns from our large academic surgical residency were identified for 5 consecutive academic years, 2011 until 2016. American Board of Surgery In-Training Examination (ABSITE) scores were collected anonymously. The program systematically instituted the following changes: night float minimization, identification of new surgical opportunities, augmenting use of midlevel care providers, identification of rotations with suboptimal operative experiences, maximizing rotations with involvement of junior residents in the operating room, and systematic review of junior case logs. After implementation, average total cases for residents completing postgraduate year (PGY)-2 increased from 176 to 330 (p < 0.001). Specifically, there was an 18% increase for interns (p = 0.059) and a 118% increase for PGY-2 residents (p < 0.001). There were statistically significant increases in skin and soft tissue cases, vascular cases, endoscopy, and complex laparoscopic cases. Average case volumes for senior residents did not change. Night float time was significantly decreased (5.7 vs 3.4 wk; p = 0.04). ABSITE scores were not significantly changed during this time. Before implementation of these interventions, our program would have had 0% compliance with the 250 junior resident case rule. Within 12 months of implementation, total case volumes for residents completing PGY-2 increased by 88%-exceeding minimum standards. Overall, 100% programmatic compliance was achieved. Our program's experience exemplifies how mandates from the American Board of Surgery can lead to programmatic changes that improve the experience of surgical house officers.

Emotional intelligence in surgery is associated with resident job satisfaction

BackgroundEmotional intelligence (EI) has been associated with improved work performance and job satisfaction in several industries. We evaluated whether EI was associated with higher measures of work performance and job satisfaction in surgical residents. MethodsWe distributed the validated Trait EI Questionnaire and job satisfaction survey to all general surgery residents at a single institution in 2015. EI and job satisfaction scores were compared with resident performance using faculty evaluations of clinical competency–based surgical milestones and standardized test scores including the United States Medical Licensing Examination (USMLE) and American Board of Surgery In-Training Examination (ABSITE). Statistical comparison was made using Pearson correlation and simple linear regression adjusting for postgraduate year level. ResultsThe survey response rate was 68.9% with 31 resident participants. Global EI was associated with scores on USMLE Step 2 (r = 0.46, P = 0.01) and Step 3 (r = 0.54, P = 0.01) but not ABSITE percentile scores (r = 0.06, P = 0.77). None of the 16 surgical milestone scores were significantly associated with global EI or EI factors before or after adjustment for postgraduate level. Global EI was associated with overall job satisfaction (r = 0.37, P = 0.04). Of the facets of job satisfaction, global EI was significantly associated with views of supervision (r = 0.42, P = 0.02) and nature of work (r = 0.41, P = 0.02). ConclusionsEI was associated with job satisfaction and USMLE performance but not ACGME competency–based milestones or ABSITE scores. EI may be an important factor for fulfillment in surgical training that is not currently captured with traditional in-training performance measures.

Effects of Technological Advances in Surgical Education on Quantitative Outcomes From Residency Programs

The purpose of this article is to review the literature on current technology for surgical education and to evaluate the effect of technological advances on the Accreditation Council of Graduate Medical Education (ACGME) Core Competencies, American Board of Surgery In-Training Examination (ABSITE) scores, and American Board of Surgery (ABS) certification. A literature search was obtained from MEDLINE via PubMed.gov, ScienceDirect.com, and Google Scholar on all peer-reviewed studies published since 2003 using the following search queries: technology for surgical education, simulation-based surgical training, simulation-based nontechnical skills (NTS) training, ACGME Core Competencies, ABSITE scores, and ABS pass rate. Our initial search list included the following: 648 on technology for surgical education, 413 on simulation-based surgical training, 51 on simulation-based NTS training, 78 on ABSITE scores, and 33 on ABS pass rate. Further, 42 articles on technological advances for surgical education met inclusion criteria based on their effect on ACGME Core Competencies, ABSITE scores, and ABS certification. Systematic review showed that 33 of 42 and 26 of 42 publications on technological advances for surgical education showed objective improvements regarding patient care and medical knowledge, respectively, whereas only 2 of 42 publications showed improved ABSITE scores, but none showed improved ABS pass rates. Improvements in the other ACGME core competencies were documented in 14 studies, 9 of which were on simulation-based NTS training. Most of the studies on technological advances for surgical education have shown a positive effect on patient care and medical knowledge. However, the effect of simulation-based surgical training and simulation-based NTS training on ABSITE scores and ABS certification has not been assessed. Studies on technological advances in surgical education and simulation-based NTS training showing quantitative evidence that surgery residency program objectives are achieved are still needed.

Effect of Process Changes in Surgical Training on Quantitative Outcomes From Surgery Residency Programs

The purpose of this article is to review the literature on process changes in surgical training programs and to evaluate their effect on the Accreditation Council of Graduate Medical Education (ACGME) Core Competencies, American Board of Surgery In-Training Examination (ABSITE) scores, and American Board of Surgery (ABS) certification. A literature search was obtained from MEDLINE via PubMed.gov, ScienceDirect.com, Google Scholar on all peer-reviewed studies published since 2003 using the following search queries: surgery residency training, surgical education, competency-based surgical education, ACGME core competencies, ABSITE scores, and ABS pass rate. Our initial search list included 990 articles on surgery residency training models, 539 on competency-based surgical education, 78 on ABSITE scores, and 33 on ABS pass rate. Overall, 31 articles met inclusion criteria based on their effect on ACGME Core Competencies, ABSITE scores, and ABS certification. Systematic review showed that 5/31, 19/31, and 6/31 articles on process changes in surgical training programs had a positive effect on patient care, medical knowledge, and ABSITE scores, respectively. ABS certification was not analyzed. The other ACGME core competencies were addressed in only 6 studies. Several publications on process changes in surgical training programs have shown a positive effect on patient care, medical knowledge, and ABSITE scores. However, the effect on ABS certification, and other quantitative outcomes from residency programs, have not been addressed. Studies on education strategies showing evidence that residency program objectives are being achieved are still needed. This article addresses the 6 ACGME Core Competencies.

Reading Habits of General Surgery Residents and Association With American Board of Surgery In-Training Examination Performance.

Few large-scale studies have quantified and characterized the study habits of surgery residents. However, studies have shown an association between American Board of Surgery In-Training Examination (ABSITE) scores and subsequent success on the American Board of Surgery Qualifying and Certifying examinations. To identify the quantity of studying, the approach taken when studying, the role that ABSITE preparation plays in resident reading, and factors associated with ABSITE performance. An anonymous 39-item questionnaire including demographic information, past performance on standardized examinations, reading habits, and study sources during the time leading up to the 2014 ABSITE and opinions pertaining to the importance of the ABSITE was administered August 1, 2014, to August 25, 2014, to 371 surgery residents in 15 residency programs nationwide. Scores from the 2014 ABSITE. A total of 273 residents (73.6%) responded to the survey. Seven respondents did not provide their January 2014 ABSITE score, leaving 266 for statistical analysis. Most respondents were male (162 of 266 [60.9%]), with a mean (SD) age of 29.8 (2.6) years. The median number of minutes spent studying per month was 240 (interquartile range, 120-600 minutes) for patient care or clinical duties and 120 for the ABSITE (interquartile range, 30-360 minutes). One hundred sixty-four of 266 respondents (61.7%) reported reading consistently throughout the year for patient care or clinical duties. With respect to ABSITE preparation, 72 of 266 residents (27.1%) reported reading consistently throughout the year, while 247 of 266 residents (92.9%) reported preparing between 1 and 8 weeks prior to the examination. Univariate analysis (with results reported as effect on median ABSITE percentile scores [95% CIs]) identified the following factors as positively correlated with ABSITE scores: prior United States Medical Licensing Examination (USMLE) 1 and 2 scores (per 1-point increase: USMLE 1, 0.1 [0.02-0.14], P = .03; USMLE 2, 0.3 [0.19-0.44], P < .001), prior Medical College Admission Test (MCAT) scores (per 1-point increase, 1.2 [1.3-2.0]; P = .002), high opinion of ABSITE significance (P < .001), surgical textbook use (11 [6-16]; P = .02), daily studying (13 [4-23]; P = .02), and high satisfaction with study materials (P < .001). On multivariable analysis, USMLE 2 score (per 1-point increase, 0.4 [0.2-0.6]; P < .001), MCAT score (0.6 [0.2-1.0]; P = .003), opinion of ABSITE significance (9.2 [6.9-11.6]; P < .001), and having an equal focus on patient care and ABSITE preparation during study (6.1 [0.6-11.5]; P = .03) were identified as positive predictors of ABSITE performance. Most residents reported reading consistently for patient care throughout the year. Daily studying and textbook use were associated with higher ABSITE scores on univariate analysis. Scores on the USMLE 2 and MCAT, as well as resident attitude regarding the importance of the ABSITE results, were independent predictors of ABSITE performance.

Predicting Academic Performance in Surgical Training

During surgical residency, trainees are expected to master all the 6 competencies specified by the ACGME. Surgical training programs are also evaluated, in part, by the residency review committee based on the percentage of graduates of the program who successfully complete the qualifying examination and the certification examination of the American Board of Surgery in the first attempt. Many program directors (PDs) use the American Board of Surgery In-Training Examination (ABSITE) as an indicator of future performance on the qualifying examination. Failure to meet an individual program's standard may result in remediation or a delay in promotion to the next level of training. Remediation is expensive in terms of not only dollars but also resources, faculty time, and potential program disruptions. We embarked on an exploratory study to determine if residents who might be at risk for substandard performance on the ABSITE could be identified based on the individual resident's behavior and motivational characteristics. If such were possible, then PDs would have the opportunity to be proactive in developing a curriculum tailored to an individual resident, providing a greater opportunity for success in meeting the program's standards. Overall, 7 surgical training programs agreed to participate in this initial study and residents were recruited to voluntarily participate. Each participant completed an online assessment that characterizes an individual's behavioral style, motivators, and Acumen Index. Residents completed the assessment using a code name assigned by each individual PD or their designee. Assessments and the residents' 2013 ABSITE scores were forwarded for analysis using only the code name, thus insuring anonymity. Residents were grouped into those who took the junior examination, senior examination, and pass/fail categories. A passing score of ≥70% correct was chosen a priori. Correlations were performed using logistic regression and data were also entered into a neural network (NN) to develop a model that would explain performance based on data obtained from the TriMetrix assessments. A total of 117 residents' TriMetrix and ABSITE scores were available for analysis. They were divided into 2 groups of 64 senior residents and 53 junior residents. For each group, the pass/fail criteria for the ABSITE were set at 70 and greater as passing and 69 and lower as failing. Multiple logistic regression analysis was complete for pass/fail vs the TriMetrix assessments. For the senior data group, it was found that the parameter Theoretical correlates with pass rate (p < 0.043, B = -0.513, exp(B) = 0.599), which means increasing theoretical scores yields a decreasing likelihood of passing in the examination. For the junior data, the parameter Internal Role Awareness correlated with pass/fail rate (p < 0.004, B = 0.66, exp(B) = 1.935), which means that an increasing Internal Role Awareness score increases the likelihood of a passing score. The NN was able to be trained to predict ABSITE performance with surprising accuracy for both junior and senior residents. Behavioral, motivational, and acumen characteristics can be useful to identify residents "at risk" for substandard performance on the ABSITE. Armed with this information, PDs have the opportunity to intervene proactively to offer these residents a greater chance for success. The NN was capable of developing a model that explained performance on the examination for both the junior and the senior examinations. Subsequent testing is needed to determine if the NN is a good predictive tool for performance on this examination.

An Evidence-Based Medicine Curriculum Improves General Surgery Residents' Standardized Test Scores in Research and Statistics.

The application of evidence-based medicine to patient care requires unique skills of the physician. Advancing residents' abilities to accurately evaluate the quality of evidence is built on understanding of fundamental research concepts. The American Board of Surgery In-Training Examination (ABSITE) provides a relevant measure of surgical residents' knowledge of research design and statistics. We implemented a research education curriculum in an independent academic medical center general residency program, and assessed the effect on ABSITE scores. The curriculum consisted of five 1-hour monthly research and statistics lectures. The lectures were presented before the 2012 and 2013 examinations. Forty residents completing ABSITE examinations from 2007 to 2013 were included in the study. Two investigators independently identified research-related item topics from examination summary reports. Correct and incorrect responses were compared precurriculum and postcurriculum. Regression models were calculated to estimate improvement in postcurriculum scores, adjusted for individuals' scores over time and postgraduate year level. Residents demonstrated significant improvement in postcurriculum examination scores for research and statistics items. Correct responses increased 27% (P < .001). Residents were 5 times more likely to achieve a perfect score on research and statistics items postcurriculum (P < .001). Residents at all levels demonstrated improved research and statistics scores after receiving the curriculum. Because the ABSITE includes a wide spectrum of research topics, sustained improvements suggest a genuine level of understanding that will promote lifelong evaluation and clinical application of the surgical literature.

Surgical resident accuracy in predicting their ABSITE score.

Background:The American Board of Surgery In-Training Examination (ABSITE) is given to all surgical residents as an assessment tool for residents and their programs in preparation for the American Board of Surgery qualifying and certifying examinations. Our objective was to ascertain how well surgical residents could predict their percentile score on the ABSITE using two predictor measures before and one immediately after the examination was completed.Methods:A survey was given to surgical residents in postgraduate year(s) (PGY) 2 through 5 as well as to research residents in November and December 2011, and immediately after the examination in January 2012, to ascertain their predicted ABSITE scores. Thirty-one general surgery residents were measured consisting of PGY-2 (22%), PGY-3 (19.4%), PGY-4 (19.4%), and PGY-5 (12.9%), and research residents 25.8%.Results:Mean prediction scores were consistently higher than actual examination scores for both junior and senior examination takers, with senior examination predictions exhibiting the highest proportion of variation on the actual examination score. Stratified linear regression analysis showed little predictive significance of all 3 examination predictions and actual score, except for the senior examination predictions in November 2011 (t test = 2.521, P = .027). We found no statistically significant difference in the proportion of residents overestimating or underestimating their predicted score. Secondary analysis using a linear regression model shows that 2011 scores were a statistically significant predictor of 2012 scores (overall F = 13.258, P = .001, R2 = 0.31) for both junior and senior examinations.Conclusion:General surgery residents were not able to accurately predict their ABSITE score; however, the previous year's actual scores were found to have the most predictive value of the next year's actual scores.

Achievement in Surgical Residency: Are Objective Measures of Performance Associated With Awards Received in Final Years of Training?

For the past 15 years at our institution's general surgery residency program, 3 of the senior residents have been chosen to be awarded either (1) Best Resident in Research, (2) Best Resident in Teaching, or (3) Best Resident Overall. Considering that these awards serve as data representing outstanding performance as surgical residents, the objective of this study was to determine the association between receiving one of these awards and objective measures of performance. Individual files were reviewed for the 103 residents who graduated from our institution's general surgery program from 1994 to 2010. These data were studied as a whole, and then divided into an award-winning group and a non-award winning group and subsequently compared across several objective parameters, including The United States Medical Licensing Examination (USMLE) scores, American Board of Surgery In-Training Examination (ABSITE) scores, first-time American Board of Surgery Certifying and Qualifying Examination pass rates, Alpha Omega Alpha membership status, and number of research years, using a logistic regression model. Overall, 103 residents completed their general surgery residency training at our institution from 1994 to 2010, and of these residents, 16 (16%) received the Best Resident in Research award, 15 (16%) received the Best Resident in Teaching award, and 17 (17%) received the Best Resident Overall award in their final years of training. Compared with those who did not receive an award, a hypothesis-based one-tailed test revealed that award winners had a significantly lower median USMLE Step 1 scores (p = 0.04) and marginally lower median USMLE Step 2 scores (p = 0.05). Alpha Omega Alpha membership status, median ABSITE percent correct overall, first-time American Board of Surgery examination pass rates, and number of research years during residency were not significantly different between the 2 groups. Many factors contribute to success during general surgery residency. Our study showed that higher USMLE and ABSITE scores were not associated with receiving top awards in final years of training at one institution over 15 years.

A Survey of Study Habits of General Surgery Residents

To understand the study habits of general surgery residents as well as their motivating factors for study. A survey was mailed to general surgical residents. Performance on the American Board of Surgery In-Training Examination (ABSITE) was correlated with reported study habits using the Pearson's correlation coefficient. Massachusetts General Hospital and Brigham and Women's Hospital, 2 urban tertiary referral academic training institutions in Boston, Massachusetts. Fifty-eight general surgical residents of all training levels (including research). On average, surgical residents studied for 3 ± 1 days per week, the average duration of each study session being 1.3 ± 0.6 hours. The average total number of study hours per week was 3.4 ± 2.3. There were strong positive correlations between increased study frequency and high overall ABSITE score (Pearson's r = 0.339; p = 0.02) and between the total number of study hours per week and high overall ABSITE score (Pearson's r = 0.423; p < 0.005). Only 10% and 3% reported complete satisfaction with current study materials and routine, respectively. Most residents (96%) reported a willingness to try a new type of study method and 75% were willing to enroll in a trial comparing study methods. Increased study frequency and overall increased study duration are positively correlated with ABSITE performance. Dissatisfaction with current study materials and study routine is high, as is willingness to adopt new methods and enroll in investigational trials comparing study methods.

Year One Outcomes Assessment of a Masters Suturing and Knot-Tying Program for Surgical Interns

We implemented and evaluated the outcomes of a proficiency-based program to enhance basic suturing and knot-tying skills in surgical interns. A suturing and knot-tying "masters program" was implemented for 9 surgical interns. Assessment consisted of timed and videotaped technical performance (TP) of 6 suturing (simple interrupted, subcuticular, and vertical mattress) and tying tasks (1-handed, 2-handed, and tie on pass) preinstruction (baseline) and at 4 months. Feedback and remediation were done after 4 months, and interns were retested until goals were met. TP was scored on a 1-5 scale (1 = novice, 3 = proficient, and 5 = expert). The results were compared with 6 end of PGY2 year residents and rotation evaluations, case numbers, and American Board of Surgery In-Training Examination (ABSITE) scores. The data are mean ± standard deviation (SD); statistical analysis was by 1-way ANOVA, Kruskal-Wallis test, and Pearson correlation. The total combined task times improved significantly from initial (1289 ± 301 seconds) to final assessment (770 ± 139 seconds, p = 0.0003) and between 4-month assessment (1092 ± 253 seconds) and final assessment (p = 0.0237), but not baseline to 4-month assessment (p = 0.213). Final TP scores were improved significantly (baseline, 1.87 ± 0.81; 4-month assessment, 2.63 ± 0.75; and final assessment 3.51 ± 0.39 [p < 0.001]. All interns reached proficiency on 6/6 tasks but not until 10.1 months (range, 4-12 months) and 2.4 assessment sessions (range, 1-3). No significant differences in final total task times or TP scores were found between interns (770 ± 139 seconds and TP, 3.51 ± 0.39) and end of PGY-2 residents (677 ± 109 seconds and TP, 3.6 ± 0.43). The total task time and rotation technical comments were associated significantly, but TP scores, case numbers, and ABSITE scores were not. Training, feedback, and remediation are necessary to reach proficiency in basic suturing and knot tying. This program provides an objective assessment of interns' skills not discernible by conventional measures.

Predicting Performance on the American Board of Surgery Qualifying and Certifying Examinations

We sought to determine whether US Medical Licensing Examination (USMLE) Step 1 score, American Board of Surgery (ABS) In-Training Examination (ABSITE) score, and other variables are associated with failing the ABS qualifying and certifying examinations. Identifying such factors may assist in the early implementation of an academic intervention for at-risk residents. Retrospective review. Seventeen general surgery training programs in the western United States. Six hundred seven residents who graduated in 2000-2007. First-time pass rates on the qualifying and certifying examinations, US vs non-US medical school graduation, USMLE Steps 1 and 2 scores, ABSITE scores, operative case volume, fellowship training, residency program type, and mandatory research. The first-time qualifying and certifying examination pass rates for the 607 graduating residents were 78% and 74%, respectively. On multivariable analysis, scoring below the 35th percentile on the ABSITE at any time during residency was associated with an increased risk of failing both examinations (odds ratio, 0.23 [95% confidence interval, 0.08-0.68] for the qualifying examination and 0.35 [0.20-0.61] for the certifying examination), as was scoring less than 200 on the USMLE Step 1 (0.36 [0.21-0.62] for the qualifying examination and 0.62 [0.42-0.93] for the certifying examination). A mandatory research year was associated with an increased likelihood of passing the certifying examination (odds ratio, 3.3 [95% confidence interval, 1.6-6.8]). Residents who are more likely to fail the ABS qualifying and certifying examinations can be identified by a low USMLE Step 1 score and by poor performance on the ABSITE at any time during residency. These findings support the use of the USMLE Step 1 score in the surgical residency selection process and a formal academic intervention for residents who perform poorly on the ABSITE.