In-Training Examination Scores Research Articles

Effects of Technological Advances in Surgical Education on Quantitative Outcomes From Residency Programs

The purpose of this article is to review the literature on current technology for surgical education and to evaluate the effect of technological advances on the Accreditation Council of Graduate Medical Education (ACGME) Core Competencies, American Board of Surgery In-Training Examination (ABSITE) scores, and American Board of Surgery (ABS) certification. A literature search was obtained from MEDLINE via PubMed.gov, ScienceDirect.com, and Google Scholar on all peer-reviewed studies published since 2003 using the following search queries: technology for surgical education, simulation-based surgical training, simulation-based nontechnical skills (NTS) training, ACGME Core Competencies, ABSITE scores, and ABS pass rate. Our initial search list included the following: 648 on technology for surgical education, 413 on simulation-based surgical training, 51 on simulation-based NTS training, 78 on ABSITE scores, and 33 on ABS pass rate. Further, 42 articles on technological advances for surgical education met inclusion criteria based on their effect on ACGME Core Competencies, ABSITE scores, and ABS certification. Systematic review showed that 33 of 42 and 26 of 42 publications on technological advances for surgical education showed objective improvements regarding patient care and medical knowledge, respectively, whereas only 2 of 42 publications showed improved ABSITE scores, but none showed improved ABS pass rates. Improvements in the other ACGME core competencies were documented in 14 studies, 9 of which were on simulation-based NTS training. Most of the studies on technological advances for surgical education have shown a positive effect on patient care and medical knowledge. However, the effect of simulation-based surgical training and simulation-based NTS training on ABSITE scores and ABS certification has not been assessed. Studies on technological advances in surgical education and simulation-based NTS training showing quantitative evidence that surgery residency program objectives are achieved are still needed.

Effect of Process Changes in Surgical Training on Quantitative Outcomes From Surgery Residency Programs

The purpose of this article is to review the literature on process changes in surgical training programs and to evaluate their effect on the Accreditation Council of Graduate Medical Education (ACGME) Core Competencies, American Board of Surgery In-Training Examination (ABSITE) scores, and American Board of Surgery (ABS) certification. A literature search was obtained from MEDLINE via PubMed.gov, ScienceDirect.com, Google Scholar on all peer-reviewed studies published since 2003 using the following search queries: surgery residency training, surgical education, competency-based surgical education, ACGME core competencies, ABSITE scores, and ABS pass rate. Our initial search list included 990 articles on surgery residency training models, 539 on competency-based surgical education, 78 on ABSITE scores, and 33 on ABS pass rate. Overall, 31 articles met inclusion criteria based on their effect on ACGME Core Competencies, ABSITE scores, and ABS certification. Systematic review showed that 5/31, 19/31, and 6/31 articles on process changes in surgical training programs had a positive effect on patient care, medical knowledge, and ABSITE scores, respectively. ABS certification was not analyzed. The other ACGME core competencies were addressed in only 6 studies. Several publications on process changes in surgical training programs have shown a positive effect on patient care, medical knowledge, and ABSITE scores. However, the effect on ABS certification, and other quantitative outcomes from residency programs, have not been addressed. Studies on education strategies showing evidence that residency program objectives are being achieved are still needed. This article addresses the 6 ACGME Core Competencies.

Development and implementation of an asynchronous emergency medicine residency curriculum using a web-based platform.

The Residency Review Committee in Emergency Medicine requires residency programs to deliver at least 5hours of weekly didactics. Achieving at least a 70% average attendance rate per resident is required for residency program accreditation, and is used as a benchmark for residency graduation in our program. We developed a web-based, asynchronous curriculum to replace 1hour of synchronous didactics, and hypothesized that the curriculum would be feasible to implement, well received by learners, and improve conference participation. This paper describes the feasibility and learner acceptability of a longitudinal asynchronous curriculum, and describes its impact on postgraduate year-1(PGY-1) resident conference participation and annual in-training examination scores. Using formal curriculum design methods, we developed modules and paired assessment exercises to replace 1hour of weekly didactics. We measured feasibility (development and implementation time and costs) and learner acceptability (measured on an anonymous survey). We compared pre- and post-intervention conference participation and in-service training examination scores using a two sample t test. The asynchronous curriculum proved feasible to develop and implement. PGY-1 resident conference participation improved compared to the pre-intervention year (85.6 vs. 62%; 95% CI 0.295-0.177; p<0.001). We are unable to detect a difference between in-training examination results in either the PGY-1 group or across all residents by the introduction of this intervention. 18/31 (58%) residents completed the post-intervention survey. 83% reported satisfaction with curriculum changes. Strengths of the curriculum included clarity and timeliness of assignments. Weaknesses included technical difficulties with the online platform. Our curriculum is feasible to develop and implement. Despite technical difficulties, residents report high satisfaction with this new curriculum. Among PGY-1 residents there is improved conference participation compared to the prior year.

Medical Knowledge Assessment by Hematology and Medical Oncology In-Training Examinations Are Better Than Program Director Assessments at Predicting Subspecialty Certification Examination Performance.

The Accreditation Council for Graduate Medical Education's Next Accreditation System requires training programs to demonstrate that fellows are achieving competence in medical knowledge (MK), as part of a global assessment of clinical competency. Passing American Board of Internal Medicine (ABIM) certification examinations is recognized as a metric of MK competency. This study examines several in-training MK assessment approaches and their ability to predict performance on the ABIM Hematology or Medical Oncology Certification Examinations. Results of a Hematology In-Service Examination (ISE) and an Oncology In-Training Examination (ITE), program director (PD) ratings, demographic variables, United States Medical Licensing Examination (USMLE), and ABIM Internal Medicine (IM) Certification Examination were compared. Stepwise multiple regression and logistic regression analyses evaluated these assessment approaches as predictors of performance on the Hematology or Medical Oncology Certification Examinations. Hematology ISE scores were the strongest predictor of Hematology Certification Examination scores (β = 0.41) (passing odds ratio [OR], 1.012; 95% confidence interval [CI], 1.008-1.015), and the Oncology ITE scores were the strongest predictor of Medical Oncology Certification Examination scores (β = 0.45) (passing OR, 1.013; 95% CI, 1.011-1.016). PD rating of MK was the weakest predictor of Medical Oncology Certification Examination scores (β = 0.07) and was not significantly predictive of Hematology Certification Examination scores. Hematology and Oncology ITEs are better predictors of certification examination performance than PD ratings of MK, reinforcing the effectiveness of ITEs for competency-based assessment of MK.

Orthopedic Residency: Are Duty Hours Predictive of Performance?

This study examines the relationship between self-recorded resident work hours and Orthopedic In-training Examination (OITE) scores, resident clinical performance, and American Board of Orthopedic Surgery pass rates. The hypothesis of this study is that increasing duty hours would have a positive correlation with clinical and OITE performance. Total duty hours and recorded operating room hours from a single orthopedic residency program were extracted from 2006 to 2012. During the same time span, OITE scores, resident clinical scores from the E-Valuation system, and American Board of Orthopedic Surgery pass rates were collected. The correlation between the variables was assessed using the Pearson correlation coefficient's precision statistic. A large public tertiary academic center in the upper Midwestern United States. A total of 82 orthopedic surgery residents over 7 years. A total of 82 residents were matriculated between 2006 and 2012. The average weekly recorded duty hours were as follows: postgraduate year 2 (PGY2) = 60 hours/week (Standard Deviation (SD) ± 4), PGY3 = 59 hours/week (SD ± 5), PGY4 = 51 hours/week (SD ± 4), PGY5 = 49 hours/week (SD ± 3). There was significant variability in the average number of hours worked among residents (range: 2128-3753h/y) for the full academic year. The OITE scores and the work hours were found to be independent of each other (ρ = 0.017, p = 0.825), and no correlation was found between OITE scores and the resident E-value scores (ρ = 0.071, p = 0.34). Residents spent 36% to 48% of their time in the operating room. Second year residents logging more hours scored higher on faculty evaluation of overall competency (ρ = 0.31, p = 0.035). Faculty assessment of technical skills had a positive correlation with operating room duty hours for PGY5 class (ρ = 0.346, p = 0.025). A large variation in duty hours exists between resident-logged duty hours. No correlation exists between in-training scores and duty hours. There is a positive correlation between senior resident operating room hours and technical skill scores.

The Continuing Quest for Meaningful Faculty Evaluations of Residents.

In this issue of JAMA Surgery, Ray et al1 report their findings at their institution, the University of Miami, Miami, Florida, on the (non)relationship betweenperformance on theAmerican Board of Surgery In-Training Examination (ABSITE) and themean scores of faculty evaluations. Several thoughts and conjectures stem from their study. First, the authors cite their use of the measures from the General Surgery Milestone Project as their base for evaluations in the New Innovations Residency Management Suite online system for the studyperiod 2011 to 2014; however, the General Surgery Milestone Project was under development in 2011 and only became effective for general surgery in July 2014. The 6-monthly residentperformance reviewand conversation with the programdirector or associate programdirector have been part of the Accreditation Council for Graduate Medical Education Surgical Residency Review Committee program requirements for decades. Second, their faculty evaluations were nonanonymous. Manyof our own faculty, citing that theywill beworkingwith residents over years, are leery of providing less than laudatory comments (constructive criticism)evenanonymouslybecause theybelieve residents can identify the sourceof the lessthan-A+ evaluation. They are not keen on nonanonymous evaluations.Ofnote,New Innovationsnowprovides away for faculty to make a “not for the resident to see” comment in a supplemental box at the bottom of the evaluation form. The residents in the studybyRay and colleaguesweremostly perceived as “A” performers (mean evaluation score, 92.24; median evaluation score, 92.65), as shown in the histogram (Figure 1A) with all annual evaluation scores in the top quartile, while ABSITE scores varied more. Finally, one can argue that the ABSITE and the faculty evaluationsmeasure different domains, not unlike the qualifying examination and the certifying examination for the American Board of Surgery, and that they complement one another rather than necessarily predict or correlate with one another. Some irascible residents ace theABSITE,while other residents with excellent professional demeanor and people skillsmaystrugglewithmultiple-choicequestions.Booksmarts and street smarts do not have an r value of 1, nor domultiplechoice question skills and the soft competencies of interpersonal and communication skills and professionalism. Should we expect an examination that measures 1 portion of 1 domain,medical knowledge, to correlatewitha subjectiveevaluation measuring 6 or 7 competency domains? Thisarticlechallengesus to improveourevaluationsof residentperformance,aworthyquestofanelusive target.Thenext major hurdle is evaluation of technical and operative skills.

Association Between American Board of Surgery In-Training Examination Scores and Resident Performance.

The American Board of Surgery In-Training Examination (ABSITE) is designed to measure progress, applied medical knowledge, and clinical management; results may determine promotion and fellowship candidacy for general surgery residents. Evaluations are mandated by the Accreditation Council for Graduate Medical Education but are administered at the discretion of individual institutions and are not standardized. It is unclear whether the ABSITE and evaluations form a reasonable assessment of resident performance. To determine whether favorable evaluations are associated with ABSITE performance. Cross-sectional analysis of preliminary and categorical residents in postgraduate years (PGYs) 1 through 5 training in a single university-based general surgery program from July 1, 2011, through June 30, 2014, who took the ABSITE. Evaluation overall performance and subset evaluation performance in the following categories: patient care, technical skills, problem-based learning, interpersonal and communication skills, professionalism, systems-based practice, and medical knowledge. Passing the ABSITE (≥30th percentile) and ranking in the top 30% of scores at our institution. The study population comprised residents in PGY 1 (n = 44), PGY 2 (n = 31), PGY 3 (n = 26), PGY 4 (n = 25), and PGY 5 (n = 24) during the 4-year study period (N = 150). Evaluations had less variation than the ABSITE percentile (SD = 5.06 vs 28.82, respectively). Neither annual nor subset evaluation scores were significantly associated with passing the ABSITE (n = 102; for annual evaluation, odds ratio = 0.949; 95% CI, 0.884-1.019; P = .15) or receiving a top 30% score (n = 45; for annual evaluation, odds ratio = 1.036; 95% CI, 0.964-1.113; P = .33). There was no difference in mean evaluation score between those who passed vs failed the ABSITE (mean [SD] evaluation score, 91.77 [5.10] vs 93.04 [4.80], respectively; P = .14) or between those who received a top 30% score vs those who did not (mean [SD] evaluation score, 92.78 [4.83] vs 91.92 [5.11], respectively; P = .33). There was no correlation between annual evaluation score and ABSITE percentile (r(2) = 0.014; P = .15), percentage correct unadjusted for PGY level (r(2) = 0.019; P = .09), or percentage correct adjusted for PGY level (r(2) = 0.429; P = .91). Favorable evaluations do not correlate with ABSITE scores, nor do they predict passing. Evaluations do not show much discriminatory ability. It is unclear whether individual resident evaluations and ABSITE scores fully assess competency in residents or allow comparisons to be made across programs. Creation of a uniform evaluation system that encompasses the necessary subjective feedback from faculty with the objective measure of the ABSITE is warranted.

Correlation of the National Board of Medical Examiners Emergency Medicine Advanced Clinical Examination Given in July to Intern American Board of Emergency Medicine in-training Examination Scores: A Predictor of Performance?

IntroductionThere is great variation in the knowledge base of Emergency Medicine (EM) interns in July. The first objective knowledge assessment during residency does not occur until eight months later, in February, when the American Board of EM (ABEM) administers the in-training examination (ITE). In 2013, the National Board of Medical Examiners (NBME) released the EM Advanced Clinical Examination (EM-ACE), an assessment intended for fourth-year medical students. Administration of the EM-ACE to interns at the start of residency may provide an earlier opportunity to assess the new EM residents’ knowledge base. The primary objective of this study was to determine the correlation of the NBME EM-ACE, given early in residency, with the EM ITE. Secondary objectives included determination of the correlation of the United States Medical Licensing Examination (USMLE) Step 1 or 2 scores with early intern EM-ACE and ITE scores and the effect, if any, of clinical EM experience on examination correlation.MethodsThis was a multi-institutional, observational study. Entering EM interns at six residencies took the EM-ACE in July 2013 and the ABEM ITE in February 2014. We collected scores for the EM-ACE and ITE, age, gender, weeks of clinical EM experience in residency prior to the ITE, and USMLE Step 1 and 2 scores. Pearson’s correlation and linear regression were performed.ResultsSixty-two interns took the EM-ACE and the ITE. The Pearson’s correlation coefficient between the ITE and the EM-ACE was 0.62. R-squared was 0.5 (adjusted 0.4). The coefficient of determination was 0.41 (95% CI [0.3–0.8]). For every increase of one in the scaled EM-ACE score, we observed a 0.4% increase in the EM in-training score. In a linear regression model using all available variables (EM-ACE, gender, age, clinical exposure to EM, and USMLE Step 1 and Step 2 scores), only the EM-ACE score was significantly associated with the ITE (p<0.05). We observed significant colinearity among the EM-ACE, ITE and USMLE scores. Gender, age and number of weeks of EM prior to the ITE had no effect on the relationship between EM-ACE and the ITE.ConclusionGiven early during intern year, the EM-ACE score showed positive correlation with ITE. Clinical EM experience prior to the in-training exam did not affect the correlation.

Performance on the adult rheumatology in-training examination and relationship to outcomes on the rheumatology certification examination.

The American College of Rheumatology (ACR) Adult Rheumatology In-Training Examination (ITE) is a feedback tool designed to identify strengths and weaknesses in the content knowledge of individual fellows-in-training and the training program curricula. We determined whether scores on the ACR ITE, as well as scores on other major standardized medical examinations and competency-based ratings, could be used to predict performance on the American Board of Internal Medicine (ABIM) Rheumatology Certification Examination. Between 2008 and 2012, 629 second-year fellows took the ACR ITE. Bivariate correlation analyses of assessment scores and multiple linear regression analyses were used to determine whether ABIM Rheumatology Certification Examination scores could be predicted on the basis of ACR ITE scores, United States Medical Licensing Examination scores, ABIM Internal Medicine Certification Examination scores, fellowship directors' ratings of overall clinical competency, and demographic variables. Logistic regression was used to evaluate whether these assessments were predictive of a passing outcome on the Rheumatology Certification Examination. In the initial linear model, the strongest predictors of the Rheumatology Certification Examination score were the second-year fellows' ACR ITE scores (β = 0.438) and ABIM Internal Medicine Certification Examination scores (β = 0.273). Using a stepwise model, the strongest predictors of higher scores on the Rheumatology Certification Examination were second-year fellows' ACR ITE scores (β = 0.449) and ABIM Internal Medicine Certification Examination scores (β = 0.276). Based on the findings of logistic regression analysis, ACR ITE performance was predictive of a pass/fail outcome on the Rheumatology Certification Examination (odds ratio 1.016 [95% confidence interval 1.011-1.021]). The predictive value of the ACR ITE score with regard to predicting performance on the Rheumatology Certification Examination supports use of the Adult Rheumatology ITE as a valid feedback tool during fellowship training.

Impact of inpatient caseload, emergency department duties, and online learning resource on General Medicine In-Training Examination scores in Japan.

BackgroundBoth clinical workload and access to learning resource are important components of educational environment and may have effects on clinical knowledge of residents.MethodsWe conducted a survey with a clinical knowledge evaluation involving postgraduate year (PGY)-1 and -2 resident physicians at teaching hospitals offering 2-year postgraduate training programs required for residents in Japan, using the General Medicine In-Training Examination (GM-ITE). An individual-level analysis was conducted to examine the impact of the number of assigned patients and emergency department (ED) duty on the residents’ GM-ITE scores by fitting a multivariable generalized estimating equations. In hospital-level analysis, we evaluated the relationship between for the number of UpToDate reviews for each hospital and for the hospitals’ mean GM-ITE score.ResultsA total of 431 PGY-1 and 618 PGY-2 residents participated. Residents with four or five times per month of the ED duties exhibited the highest mean scores compared to those with greater or fewer ED duties. Those with largest number of inpatients in charge exhibited the highest mean scores compared to the residents with fewer inpatients in charge. Hospitals with the greater UpToDate topic viewing showed significantly greater mean score.ConclusionAppropriate ED workload and inpatient caseload, as well as use of evidence-based electronic resources, were associated with greater clinical knowledge of residents.

The Effect of a Targeted Educational Activity on Resident In-Training Examination Scores

PURPOSE: To evaluate the effect of a resident study program involving the development of a repository based on program-specific high yield topics on the Council on Resident Education in Obstetrics and Gynecology (CREOG) In-Training Examination. BACKGROUND: A CREOG Test Item Summary Booklet may help improve CREOG test scores. METHODS: In 2012, obstetrics and gynecology residents at the University of Hawaii participated in a study program involving directed research based on selected high yield questions from the CREOG Test Item Summary Booklet. Questions were considered high yield if more than 75% of the program answered them incorrectly. Topics were assigned to each resident by the program director, researched and presented at monthly teaching sessions. They were then made assessable in a wiki website. The intervention was discontinued in 2013 due to lack of resident interest. Paired Student t tests and one-way analysis of variance were used to compare the differences in mean level-specific standardized scores over that 3-year period. Only scores of residents present for the entire 3-year study period were included. RESULTS: Eleven residents were present during the study period and sat for all three CREOG examinations. During the year of the study program, mean level-specific CREOG in-training examination standardized scores increased significantly compared with the prior year (194.7 compared with 208.2, P<.05). Scores decreased when the program was discontinued (201.6, P<.05). DISCUSSION: These findings demonstrate that the CREOG Test Item Summary Booklet can be used to effectively direct educational efforts resulting in improvements in CREOG examination performance.

Council on Resident Education in Obstetrics and Gynecology In-Training Examination Results

BACKGROUND: The Council on Resident Education in Obstetrics and Gynecology (CREOG) In-Training Examination (ITE) was designed to serve as a tool to provide feedback on resident progress along with training program strengths and weaknesses. OBJECTIVE: The objective of this study was to identify predictive factors for success on American Board of Obstetrics and Gynecology (ABOG) Written Examination using CREOG ITE scores. METHODS: CREOG ITE and ABOG written examination results collected from 2002 to 2012, inclusive, from Reading Hospital and Greenville Health System residents who took the CREOG ITE exam for 4 years. Data were analyzed using logistic regression to identify key factors in predicting ABOG written examination outcomes. Categorical data were analyzed using χ2. RESULTS: Seventy-one residents were included for analysis, of whom 68.9% were female. Gender differences existed. For women, a low score starting in postgraduate year (PGY)-1 was correlated with ABOG failure (P<.001), but it was not associated in men (P=NS) until PGY-3. Results showed that while each year's score contributed to the chance of pass–fail, the contribution increased each year. For every point increase in year 4 examination score, chances of passing increased by 21.5% (P=.001). Calculations using all 4 years resulted in an area-under-the-curve (AUC) of 0.920, significantly better (P<.001) than the null hypothesis of 50%. CONCLUSION: Results indicate that the score of each subsequent PGY is more important than the previous PGY's score. CREOG ITE scores could be a useful tool for identifying residents requiring structured interventions.

Reading Habits of General Surgery Residents and Association With American Board of Surgery In-Training Examination Performance.

Few large-scale studies have quantified and characterized the study habits of surgery residents. However, studies have shown an association between American Board of Surgery In-Training Examination (ABSITE) scores and subsequent success on the American Board of Surgery Qualifying and Certifying examinations. To identify the quantity of studying, the approach taken when studying, the role that ABSITE preparation plays in resident reading, and factors associated with ABSITE performance. An anonymous 39-item questionnaire including demographic information, past performance on standardized examinations, reading habits, and study sources during the time leading up to the 2014 ABSITE and opinions pertaining to the importance of the ABSITE was administered August 1, 2014, to August 25, 2014, to 371 surgery residents in 15 residency programs nationwide. Scores from the 2014 ABSITE. A total of 273 residents (73.6%) responded to the survey. Seven respondents did not provide their January 2014 ABSITE score, leaving 266 for statistical analysis. Most respondents were male (162 of 266 [60.9%]), with a mean (SD) age of 29.8 (2.6) years. The median number of minutes spent studying per month was 240 (interquartile range, 120-600 minutes) for patient care or clinical duties and 120 for the ABSITE (interquartile range, 30-360 minutes). One hundred sixty-four of 266 respondents (61.7%) reported reading consistently throughout the year for patient care or clinical duties. With respect to ABSITE preparation, 72 of 266 residents (27.1%) reported reading consistently throughout the year, while 247 of 266 residents (92.9%) reported preparing between 1 and 8 weeks prior to the examination. Univariate analysis (with results reported as effect on median ABSITE percentile scores [95% CIs]) identified the following factors as positively correlated with ABSITE scores: prior United States Medical Licensing Examination (USMLE) 1 and 2 scores (per 1-point increase: USMLE 1, 0.1 [0.02-0.14], P = .03; USMLE 2, 0.3 [0.19-0.44], P < .001), prior Medical College Admission Test (MCAT) scores (per 1-point increase, 1.2 [1.3-2.0]; P = .002), high opinion of ABSITE significance (P < .001), surgical textbook use (11 [6-16]; P = .02), daily studying (13 [4-23]; P = .02), and high satisfaction with study materials (P < .001). On multivariable analysis, USMLE 2 score (per 1-point increase, 0.4 [0.2-0.6]; P < .001), MCAT score (0.6 [0.2-1.0]; P = .003), opinion of ABSITE significance (9.2 [6.9-11.6]; P < .001), and having an equal focus on patient care and ABSITE preparation during study (6.1 [0.6-11.5]; P = .03) were identified as positive predictors of ABSITE performance. Most residents reported reading consistently for patient care throughout the year. Daily studying and textbook use were associated with higher ABSITE scores on univariate analysis. Scores on the USMLE 2 and MCAT, as well as resident attitude regarding the importance of the ABSITE results, were independent predictors of ABSITE performance.

Improving American Board of Surgery In-Training Examination Scores: Motivation Matters.

Improving American Board of Surgery In-Training Examination Scores: Motivation Matters.

A Structured Educational Curriculum Including Online Training Positively Impacts American Board of Surgery In-Training Examination Scores

To assess the effect of a structured postgraduate year 1 educational curriculum, including online surgical training, on American Board of Surgery In-Training Examination (ABSITE) scores. This was a retrospective cohort study. The study was performed in an academic surgical residency program in a tertiary care hospital, Cleveland Clinic Foundation, Cleveland, Ohio. The participants were 140 surgical postgraduate year 1 residents from 2000 to 2009. Interns from 2000 to 2004 were grouped together and completed a self-directed learning curriculum. Interns from 2005 to 2009 participated in a structured educational curriculum that included lectures and the use of an online program. Lectures were based on the American College of Surgeons curriculum. The online program consisted of 8 to 12 hours of assigned tutorials and quizzes that corresponded to the lectures and 3 multiple-choice (MC) examinations. Use of a structured educational curriculum led to improved ABSITE scores (66 ± 9%) compared with that of those who had no curriculum (55 ± 10%, p < 0.001). Several variables positively correlated with the ABSITE score: United States Medical Licensing Examination step 1 score (p < 0.001), monthly quiz scores (p = 0.003), average MC examination scores (p = 0.005), lecture attendance (p = 0.02), and time spent online (p = 0.04). Multivariable analysis demonstrated that the step 1 United States Medical Licensing Examination score, time spent online, and MC examination score are predictive of total the ABSITE score. When ABSITE subscores (basic science and clinical science) were compared, the online curriculum had a greater effect on basic science subscores, whereas lectures had a greater effect on clinical science subscores. Providing surgery residents a structured curriculum with lectures and an online component positively impacts ABSITE scores.

Mobile computing as a tool in orthopaedic surgery residency training

Background: As mobile computing technology continues to improve and grow in popularity, such devices are becoming more prevalent in modern medical practice and education. The purpose of this study was to quantify resident perceptions, usage patterns, and the utility of an iPad/mobile tablet computer in orthopaedic surgery residency training. Methods: A total of nine residents were included in the pilot study. A mixed methods approach (i.e. quantitative and qualitative methods) was utilized, given the relatively small sample size. Each resident participated in a baseline survey and post survey for the study period of 1 academic year. The surveys attempted to quantify the usage patterns in three main areas: (1) resident education, (2) clinical practice, and (3) patient education. In addition, a qualitative component (i.e. focus group) was conducted to develop themes related to usage patterns. To determine quantitative effect, resident percentile score changes on in-training examination were compared before and after implementation of the pilot study with paired t-testing. A Student t-test was used to compare change in mean program percentile rank on in-training examination before and after implementation Results: An overall increase in usage of portable computing (the iPad) as a way for residents to study as well as perform administrative and clinical duties was observed. There was a trend towards increased overall study time and study time “on-the-go” as participants indicated increased study time in “hospital work areas.” There were also observed increases in professional activities such as referencing treatment options and ordering lab tests. Conversely, few participants reported using the iPad for accessing radiographic studies or demonstrating conditions to patients. The focus group discussion emphasized improved efficiency gained by having quick and easy access to reference and study material. However, the lack of informational technology support and infrastructure for integration with the electronic medical records (EMRs) and Picture Archiving and Communication System (PACS) hindered clinical and patient educational usage. Mean annual increase in in-training examination scores increased from 3.4 to 8.1 percentile points, similarly the mean annual program percentile rank based on scores increased from the 72nd to the 85th percentile. Conclusions: Mobile computers such as the iPad can be useful for residents in orthopaedic surgery by improving their access to educational resources and allowing more study time “on-the-go” which may lead to more time studying in general and improvement in test scores. Patient education and clinical usage is possible but information technology support is needed for these devices to be used to their fullest potential.

Faculty Prediction of In-Training Examination Scores of Emergency Medicine Residents: A Multicenter Study

BackgroundThe Emergency Medicine In-Training Examination (EMITE) is one of the few validated instruments for medical knowledge assessment of emergency medicine (EM) residents. The EMITE is administered only once annually, with results available just 2 months before the end of the academic year. An earlier predictor of EMITE scores would be helpful for educators to institute timely remediation plans. A previous single-site study found that only 69% of faculty predictions of EMITE scores were accurate. ObjectiveThe goal of this article was to measure the accuracy with which EM faculty at five residency programs could predict EMITE scores for resident physicians. MethodsWe asked EM faculty at five different residency programs to predict the 2014 EMITE scores for all their respective resident physicians. The primary outcome was prediction accuracy, defined as the proportion of predictions within 6% of the actual scores. The secondary outcome was prediction precision, defined as the mean deviation of predictions from the actual scores. We assessed faculty background variables for correlation with the two outcomes. ResultsOne hundred and eleven faculty participated in the study (response rate 68.9%). Mean prediction accuracy for all faculty was 60.0%. Mean prediction precision was 6.3%. Participants were slightly more accurate at predicting scores of noninterns compared to interns. No faculty background variable correlated with the primary or secondary outcomes. Eight participants predicted scores with high accuracy (>80%). ConclusionsIn this multicenter study, EM faculty possessed only moderate accuracy at predicting resident EMITE scores. A very small subset of faculty members is highly accurate.

Integration of Mobile Technology in Educational Materials Improves Participation: Creation of a Novel Smartphone Application for Resident Education

Traditional education consists of didactics and book learning. Recently, technology has been integrated into graduate medical education, primarily in the form of simulation. The primary aim of this study was to investigate if a novel smartphone application using technology to engage learners would improve participation in an educational activity when compared with a daily e-mail format and how this use translated to performance on standardized testing. The UF Surgery App (App), which is a smartphone application, was developed to deliver 2 questions from a general surgery educational database every weekday from October to February 2013. The App, developed for iOS, featured a notification alarm and a reminder icon to actively engage the learner. Learners who used the App responded to multiple-choice questions and were provided instantaneous feedback in the form of a correct answer with an explanation. The response rate and answers were collected prospectively and compared with the American Board of Surgery In-Training Examination score. University of Florida, College of Medicine, Gainesville, Florida, a university teaching hospital. A total of 46 general surgical residents were enrolled in a university training program. Participation was voluntary. Overall, 26 of 46 (57%) residents participated. Of the users, 70% answered more than 20% of the questions, while 46% responded to more than 70% of questions. The percentage of correct answers on the App was positively correlated with standardized score (p = 0.005), percentage correct (p = 0.02), and percentile (p = 0.034) on the ABSITE examination. Technology can be used to actively engage residents. Deployment of this novel App improved participation over a daily question-answer e-mail format, and answers correlated with standardized test performance. The effect of the App on overall education is unclear, and a multi-institutional study has been initiated.

The Effect of Surgical Resident Learning Style Preferences on American Board of Surgery In-Training Examination Scores

There is a growing body of literature that suggests that learners assimilate information differently, depending on their preferred learning style. The VARK model categorizes learners as visual (V), aural (A), read/write (R), kinesthetic (K), or multimodal (MM). We hypothesized that resident VARK learning style preferences and American Board of Surgery In-Training Examination (ABSITE) performance are associated. The Fleming VARK learning styles inventory was administered to all general surgery residents at a university hospital-based program each year to determine their preferred learning style. Resident scores from the 2012 and 2013 ABSITE were examined to identify any correlation with learning style preferences. Over a 2-year period, residents completed 53 VARK inventory assessments. Most (51%) had a multimodal preference. Dominant aural and read/write learners had the lowest and highest mean ABSITE scores, respectively (p = 0.03). Residents with dominant read/write learning preferences perform better on the ABSITE than their peers did, whereas residents with dominant aural learning preferences underperform on the ABSITE. This may reflect an inherent and inadvertent bias of the examination against residents who prefer to learn via aural modalities.

A time-efficient web-based teaching tool to improve medical knowledge and decrease ABIM failure rate in select residents

AimThe American Board of Internal Medicine (ABIM) exam's pass rate is considered a quality measure of a residency program, yet few interventions have shown benefit in reducing the failure rate. We developed a web-based Directed Reading (DR) program with an aim to increase medical knowledge and reduce ABIM exam failure rate.MethodsInternal medicine residents at our academic medical center with In-Training Examination (ITE) scores ≤35th percentile from 2007 to 2013 were enrolled in DR. The program matches residents to reading assignments based on their own ITE-failed educational objectives and provides direct electronic feedback from their teaching physicians. ABIM exam pass rates were analyzed across various groups between 2002 and 2013 to examine the effect of the DR program on residents with ITE scores ≤35 percentile pre- (2002–2006) and post-intervention (2007–2013). A time commitment survey was also given to physicians and DR residents at the end of the study.ResultsResidents who never scored ≤35 percentile on ITE were the most likely to pass the ABIM exam on first attempt regardless of time period. For those who ever scored ≤35 percentile on ITE, 91.9% of residents who participated in DR passed the ABIM exam on first attempt vs 85.2% of their counterparts pre-intervention (p<0.001). This showed an improvement in ABIM exam pass rate for this subset of residents after introduction of the DR program. The time survey showed that faculty used an average of 40±18 min per week to participate in DR and residents required an average of 25 min to search/read about the objective and 20 min to write a response.ConclusionsAlthough residents who ever scored ≤35 percentile on ITE were more likely to fail ABIM exam on first attempt, those who participated in the DR program were less likely to fail than the historical control counterparts. The web-based teaching method required little time commitment by faculty.