The relationship of national inservice examination scores, emergency medicine faculty evaluations, and level of training of emergency medicine residents

Abstract

Study objectives: We determine the relationship between national in-service examination scores, level of training, and faculty assessment of emergency medicine resident clinical performance. Methods: This study was conducted at an academic emergency department that has a postgraduate year (PGY) 1 to 3 emergency medicine residency with 24 emergency medicine residents and 25 board-certified or board-prepared emergency medicine faculty who rate each of the residents quarterly using a standard form. The form has 13 characteristics and a composite score. Each resident is rated on a 1-to-5 fixed-reference scale that rates the resident relative to the performance of an attending physician. Each resident's raw score from the national in-service examination was used as the measure of medical knowledge. The composite score for each resident was the variable used to measure resident clinical performance. We performed a regression analysis to determine the relationship between the in-service examination score, the PGY level, and the clinical evaluation by the faculty. Results: During the 5 years of this study, 57 residents were included. These residents had 2,095 total faculty evaluations, yielding 119 composite faculty evaluations. Two residents missed the in-service examination, yielding 117 in-service examination scores. The regression analysis found a strong overall relationship between the faculty composite evaluation score and both the in-service examination score and PGY variables (R=.86, P<.0001). The in-service examination score and the PGY level were found to independently correlate with the faculty evaluations. The PGY level was found to correlate more highly with the faculty composite score (R=.73, P<.0001) than the in-service examination score correlated with the faculty composite score(R=.57, P<.0001). Conclusion: Emergency medicine faculty assessment of resident clinical performance correlates with medical resident national in-service examination scores and PGY level independently. The PGY level of training has a much higher degree of correlation with clinical performance than the in-service examination score. Study objectives: We determine the relationship between national in-service examination scores, level of training, and faculty assessment of emergency medicine resident clinical performance. Methods: This study was conducted at an academic emergency department that has a postgraduate year (PGY) 1 to 3 emergency medicine residency with 24 emergency medicine residents and 25 board-certified or board-prepared emergency medicine faculty who rate each of the residents quarterly using a standard form. The form has 13 characteristics and a composite score. Each resident is rated on a 1-to-5 fixed-reference scale that rates the resident relative to the performance of an attending physician. Each resident's raw score from the national in-service examination was used as the measure of medical knowledge. The composite score for each resident was the variable used to measure resident clinical performance. We performed a regression analysis to determine the relationship between the in-service examination score, the PGY level, and the clinical evaluation by the faculty. Results: During the 5 years of this study, 57 residents were included. These residents had 2,095 total faculty evaluations, yielding 119 composite faculty evaluations. Two residents missed the in-service examination, yielding 117 in-service examination scores. The regression analysis found a strong overall relationship between the faculty composite evaluation score and both the in-service examination score and PGY variables (R=.86, P<.0001). The in-service examination score and the PGY level were found to independently correlate with the faculty evaluations. The PGY level was found to correlate more highly with the faculty composite score (R=.73, P<.0001) than the in-service examination score correlated with the faculty composite score(R=.57, P<.0001). Conclusion: Emergency medicine faculty assessment of resident clinical performance correlates with medical resident national in-service examination scores and PGY level independently. The PGY level of training has a much higher degree of correlation with clinical performance than the in-service examination score.