Where national board examinations pass and fail in evaluating knowledge of surgical clerks

Abstract

The aim of this study is to test a multiple choice test generated by the computer (CGE) with the National Board Examination (NBE) used for evaluation of surgical clerks. The CGE was developed and tested over a 2-year period prior to the study. It resembles the NBE in length and format but is geared to content covered in the 12-week surgical rotation. The research question concerned whether the CGE could replace the NBE. One hundred seventy-two students were randomly assigned to four groups at the end of each quarter of the year. They were tested twice, 1 day apart as follows: 1. NBE-NBE; 2. CGE-CGE; 3. NBE-CGE; 4. CGE-NBE. Correlations between NBE and CGE were in the moderate range. Analysis of variance between scores on CGE and NBE showed differences that were almost of borderline significance ( P < 0.08). Correlation of CGE and NBE with faculty and peer ratings on the students' level of knowledge showed a higher degree of correlation for the CGE than the NBE. The decision was made to continue both examinations as a part of the evaluation system, since they seemed to measure somewhat different areas of cognitive ability. Furthermore, CGE provides immediate and specific feedback to students on their performance before they left the clerkship and it helps faculty to judge effectiveness in reaching Departmental learning objectives. The NBE, however, prepares students for subsequent testing on the same type of examination used for licensure and gives the faculty a basis for judging the curriculum against national norms.