Content bias in the neoplastic-related items of the National Board of Medical Examiners Part II Examination.

Abstract

The Education Evaluation Committee of the American Association for Cancer Education (AACE) attempted to employ the neoplastic-related items from the National Board of Medical Examiners (NBME) Part II examination in a study correlating program characteristics at various medical schools with performance on the neoplastic-related items. All questions on the September, 1977 and April, 1978 NBME Part II examinations were reviewed by board certified medical oncologists who found an approximate 20% discordance between the AACE and NBME classifications. Content analysis of the neoplastic-related items disclosed a preponderance of questions related to gynecologic cancer with few questions concerning the major solid tumors: lung, breast and colorectal cancer. In addition, virtually no questions related to the psychologic impact of cancer or to the therapy of cancer were asked. We conclude that the use of item by item performance scores from the neoplastic-related items is inadequate for program analysis due to disagreements over what constitutes such an item and to content bias in the agreed upon items. We propose a coding system that may allow the results from current NBME examinations to be used in program evaluation.