The Introduction of Single Best Answer Questions as a Test of Knowledge in the Final Examination for the Fellowship of the Royal College of Radiologists in Clinical Oncology

Abstract

AimsThe single best answer (SBA) format of multiple choice questions (MCQ) is recognised to be better suited to the assessment of the higher levels of knowledge essential for clinical practice, such as data interpretation, problem solving and decision making, than traditional true/false MCQ. In autumn 2006, the Royal College of Radiologists (RCR) introduced SBA questions as its sole written test of knowledge for the Final Fellowship Examination in Clinical Oncology (Final FRCR Examination). This article reviews the application of SBA questions to clinical oncology and analyses the first year's experience of the new examination format. MethodsThe results of the last two true/false MCQ examinations (autumn 2005 and spring 2006) and the first two SBA examinations (autumn 2006 and spring 2007) were analysed. The predictive values of the different components of the Final FRCR Examination (SBA, true/false MCQ, case orientated questions COQ, clinical and oral examinations) were compared. ResultsIn autumn 2005, 86% of candidates passed the true/false MCQ paper but only 48% passed the examination overall. In spring 2006, 91% of candidates passed the true/false MCQ paper but the overall pass rate was only 36%. In contrast, the pass rate for the SBA papers was 66% for both autumn 2006 and spring 2007, which was comparable to the overall pass rate of 53% and 52% respectively. All the components of the examination (SBA, true/false MCQ, COQ, clinical and oral examinations) had similar negative predictive values of between 80% and 90% (p=0.3, chi-square test). However, the positive predictive value of true/false MCQ was inferior to the other sections of the examination (46% compared to 74%, 80%, 74% and 72% for SBA, COQ, clinicals and orals respectively, p<0.001, chi-square test). ConclusionThe new format SBA questions are more reliable than the previous true/false MCQ in discriminating between knowledgeable and unknowledgeable candidates in the Final FRCR Examination in Clinical Oncology.