Testing the test: an analysis of the MRCGP Applied Knowledge Test as an assessment tool

Abstract

The new membership examination of the Royal College of General Practitioners (MRCGP) has been in existence since autumn 2007. The MRCGP assessments aim to assure the RCGP of the competence of a general practitioner specialty trainee (GPST). Once the MRCGP is obtained, the GPST is then able to apply for a certificate of completion of training (CCT) from the General Medical Council (GMC). The MRCGP demonstrates the commonly described educational theory model of Miller's pyramid (see Figure 1). The Applied Knowledge Test (AKT), as the name suggests, aims for more than mere recall and arguably is at the 'knows how' level. Current literature suggests that the 'shows how' level constitutes competency-based assessment rather than performance. In the MRCGP, this takes the form of Clinical Skills Assessment (CSA). The level of 'does' is assessed via workplace-based assessments. These are performed throughout the three years of GPST training. They also take into account that the skill of performance incorporates not only competence, but also a combination of other influences mentioned in the Cambridge model for delineating performance and competence. This article will focus on the AKT. It is a three-hour 200-item multi-choice question (MCQ) examination that covers the three broad domains described in Box 1. The questions are in one of three formats: single best answer, extended matching questions and completion of algorithms. Literature exists that looks at the characteristics of a good assessment system. However, it is the work of van der Vleuten that is widely cited. He suggested that validity, reliability, educational impact, cost-effectiveness and acceptability must all be considered when constructing an assessment. I will now focus on the AKT in relation to each of the factors.