A question of quality: Conceptualisations of quality in the context of educational test questions

Abstract

In educational contexts, questioning performs a number of functions. These include facilitating learning in the classroom and the recognition of achievement through examinations and other assessments. Good quality questions are important to ensuring that these functions are achieved. This research focused on educational exams and used views from question writers to explore conceptualisations of question quality and features thought to affect question quality. Seven examination question writers from four subjects were shown some example exam questions. Participants were asked to comment on the quality of the question, reflect on performance data, rate question quality and comment more generally on how they define quality in question writing. Three conceptualisations of question quality emerged, two relating to aspects of validity and one relating to pedagogical concerns. Participants varied in which definition dominated their views. Discussions also identified question features thought to affect quality. These are similar to features previously identified as affecting difficulty and fairness by studies analysing student performance.