Statistically guided grading judgements: contextualisation or contamination?

Abstract

ABSTRACT Different sources of assessment evidence are reviewed during International Baccalaureate (IB) grade awarding to convert marks into grades and ensure fair results for students. Qualitative and quantitative evidence are analysed to determine grade boundaries, with statistical evidence weighed against examiner judgement and teachers’ feedback on examinations. A trial was conducted to explore how examiners’ grading decisions were influenced by having access to statistical evidence. Grade awards were replicated in nine exams across five subjects, with examiners accessing all available evidence in one model, and only scripts and grade descriptors in the other. Preliminary findings suggest that both approaches lead to broadly comparable grading outcomes. Focus group feedback indicates that examiners consider judging the grade-worthiness of student work to be their primary role in grade award. Whilst they found item-level data helpful for prioritising questions for review, participants reported that access to evidence such as statistically recommended boundaries can cloud their judgement or encourage strategic grading. This study also raises further questions about the purposes and uses of different forms of statistical evidence, as well as how and when they should be integrated with qualitative evidence in grade awarding.