When precedence sets a bad example for reform: conceptions and reliability of a questionable high stakes assessment practice in Norwegian universities

Abstract

The validity of the Norwegian university grading standard has been called into serious question. The implicit standards used for assessing exams and the reliability of that understanding among examiners and psychology students were investigated in three studies. Studies 1 and 2 investigated the implicit standards that examiners used when assessing exams and the implicit standards that students presume examiners use in that process. Study 3 investigated how reliable expert examiners’ assessments actually are. The validity of the grading standards is challenged by findings that indicate that examiners and students shared similar but not identical conceptions of them, and tests of grader reliability revealed significant variance at critical thresholds. The findings are discussed in terms of their relevance to educational reforms currently going on in Norwegian higher education. The human cost of not completely reforming past practices is highlighted, and six concrete improvements are suggested.