Quality standards for new modes modes of assessment. An exploratory study of the consequential validity of the OverAll Test

Abstract

During the past decade, due to societal developments, methods of instruction as well as the assessment of students’ performances have changed to an important considerable extent. Two of the elements of this change are the accents on cognitive competencies such as problem solving and on learning in an authentic context. In conjunction with the development of such learning methods, new modes of assessment were implemented. It was expected that this change would have positive feedback effects on learning and teaching. These feedback effects are the central issue of this article. They are discussed in terms of the experiences of the Maastricht School of Economics and Business Administration. This school places the analysis of authentic problems at the core of the curriculum, including the learning process as well as the assessment system. The OverAll Test, a case-based assessment instrument aiming to assess problem solving skills, was implemented as part of this. Different quality issues related to the OverAll Test have been evaluated. This article presents the results of one of the four validity studies conducted; an exploratory study of the consequential validity of the OverAll Test. It starts with the an outline of the main features of the new modes of assessment and the OverAll test as an example. There is then a discussed discussion of effectively the OverAll test fits these features as well as the goals and characteristics of problem-based learning. The study of the consequential validity of the OverAll Test is then described in depth. The results of the survey, as well as the results of the semi-structured interviews with staff and students, indicate a friction between the intended characteristics of the learning and assessment environment and the practice of instruction and assessment.