Low-stakes performance testing in Germany by the VERA assessment: analysis of the mode effects between computer-based testing and paper-pencil testing

Inga Wagner,Philipp Loesche,Steven Bißantz

doi:10.1007/s10212-021-00532-6

Inga Wagner, Philipp Loesche + Show 1 more

Open Access

https://doi.org/10.1007/s10212-021-00532-6

Copy DOI

Abstract

The German school system employs centrally organized performance assessments (some of which are called “VERA”) as a way of promoting lesson development. In recent years, several German federal states introduced a computer-based performance testing system which will replace the paper-pencil testing system in the future. Scores from computer-based testing are required to be equivalent to paper-pencil testing scores so that the new testing medium does not lead to disadvantages for students. Therefore, the current study aimed at investigating the size of the mode effect and the moderating impact of students’ gender, academic achievement and mainly spoken language in everyday life. In addition, the variance of the mode effect across tasks was investigated. The study was conducted in four German federal states in 2019 using a field experimental design. The test scores of 5140 eighth-graders from 165 schools in the subject German were analysed. The results of multi-level modelling revealed that students’ test scores in the computerized version of the VERA test were significantly lower than in the paper-pencil version. Students with a lower academic achievement were more disadvantaged by the VERA computerized test. The results were inconsistent regarding the interactions between testing mode and students’ gender and mainly spoken language in everyday life. The variance of the mode effect across tasks was high. Research into different subjects and in other federal states and countries under different testing conditions might yield further evidence about the generalizability of these results.

Highlights

Performance tests were introduced in Germany about 15 years ago as part of an external evaluation system conducted in schools
The results of the current study revealed that student performance in the German VERA tests in the year 2019 was significantly lower in the computer-based test than in the paper-pencil test in the subject German
This result applied to the testing domains orthography and reading and for both test booklets, which were completed by different student samples with varying levels of academic achievement

Summary

Introduction

Performance tests were introduced in Germany about 15 years ago as part of an external evaluation system conducted in schools. These tests are called “VERA” in several German federal states. By reflecting on the VERA test results, teachers are supposed to obtain a better insight into their classes’ deficits. Based on this knowledge they can adapt their teaching, e.g. by repeating content and tasks that were poorly solved by their students in the VERA tests. In contrary to the United States, where poor results in performance tests can be sanctioned (e.g. Stecher, 2002), there are no severe consequences for teachers of low-performing classes in Germany

Objectives

Methods

Results

Conclusion