The merits of representational pictures in educational assessment: Evidence for cognitive and motivational effects in a time-on-task analysis

Abstract

Adding representational pictures (RPs) to text-based items has been shown to improve students’ test performance. Focusing on potential explanations for this multimedia effect in testing, we propose two functions of RPs in testing, namely, (1) a cognitive facilitation function and (2) a motivational function. We found empirical support for both functions in this computer-based classroom experiment with N=410 fifth and sixth graders. All students answered 36 manipulated science items that either contained (text-picture) or did not contain (text-only) an RP that visualized the text information in the item stem. Each student worked on both item types, following a rotated within-subject design. We measured students’ (a) solution success, (b) time on task (TOT), and identified (c) rapid-guessing behavior (RGB). We used generalized and linear mixed-effects models to investigate RPs’ impact on these outcome parameters and considered students’ level of test engagement and item positions as covariates. The results indicate that (1) RPs improved all students’ performance across item positions in a comparable manner (multimedia effect in testing). (2) RPs have the potential to accelerate item processing (cognitive facilitation function). (3) The presence of RPs reduced students’ RGB rates to a meaningful extent (motivational function). Overall, our data indicate that RPs may promote more reliable test scores, supporting a more valid interpretation of students’ achievement levels.