Self-referenced vs. reward-based feedback messages in online courses with formative mastery assessments: A randomized controlled trial in secondary classrooms

Abstract

Mastery assessments are used in many digital learning environments to control learning progression and support low-achieving students relearning important content. But research on personalized feedback following a mastery assessment is scarce and yielded ambiguous effects. Therefore, this study seeks to contribute to this area of research by investigating the effect of two different feedback messages in short online courses for German and English language grammar and spelling in secondary classrooms (grades 6–10). The study compares a reward-based feedback message to a self-referenced feedback message. The reward-based message indicates if the learner reaches the mastery criterion and how many points the learning app awarded for completing the test. The self-referenced feedback message applies motivation theory to strengthen students' internal attribution of causes for the test performance. The web-app MasteryX (www.masteryx.de) randomly assigned students to either the reward-based or the self-referenced feedback message. The study sampled 620 students (309 female, 311 male) in 53 classrooms from 27 secondary schools. It analyzed the effect of the two types of feedback messages on the level of test-retest-sequences (n = 2450). Results indicate small though significant positive effects of the self-referenced feedback message on subsequent learning behavior (reading elaborated item feedback, training behavior). However, a multilevel regression model showed small to medium effects of the reward-based feedback message on the higher course levels' follow-up mastery assessment score. The findings emphasize the complexity of designing personalized feedback strategies in online learning environments with mastery assessments.