A longitudinal quasi-experiment of leaderboard effectiveness on learner behaviors and course performance

Abstract

Leaderboards are increasingly viewed as a useful gamification technique to improve learning, partly due to their ease of implementation into common learning management systems. However, past research has revealed mixed results in terms of their effectiveness. This inconsistency may be attributable to a lack of attention to research design, conflation of multiple design factors, or overreliance on correlational studies. In the present study, a longitudinal quasi-experimental design incorporating random assignment of groups to leaderboard conditions was used to examine the effect of leaderboards on optional practice behaviors and course performance in a college course. Although practice behaviors were linked to course performance, the leaderboard did not meaningfully affect those practice behaviors and also negatively affected course performance through other mediators. These results highlight the critical role of rigorous theory-based design of gamified systems and also demonstrate the need for rigorous research designs to understand actual impacts. Educational relevance and implications statementOur research explored the impact of using leaderboards, a common gamification strategy, in an educational setting to see if they would motivate students to engage in optional practice activities, with the aim of improving their course performance. Surprisingly, we found that leaderboards did not encourage additional practice and, unexpectedly, led to lower exam scores. This suggests that leaderboards, while intended to boost motivation and learning through competition, might have the opposite effect by potentially undermining students' motivation or altering their perception of the course's value. These findings underscore the complexity of integrating gamification into education and highlight the necessity for careful design and testing of such strategies to ensure they align with educational goals. Our study is crucial for educators and instructional designers as it provides evidence that not all gamification techniques work as intended and stresses the importance of understanding the underlying psychological impacts on students.