Comparing the effectiveness of two remedial mathematics courses using modern regression discontinuity techniques

Abstract

ABSTRACT Studying the effectiveness of remedial courses in higher education has attracted a lot of interest from educational researchers and practitioners. Remediation is associated with significant economic and social costs while the results are usually dubious. In this paper, we apply a widely used method called Regression Discontinuity Design (RDD) to measure the effectiveness of two differently designed remedial mathematics courses at the Budapest University of Technology and Economics. Our large-scale study is based on data of almost 20,000 undergraduate students enrolled between 2010 and 2018. Using modern RDD tools in various settings, we study both the direct and longer-term effects of remediation; and find that the design of the remedial course matters a lot. We measured a statistically significant positive effect on subsequent academic achievement for both course designs; however, the magnitude of the effect differs substantially. We measured a higher effect for the remedial course that serves as an extra practice class for the university level calculus course than for traditional remediation. As a methodological novelty, we propose a novel alternative method to handle discrete running variable in the RDD setting. We also provide some suggestions on how to improve mathematical remediation using personalized e-learning systems. Abbreviations: BME: Budapest University of Technology and Economics; STEM: science; technology; engineering; and mathematics; IntroMathHS: the general introductory mathematics course; IntroMathCalc: the introductory mathematics course at the VBK; Calc1: the first-year calculus course; AdvCalc: the advanced calculus course at the VBK; ACWA-C: adjusted credit weighted average without the result of the Calc1 course; RDD: regression discontinuity design; IK: Imbens–Kalyanaraman; VBK: Faculty of Chemical Technology and Biotechnology; ÉMK: Faculty of Civil Engineering; GPK: Faculty of Mechanical Engineering; GTK: Faculty of Economic and Social Sciences; KJK: Faculty of Transportation Engineering and Vehicle Engineering; VIK: Faculty of Electricalengineering and Informatics