Measuring the Impact of an Individual Course on Students' Success

Abstract

BackgroundAt the University of Michigan, qualified first‐year students who place out of the first‐semester calculus course may enroll in either the regular second‐semester calculus course or Applied Honors Calculus II. Students who enroll in Applied Honors Calculus II show higher academic performance than students enrolling in the Regular Calculus II.Purpose (Hypothesis)The study addressed the question: does enrollment in Applied Honors Calculus II have a positive causal impact on subsequent academic performance for engineering students at the University of Michigan?Design/MethodWe acquired seven years of institutional data for engineering students who entered the University of Michigan from 1996 through 2003 and who qualified to enroll in Applied Honors Calculus II. Using regression analyses, we tested a causal model of impact of Applied Honors Calculus II on four measures of subsequent academic performance: grade in Physics II and average grade in all subsequent physics, mathematics, and engineering courses.ResultsAfter controlling for students' personal characteristics and prior academic achievement, the impact of Applied Honors Calculus II on students' academic performance was not statistically significant. In particular Advanced Placement scores accounted for the higher performance observed in Applied Honors Calculus II students.ConclusionsWe recommend including Advanced Placement scores in models that predict academic performance. Future research should also include measures of socioeconomic status (SES) and explore interactions between SES and academic background. Finally, in evaluations of specific curricula, the treatment effect—measured as treatment group mean minus control group mean, after controlling for covariates—is unlikely to be large if the control group receives high quality instruction.