Do we always need a difference? Testing equivalence in a blended learning setting

Abstract

ABSTRACTEvidence-based research is becoming increasingly important in educational research. Calculation and test methods available in statistical software packages such as SPSS and STATA are widely used. To evaluate teaching innovations such as blended learning against classical classroom settings, for example, previous studies have mainly applied inference methods such as the t-test or variance analyses. The problem with these methods is that they test for the difference. A non-significant result does not automatically mean equivalence of the treatments examined, which is why we propose the use of equivalence testing. This paper introduces the equivalence test as complementary to the classical t-test and briefly discusses other approaches based on confidence intervals and Bayesian methods. As an example, the introduction of a blended learning format to a Bachelor's degree programme is used to demonstrate the procedure and discuss the results of conducting an equivalence test. By combining tests for difference and equivalence successfully, it was possible to arrive at more informative statistical statements: Whereas a t-test alone only produced results for three out of 22 courses, a t-test and an equivalence test in combination yielded statistically confirmed statements for 12 out of 22 courses.