Reviewing the research on instructional development programs for academics. Trying to tell a different story: A meta-analysis

Abstract

This paper presents a meta-analysis that investigates the effectiveness of instructional development programs (IDPs) dedicated to academics. IDPs are instructional activities specifically planned to improve quality of teaching in higher education by enhancing academics' instructional approach to support student learning. We analyzed 1555 unique results from online searches and from the references lists of the previous nine reviews on this topic. The final sample contains 20 controlled studies, carried out between 1973 and 2019, in which 23 independent effect sizes were reported. The results indicated that IDPs have a small effect size (d = 0.385, SE = 0.055, Z = 6.985, p < .001, k = 23). We did not find evidence of publication bias. Additionally, we analyzed the influences of different trainee's characteristics, features and outcome of IDPs on the final effectiveness of such programs. Trainees' mandatory enrollment on IDPs reported stronger effect (d = 0.515), while IDPs that lasted less than 15 h reported the highest effect size (d = 0.571). The effect size of IDPs reported on students' outcomes is small (d = 0.396), but that on teachers' outcomes is even smaller (d = 0.315). For the first time, our findings summarize quantitative evidence regarding the overall IDPs effectiveness.