Using Propensity Scores for Estimating Causal Effects: A Study in the Development of Moral Reasoning

Abstract

The purpose of this study was to illustrate the use of propensity scores for creating comparison groups, partially controlling for pretreatment course selection bias, and estimating the treatment effects of selected courses on the development of moral reasoning in undergraduate students. Specifically, we used a sample of convenience for comparing differences in moral reasoning development scores among students enrolled in intergroup dialogue, service learning, psychology and philosophy courses with those of an introductory sociology course. Adopting a propensity score approach included reviewing the empirical literature for its guidance in substantiating the reasons for including pretreatment variables (i.e., pretreatment course-taking behaviors, race, sex, political identification, need for cognition, major, age, pretreatment moral reasoning scores) in our analysis, measuring these variables, and reducing them into a single composite propensity score for each student in our analytic sample. This score then served as the basis for creating a new comparison group and for allowing us to estimate unbiased (or less biased) course-related treatment effects on moral reasoning development. Implications for higher education researchers are discussed.