Alternatives to traditional outcome modelling approaches in applied linguistics: A primer on propensity score matching

Abstract

Observational studies and quasi-experiments are commonplace in applied linguistics. Causal inference based on such a design hinges on different assumptions, one of which is commonly referred to as the assumption of no unmeasured confounding. To address this assumption, researchers typically consider different confounders using regression adjustment methods (i.e., entering the confounders into the model as covariates). Although these methods are theoretically sound procedures, they are not without criticism, for instance, regarding potentially extrapolated effect estimates or implausible linear functional form assumptions. In this article, we introduce an alternative statistical method, known as propensity score matching (PSM), which can help address the limitations of traditional regression adjustment methods. We use an example case study, where we first discuss the challenges applied linguists face when they intend to investigate causal effects. We illustrate how PSM can be used with applied linguistics data and examine theoretical and practical issues around each analytical step, including electing relevant covariates, estimating the propensity score, performing matching, assessing the matches, and estimating the treatment effects. Our overall goal is to provide applied linguists with an alternative method (PSM), which might be beneficial and reasonably applied in several practical situations.