Heterogeneous Treatment Effects and Optimal Targeting Policy Evaluation

Abstract

We discuss how to construct optimal targeting policies and document the difference in profits from alternative targeting policies by using estimation approaches that are based on recent advances in causal inference and machine learning. We introduce an approach to evaluate the profit of any targeting policy using only one single randomized sample. This approach is qualitatively equivalent to conducting a field test, but reduces the cost of multiple field tests because all comparisons can be conducted in only one sample. The approach allows us to compare many alternative optimal targeting policies that are constructed based on different estimates of the conditional average treatment effect, i.e. the incremental effect of targeting. We draw a conceptual distinction between methods that predict the conditional average treatment effect indirectly via the conditional expectation function trained on the outcome level, and methods that directly predict the conditional average treatment effect. We propose a new direct estimation method, called treatment effect projection. The empirical application is to a catalog mailing with a high-dimensional set of customer features. We find that the optimal targeting policies based on the direct estimation methods typically outperform the indirect estimation methods, both in the validation sets from the same population from which the training set is drawn and in the data obtained one year after the training set was collected. In particular, the treatment effect projection performs similar to the recently introduced causal forest of Wager and Athey (2017). We also compare targeting policies based on conditional average treatment effects with a sophisticated application of the traditional CRM approach that is based on a prediction of the outcome level. Even though based on a conceptually incorrect metric — outcome levels — the sophisticated application of the traditional CRM approach often yields larger profits than the targeting policies based on the indirect estimation methods.