Empirical Evaluation of Attribution Models

Abstract

Advertisers are demanding more accurate estimates of the impact of advertisements. The gold standard for evaluating the advertising effectiveness is to run randomized experience. However, due to costs and other issues, few advertisers use randomized experiments. Instead, advertiser use attribution models to estimate advertising effectiveness. These models make a number implicit and explicit assumptions. In this work, we use a natural randomized field experiment to empirically validate the attribution models. Unlike previous work that focuses on model prediction accuracy, we also compute the model's average treatment effect on the treated ATET. We compare the computed ATET to observed ATET. We use the ATET prediction accuracy to select the model specification. Our results indicate that simple regression models can adequately estimate the treatment effect on the treated. Furthermore, we explored a number of functional forms for modeling the impact of impressions. While results suggest that both a linear and logarithmic models produce best results, we favor the logarithmic model because its concave and captures the diminishing marginal return.