A survey and benchmarking study of multitreatment uplift modeling

Diego Olaya,Kristof Coussement,Wouter Verbeke

doi:10.1007/s10618-019-00670-y

Diego Olaya, Kristof Coussement + Show 1 more

Open Access

https://doi.org/10.1007/s10618-019-00670-y

Copy DOI

Abstract

Uplift modeling is an instrument used to estimate the change in outcome due to a treatment at the individual entity level. Uplift models assist decision-makers in optimally allocating scarce resources. This allows the selection of the subset of entities for which the effect of a treatment will be largest and, as such, the maximization of the overall returns. The literature on uplift modeling mostly focuses on queries concerning the effect of a single treatment and rarely considers situations where more than one treatment alternative is utilized. This article surveys the current literature on multitreatment uplift modeling and proposes two novel techniques: the naive uplift approach and the multitreatment modified outcome approach. Moreover, a benchmarking experiment is performed to contrast the performances of different multitreatment uplift modeling techniques across eight data sets from various domains. We verify and, if needed, correct the imbalance among the pretreatment characteristics of the treatment groups by means of optimal propensity score matching, which ensures a correct interpretation of the estimated uplift. Conventional and recently proposed evaluation metrics are adapted to the multitreatment scenario to assess performance. None of the evaluated techniques consistently outperforms other techniques. Hence, it is concluded that performance largely depends on the context and problem characteristics. The newly proposed techniques are found to offer similar performances compared to state-of-the-art approaches.

Highlights

Predictive analytics supports decision-making by exploiting the patterns present in historical data to obtain insights about future states
We contribute to the state-of-the-art in the field of uplift modeling by: (1) providing an exhaustive survey of the literature on Multitreatment uplift modeling (MTUM) and applying a framework to classify these methods; (2) proposing two new MTUM techniques; and (3) presenting the results of an extensive benchmarking study, and providing ample empirical evidence with respect to the performances of 13 MTUM methods for eight multitreatment uplift data sets
The performances of the models are evaluated by means of the Qini metric and the expected responses in order to facilitate their comparison

Summary

Introduction

Predictive analytics supports decision-making by exploiting the patterns present in historical data to obtain insights about future states. A key concern in real-world applications lies in foreseeing the effects of different actions on an outcome variable. This task is performed by uplift modeling techniques and allows decision-makers to prescribe the course of action that maximizes a given objective at the individual level. The identification of the most favorable action (hereafter referred to as treatment) for an individual corresponds to estimating the effect that a decision variable (e.g., treatment) has on an outcome variable (e.g., response) This association is known in the causal literature as the individual treatment effect (ITE) and frames uplift modeling as a causal inference task. From a machine learning perspective, this consists of contrasting the predicted values of the outcome variable for each of the treatments at the individual level

Methods

Results

Conclusion