A fundamental measure of treatment effect heterogeneity

Jonathan Levy,Mark Van Der Laan,Romain Pirracchio,Alan Hubbard

doi:10.1515/jci-2019-0003

Abstract

Abstract The stratum-specific treatment effect function is a random variable giving the average treatment effect (ATE) for a randomly drawn stratum of potential confounders a clinician may use to assign treatment. In addition to the ATE, the variance of the stratum-specific treatment effect function is fundamental in determining the heterogeneity of treatment effect values. We offer a non-parametric plug-in estimator, the targeted maximum likelihood estimator (TMLE) and the cross-validated TMLE (CV-TMLE), to simultaneously estimate both the average and variance of the stratum-specific treatment effect function. The CV-TMLE is preferable because it guarantees asymptotic efficiency under two conditions without needing entropy conditions on the initial fits of the outcome model and treatment mechanism, as required by TMLE. Particularly, in circumstances where data adaptive fitting methods are very important to eliminate bias but hold no guarantee of satisfying the entropy condition, we show that the CV-TMLE sampling distributions maintain normality with a lower mean squared error than TMLE. In addition to verifying the theoretical properties of TMLE and CV-TMLE through simulations, we highlight some of the challenges in estimating the variance of the treatment effect, which lack double robustness and might be biased if the true variance is small and sample size insufficient.

Highlights

Introduction and backgroundThe stratum-specific treatment effect function is a random variable defined as the average treatment effect (ATE) for a randomly drawn stratum of patient characteristics, which are confounders in that they possibly affect the outcome and occur before treatment assignment
Some degree of overfitting introduced by random forest about 20% of the time out of the library of 18 learners generated outliers for targeted maximum likelihood estimator (TMLE), resulting in lack of normality of the sampling distribution and causing higher bias and variance, where as cross-validated TMLE (CV-TMLE) appeared unaffected by the overfitting
The results of the simulation and the real case study highlight the existence of two great challenges in estimating variance of treatment effects (VTEs), one being the fact that the parameter is bounded below at 0, skewing and biasing the estimates when the true variance is too small for the sample size

Summary

Introduction

The stratum-specific treatment effect function ( referred to as the CATE function for conditional average treatment effect [ATE]) is a random variable defined as the ATE for a randomly drawn stratum of patient characteristics, which are confounders (potential causal parents of treatment) in that they possibly affect the outcome and occur before treatment assignment. Beyond the mean effect, of interest is the variation in this effect across the strata of potential confounders. One measure of this variation is the variance of the CATE function (as a random function of confounders), or variance of treatment effects (VTEs). Though the ATE might indicate that a treatment is beneficial on average, a comparatively large VTE would indicate there might be a significant portion of the population that has deleterious effects from treatment.

Objectives

Methods

Findings

Conclusion