Estimation Of Average Treatment Effects Research Articles

Partially Pooled Propensity Score Models for Average Treatment Effect Estimation with Multilevel Data

Abstract Causal inference analyses often use existing observational data, which in many cases has some clustering of individuals. In this paper, we discuss propensity score weighting methods in a multilevel setting where within clusters individuals share unmeasured confounders that are related to treatment assignment and the potential outcomes. We focus in particular on settings where models with fixed cluster effects are either not feasible or not useful due to the presence of a large number of small clusters. We found, both through numerical experiments and theoretical derivations, that a strategy of grouping clusters with similar treatment prevalence and estimating propensity scores within such cluster groups is effective in reducing bias from unmeasured cluster-level covariates under mild conditions on the outcome model. We apply our proposed method in evaluating the effectiveness of centre-based pre-school programme participation on children’s achievement at kindergarten, using the Early Childhood Longitudinal Study Kindergarten data.

Estimation of average treatment effect based on a semiparametric propensity score

This paper considers the estimation of average treatment effect using propensity score method. We propose to use a semiparametric single-index model to estimate the propensity score. This avoids the curse of dimensionality problem with the nonparametric method based propensity score estimator. We establish the asymptotic distribution of the average treatment effect estimator. Monte Carlo simulation results show that the proposed method works well in finite samples and outperforms the conventional nonparametric kernel approach. We apply the proposed method to an empirical data examining the efficacy of right heart catheterization on medical outcomes.

Adoption of rural bank credit programs among smallholder farmers in Ghana: an average treatment effect estimation of rates of exposure and adoption and their determinants

PurposeThe study aims to estimate the rates of exposure to, and adoption of, rural bank credit programs by smallholder farmers in rural Ghana and the factors responsible for those rates.Design/methodology/approachThe study used a random sample of 300 smallholder farmers in the Fanteakwa District of Ghana, obtained through the multistage sampling technique. The study also employed the average treatment effects approach to estimate the average treatment effect of farmers’ exposure to rural bank credit programs, on their adoption of such programs.FindingsThe actual adoption rate is approximately 41%, and the potential, conditional on the whole population being aware of rural bank credit programs, is approximately 61%. Accordingly, there is a gap of about 20% in the adoption of rural bank credit programs, and is due to the incomplete exposure of smallholder farmers to the rural bank credit programs. Age of the household head, access to extension services, membership in farmer-based organizations and active savings accounts with a rural bank are the major contributors to smallholder farmer exposure to and the adoption of rural bank credit programs.Originality/valueThe current study is the first of its kind to be conducted in Ghana on rural bank credit programs. It takes into account the extent to which smallholder farmers are exposed to such credit programs and how it influences their decisions to access or adopt.

Transporting experimental results with entropy balancing.

We show how entropy balancing can be used for transporting experimental treatment effects from a trial population onto a target population. This method is doubly robust in the sense that if either the outcome model or the probability of trial participation is correctly specified, then the estimate of the target population average treatment effect is consistent. Furthermore, we only require the sample moments of the effect modifiers drawn from the target population to consistently estimate the target population average treatment effect. We compared the finite-sample performance of entropy balancing with several alternative methods for transporting treatment effects between populations. Entropy balancing techniques are efficient and robust to violations of model misspecification. We also examine the results of our proposed method in an applied analysis of the Action to Control Cardiovascular Risk in Diabetes Blood Pressure trial transported to a sample of US adults with diabetes taken from the National Health and Nutrition Examination Survey cohort.

Design-Based Ratio Estimators and Central Limit Theorems for Clustered, Blocked RCTs

This article develops design-based ratio estimators for clustered, blocked randomized controlled trials (RCTs), with an application to a federally funded, school-based RCT testing the effects of behavioral health interventions. We consider finite population weighted least-square estimators for average treatment effects (ATEs), allowing for general weighting schemes and covariates. We consider models with block-by-treatment status interactions as well as restricted models with block indicators only. We prove new finite population central limit theorems for each block specification. We also discuss simple variance estimators that share features with commonly used cluster-robust standard error estimators. Simulations show that the design-based ATE estimator yields nominal rejection rates with standard errors near true ones, even with few clusters.

Treatment Effect Estimation with Disentangled Latent Factors

Much research has been devoted to the problem of estimating treatment effects from observational data; however, most methods assume that the observed variables only contain confounders, i.e., variables that affect both the treatment and the outcome. Unfortunately, this assumption is frequently violated in real-world applications, since some variables only affect the treatment but not the outcome, and vice versa. Moreover, in many cases only the proxy variables of the underlying confounding factors can be observed. In this work, we first show the importance of differentiating confounding factors from instrumental and risk factors for both average and conditional average treatment effect estimation, and then we propose a variational inference approach to simultaneously infer latent factors from the observed variables, disentangle the factors into three disjoint sets corresponding to the instrumental, confounding, and risk factors, and use the disentangled factors for treatment effect estimation. Experimental results demonstrate the effectiveness of the proposed method on a wide range of synthetic, benchmark, and real-world datasets.

Commentary: Using potential outcomes causal methods to assess whether reductions in PM2.5 result in decreased mortality

Causal inference regarding exposures to ambient fine particulate matter (PM2.5) and mortality estimated from observational studies is limited by confounding, among other factors. In light of a variety of causal inference frameworks and methods that have been developed over the past century to specifically quantify causal effects, three research teams were selected in 2016 to evaluate the causality of PM2.5-mortality association among Medicare beneficiaries, using their own selections of causal inference methods and study designs but the same data sources. With a particular focus on controlling for unmeasured confounding, two research teams adopted an instrumental variables approach under a quasi-experiment or natural experiment study design, whereas one team adopted a structural nested mean model under the traditional cohort study design. All three research teams reported results supporting an estimated counterfactual causal relationship between ambient PM2.5 and all-cause mortality, and their estimated causal relationships are largely of similar magnitudes to recent epidemiological studies based on regression analyses with omitted potential confounders. The causal methods used by all three research teams were built upon the potential outcomes framework. This framework has marked conceptual advantages over regression-based methods in addressing confounding and yielding unbiased estimates of average treatment effect in observational epidemiological studies. However, potential violations of the unverifiable assumptions underlying each causal method leave the results from all three studies subject to biases. We also note that the studies are not immune to some other common sources of bias, including exposure measurement errors, ecological study design, model uncertainty and specification errors, and irrelevant exposure windows, that can undermine the validity of causal inferences in observational studies. As a result, despite some apparent consistency of study results from the three research teams with the wider epidemiological literature on PM2.5-mortality statistical associations, caution seems warranted in drawing causal conclusions from the results. A possible way forward is to improve study design and reduce dependence of conclusions on untested assumptions by complementing potential outcomes methods with structural causal modeling and information-theoretic methods that emphasize empirically tested and validated relationships.

Time-uniform, nonparametric, nonasymptotic confidence sequences

A confidence sequence is a sequence of confidence intervals that is uniformly valid over an unbounded time horizon. Our work develops confidence sequences whose widths go to zero, with nonasymptotic coverage guarantees under nonparametric conditions. We draw connections between the Cram\'er-Chernoff method for exponential concentration, the law of the iterated logarithm (LIL), and the sequential probability ratio test -- our confidence sequences are time-uniform extensions of the first; provide tight, nonasymptotic characterizations of the second; and generalize the third to nonparametric settings, including sub-Gaussian and Bernstein conditions, self-normalized processes, and matrix martingales. We illustrate the generality of our proof techniques by deriving an empirical-Bernstein bound growing at a LIL rate, as well as a novel upper LIL for the maximum eigenvalue of a sum of random matrices. Finally, we apply our methods to covariance matrix estimation and to estimation of sample average treatment effect under the Neyman-Rubin potential outcomes model.

Using Wasserstein Generative Adversarial Networks for the design of Monte Carlo simulations

When researchers develop new econometric methods it is common practice to compare the performance of the new methods to those of existing methods in Monte Carlo studies. The credibility of such Monte Carlo studies is often limited because of the discretion the researcher has in choosing the Monte Carlo designs reported. To improve the credibility we propose using a class of generative models that has recently been developed in the machine learning literature, termed Generative Adversarial Networks (GANs) which can be used to systematically generate artificial data that closely mimics existing datasets. Thus, in combination with existing real data sets, GANs can be used to limit the degrees of freedom in Monte Carlo study designs for the researcher, making any comparisons more convincing. In addition, if an applied researcher is concerned with the performance of a particular statistical method on a specific data set (beyond its theoretical properties in large samples), she can use such GANs to assess the performance of the proposed method, e.g. the coverage rate of confidence intervals or the bias of the estimator, using simulated data which closely resembles the exact setting of interest. To illustrate these methods we apply Wasserstein GANs (WGANs) to the estimation of average treatment effects. In this example, we find that (i) there is not a single estimator that outperforms the others in all three settings, so researchers should tailor their analytic approach to a given setting, (ii) systematic simulation studies can be helpful for selecting among competing methods in this situation, and (iii) the generated data closely resemble the actual data.

Bootstrapping inference of average treatment effect in completely randomized experiments with high-dimensional covariates

Investigators often use regression adjustment methods to analyze the results of randomized experiments when baseline covariates are available. Their aim is to improve the estimation efficiency of treatment effects by adjusting for imbalance of covariates. Under mild conditions, the regression-adjusted average treatment effect estimator is asymptotically normal with asymptotic variance no greater than that of the unadjusted estimator. The asymptotic variance can be estimated conservatively based on residual sum of squares. This article studies alternative inference methods based on the bootstrap and investigates their asymptotic properties under the Neyman–Rubin causal model and randomization-based inference framework. We show that the weighted, residual and paired bootstrap methods provide asymptotically conservative variance estimators that perform at least as good as the estimator based on residual sum of squares. We further provide counterexamples, where the original estimator is asymptotically normal, but the bootstrap counterpart is inconsistent for estimating its limiting distribution. Simulation studies indicate that the paired bootstrap method is preferable, in terms of preserving type I errors, for a small sample size. Finally, our methods analyze HER2+ breast cancer data from the NeOAdjuvant Herceptin trial to examine the effectiveness of trastuzumab in combination with neoadjuvant chemotherapy.

A comparison of reweighting estimators of average treatment effects in real world populations.

Regulatory agencies typically evaluate the efficacy and safety of new interventions and grant commercial approval based on randomized controlled trials (RCTs). Other major healthcare stakeholders, such as insurance companies and health technology assessment agencies, while basing initial access and reimbursement decisions on RCT results, are also keenly interested in whether results observed in idealized trial settings will translate into comparable outcomes in real world settings—that is, into so‐called “real world” effectiveness. Unfortunately, evidence of real world effectiveness for new interventions is not available at the time of initial approval. To bridge this gap, statistical methods are available to extend the estimated treatment effect observed in a RCT to a target population. The generalization is done by weighting the subjects who participated in a RCT so that the weighted trial population resembles a target population. We evaluate a variety of alternative estimation and weight construction procedures using both simulations and a real world data example using two clinical trials of an investigational intervention for Alzheimer's disease. Our results suggest an optimal approach to estimation depends on the characteristics of source and target populations, including degree of selection bias and treatment effect heterogeneity.

More robust estimation of average treatment effects using kernel optimal matching in an observational study of spine surgical interventions.

Inverse probability of treatment weighting (IPTW), which has been used to estimate average treatment effects (ATE) using observational data, tenuously relies on the positivity assumption and the correct specification of the treatment assignment model, both of which are problematic assumptions in many observational studies. Various methods have been proposed to overcome these challenges, including truncation, covariate-balancing propensity scores, and stable balancing weights. Motivated by an observational study in spine surgery, in which positivity is violated and the true treatment assignment model is unknown, we present the use of optimal balancing by kernel optimal matching (KOM) to estimate ATE. By uniformly controlling the conditional mean squared error of a weighted estimator over a class of models, KOM simultaneously mitigates issues of possible misspecification of the treatment assignment model and is able to handle practical violations of the positivity assumption, as shown in our simulation study. Using data from a clinical registry, we apply KOM to compare two spine surgical interventions and demonstrate how the result matches the conclusions of clinical trials that IPTW estimates spuriously refute.

On IPW-based estimation of conditional average treatment effects

The research in this paper gives a systematic investigation of the asymptotic behaviors of four inverse probability weighting (IPW)-based estimators for conditional average treatment effects, with nonparametrically, semiparametrically, parametrically estimated, and true propensity score, respectively. To this end, we first pay particular attention to semiparametric dimension reduction structure such that we can study the semiparametric-based estimator that can alleviate the curse of dimensionality and greatly avoid model misspecification. We also derive some further properties of the existing estimator with a nonparametrically estimated propensity score. According to their asymptotic variance functions, the studies reveal the general ranking of their asymptotic efficiencies; in which scenarios the asymptotic equivalence can hold; the critical roles of the affiliation of the given covariates in the set of arguments of the propensity score, the bandwidth and kernel function selections. The results show an essential difference from the IPW-based estimator of the unconditional average treatment effects(ATE). The numerical studies indicate that for high-dimensional paradigms, the semiparametric-based estimator performs well in general whereas the nonparametric-based estimator, even sometimes parametric-based estimator, is more susceptible to dimensionality. Some numerical studies are carried out to examine their performance. A real data example is analyzed for illustration.

Estimation of local treatment effects under the binary instrumental variable model

Summary Instrumental variables are widely used to deal with unmeasured confounding in observational studies and imperfect randomized controlled trials. In these studies, researchers often target the so-called local average treatment effect as it is identifiable under mild conditions. In this paper we consider estimation of the local average treatment effect under the binary instrumental variable model. We discuss the challenges of causal estimation with a binary outcome and show that, surprisingly, it can be more difficult than in the case with a continuous outcome. We propose novel modelling and estimation procedures that improve upon existing proposals in terms of model congeniality, interpretability, robustness and efficiency. Our approach is illustrated via simulation studies and a real data analysis.

Generalizability of Subgroup Effects.

Generalizability methods are increasingly used to make inferences about the effect of interventions in target populations using a study sample. Most existing methods to generalize effects from sample to population rely on the assumption that subgroup-specific effects generalize directly. However, researchers may be concerned that in fact subgroup-specific effects differ between sample and population. In this brief report, we explore the generalizability of subgroup effects. First, we derive the bias in the sample average treatment effect estimator as an estimate of the population average treatment effect when subgroup effects in the sample do not directly generalize. Next, we present a Monte Carlo simulation to explore bias due to unmeasured heterogeneity of subgroup effects across sample and population. Finally, we examine the potential for bias in an illustrative data example. Understanding the generalizability of subgroup effects may lead to increased use of these methods for making externally valid inferences of treatment effects using a study sample.

Incremental Propensity Score Effects for Time-fixed Exposures.

When causal inference is of primary interest, a range of target parameters can be chosen to define the causal effect, such as average treatment effects (ATEs). However, ATEs may not always align with the research question at hand. Furthermore, the assumptions needed to interpret estimates as ATEs, such as exchangeability, consistency, and positivity, are often not met. Here, we present the incremental propensity score (PS) approach to quantify the effect of shifting each person's exposure propensity by some predetermined amount. Compared with the ATE, incremental PS may better reflect the impact of certain policy interventions and do not require that positivity hold. Using the Nulliparous Pregnancy Outcomes Study: monitoring mothers-to-be (nuMoM2b), we quantified the relationship between total vegetable intake and the risk of preeclampsia and compared it to average treatment effect estimates. The ATE estimates suggested a reduction of between two and three preeclampsia cases per 100 pregnancies for consuming at least half a cup of vegetables per 1,000 kcal. However, positivity violations obfuscate the interpretation of these results. In contrast, shifting each woman's exposure propensity by odds ratios ranging from 0.20 to 5.0 yielded no difference in the risk of preeclampsia. Our analyses show the utility of the incremental PS effects in addressing public health questions with fewer assumptions.

The Minimax Estimator of the Average Treatment Effect, among Linear Combinations of Conditional Average Treatment Effects Estimators

I consider the estimation of the average treatment effect (ATE), in a population that can be divided into $G$ groups, and such that one has unbiased and uncorrelated estimators of the conditional average treatment effect (CATE) in each group. These conditions are for instance met in stratified randomized experiments. I assume that the outcome is homoscedastic, and that each CATE is bounded in absolute value by $B$ standard deviations of the outcome, for some known constant $B$. I derive, across all linear combinations of the CATEs' estimators, the estimator of the ATE with the lowest worst-case mean-squared error. This optimal estimator assigns a weight equal to group $g$'s share in the population to the most precisely estimated CATEs, and a weight proportional to one over the CATE's variance to the least precisely estimated CATEs. This optimal estimator is feasible: the weights only depend on known quantities. I then allow for positive covariances known up to the outcome's variance between the estimators. This condition is met in differences-in-differences designs, if errors are homoscedastic and uncorrelated. Under those assumptions, I show that the minimax estimator is still feasible and can easily be computed.

Estimation of Average Treatment Effect on Residuals: Bias Derivation.

Estimation of Average Treatment Effect on Residuals: Bias Derivation.

The Effects of Smoking on Metabolic Syndrome and Its Components Using Causal Methods in the Iranian Population.

Background:The aim of this study was to estimate the effect of smoking on metabolic syndrome (MS) and its components applying inverse probability-of-treatment weighting (IPTW) and propensity score (PS) matching.Methods:Using data from Tehran Lipid and Glucose Study, 4857 participants aged over 20 years with information on smoking and confounders in the third phase (2005–2008) were included, and the MS was assessed in the fifth phase (2011–2014). IPTW and PS matching were used to adjust for confounders.Results:Based on average treatment effect (ATE) estimates, smoking decreased the risk of hypertension (RR: 0.62; 95% CI: 0.43, 0.88), but increased the risk of low HDL cholesterol (1.20; 0.98, 1.48). Similarly, the average treatment effect in the treated (ATT) estimates using IPTW and PS matching suggested that smoking decreased the risk of hypertension (0.63; 0.52, 0.76, and 0.68; 0.54, 0.85), and increased the risk of low HDL cholesterol (1.24; 1.07, 1.43, and 1.28; 1.06, 1.54), respectively.Conclusions:Smoking seems to increase the risk of low HDL cholesterol but decreases the risk of hypertension.

Average treatment effect of robotic versus laparoscopic rectal surgery for rectal cancer.

The aim of this study is to estimate what would have happened if all patients treated with laparoscopy for rectal cancer had instead been treated with the robotic technique. To estimate the average treatment effect (ATE) of the robotic technique over the laparoscopic approach, data from patients treated at two centres between 2007 and 2018 were used to obtain counterfactual outcomes using an inverse probability weighting (IPW) adjustment. This study enrolled 261 patients, of which 177 and 84 patients had undergone robotic surgery and standard laparoscopy, respectively. After IPW adjustment, the difference between the groups was similar in the pseudo-population. The average conversion rate would fall by an estimated 6.1% if all procedures had been robotic (p=0.045). All other post-operative variables showed no differences regardless of the approach. ATE estimation suggests that robotic rectal cancer surgery could be associated with a lower conversion rate. The approach did not affect the post-operative morbidity rates or the operative time.