Estimation Of Average Treatment Effects Research Articles

Conditional average treatment effect estimation with marginally constrained models

Abstract Treatment effect estimates are often available from randomized controlled trials as a single average treatment effect for a certain patient population. Estimates of the conditional average treatment effect (CATE) are more useful for individualized treatment decision-making, but randomized trials are often too small to estimate the CATE. Examples in medical literature make use of the relative treatment effect (e.g. an odds ratio) reported by randomized trials to estimate the CATE using large observational datasets. One approach to estimating these CATE models is by using the relative treatment effect as an offset, while estimating the covariate-specific untreated risk. We observe that the odds ratios reported in randomized controlled trials are not the odds ratios that are needed in offset models because trials often report the marginal odds ratio. We introduce a constraint or a regularizer to better use marginal odds ratios from randomized controlled trials and find that under the standard observational causal inference assumptions, this approach provides a consistent estimate of the CATE. Next, we show that the offset approach is not valid for CATE estimation in the presence of unobserved confounding. We study if the offset assumption and the marginal constraint lead to better approximations of the CATE relative to the alternative of using the average treatment effect estimate from the randomized trial. We empirically show that when the underlying CATE has sufficient variation, the constraint and offset approaches lead to closer approximations to the CATE.

Federated causal inference in heterogeneous observational data.

We are interested in estimating the effect of a treatment applied to individuals at multiple sites, where data is stored locally for each site. Due to privacy constraints, individual-level data cannot be shared across sites; the sites may also have heterogeneous populations and treatment assignment mechanisms. Motivated by these considerations, we develop federated methods to draw inferences on the average treatment effects of combined data across sites. Our methods first compute summary statistics locally using propensity scores and then aggregate these statistics across sites to obtain point and variance estimators of average treatment effects. We show that these estimators are consistent and asymptotically normal. To achieve these asymptotic properties, we find that the aggregation schemes need to account for the heterogeneity in treatment assignments and in outcomes across sites. We demonstrate the validity of our federated methods through a comparative study of two large medical claims databases.

Labisia pumila standardized extract (SKF7®) reduces percentage of waist circumference and waist-to-height ratio in individuals with obesity.

Evidence from the literature points towards a viable choice of utilizing Labisia pumila to improve the metabolic profile in animal studies. To that end, this prospective study was designed to assess the health impact of the consumption of L. pumila standardized extract (SKF7®) on key parameters of obesity in humans such as body weight (BW), body mass index (BMI), waist circumference (WC) and waist-to-height ratio (WHtR). A dose-ranging analysis using SKF7® was conducted through a randomized, double-blind, multicentre, placebo-controlled, phase 2 clinical trial involving individuals with obesity (N = 133) between January 2020 and April 2021. The potential percentage of change was assessed in relation to BW, BMI, WC and WHtR. Average treatment effect estimates (treatment group vs. placebo) show a statistically significant reduction in the percentage of change for BW (mean = -2.915; CI: -4.546, -1.285), BMI (-2.921; CI: -4.551, -1.291), WC (mean = -2.187; CI: -3.784, -0.589) and WHtR (mean = -2.294, CI: -3.908, -0.681) in the group with a total of 750 mg of SKF7® (p < .01). An incremental reduction in WC and WHtR was consistent with the gradual increase in the total daily concentration of SKF7® from 375 to 750 mg. WC and WHtR had higher effect size (f 2 = 0.11 and f 2 = 0.13 respectively) in comparison with BW and BMI. SKF7® is potentially a novel therapeutictreatment for obesity, reflected by reductions in BW, BMI, WC and WHtR. The use of SKF7® suggests a dose-dependent reduction in abdominal obesity, exemplified by a decline in WC and WHtR.

Targeting for Long-Term Outcomes

Decision makers often want to target interventions so as to maximize an outcome that is observed only in the long term. This typically requires delaying decisions until the outcome is observed or relying on simple short-term proxies for the long-term outcome. Here, we build on the statistical surrogacy and policy learning literatures to impute the missing long-term outcomes and then approximate the optimal targeting policy on the imputed outcomes via a doubly robust approach. We first show that conditions for the validity of average treatment effect estimation with imputed outcomes are also sufficient for valid policy evaluation and optimization; furthermore, these conditions can be somewhat relaxed for policy optimization. We apply our approach in two large-scale proactive churn management experiments at The Boston Globe by targeting optimal discounts to its digital subscribers with the aim of maximizing long-term revenue. Using the first experiment, we evaluate this approach empirically by comparing the policy learned using imputed outcomes with a policy learned on the ground-truth, long-term outcomes. The performance of these two policies is statistically indistinguishable, and we rule out large losses from relying on surrogates. Our approach also outperforms a policy learned on short-term proxies for the long-term outcome. In a second field experiment, we implement the optimal targeting policy with additional randomized exploration, which allows us to update the optimal policy for future subscribers. Over three years, our approach had a net-positive revenue impact in the range of $4–$5 million compared with the status quo. This paper was accepted by Eric Anderson, marketing. Funding: This work was supported by Boston Globe Media. Supplemental Material: The online appendix and data are available at https://doi.org/10.1287/mnsc.2023.4881 .

Poor economics and its missing mechanisms: The case for causal mediation

AbstractA key aim of studying development is to understand the factors that shape socioeconomic progress and explain inequalities. In empirical work, the predominant focus has been on posing these questions in the language of causal inference: how one or more variables effect an outcome of interest, with the estimation of Average Treatment Effects (ATE) becoming prioritised as the key objective. The ‘credibility revolution’ and the emphasis on randomised controlled trials in research on development has cemented this dominance, because randomisation is well‐suited to estimating the ATE. This paper argues that this dual dominance—ATE as main question of interest, and experiment as preferred method—is narrow and restrictive. We propose causal mediation frameworks as an alternative, which are routinely used in disciplines including epidemiology, psychology, sociology and political science where causal mechanisms are an equally important focus. We introduce key concepts and definitions of path‐specific effects, and discuss identification and estimation approaches. We illustrate applications for development and demonstrate how causal mediation brings the focus back to contextual knowledge, combining this with empirical rigour.

Model-Assisted Complier Average Treatment Effect Estimates in Randomized Experiments with Noncompliance

ABSTRACT Noncompliance is a common problem in randomized experiments in various fields. Under certain assumptions, the complier average treatment effect is identifiable and equal to the ratio of the intention-to-treat effects of the potential outcomes to that of the treatment received. To improve the estimation efficiency, we propose three model-assisted estimators for the complier average treatment effect in randomized experiments with a binary outcome. We study their asymptotic properties, compare their efficiencies with that of the Wald estimator, and propose the Neyman-type conservative variance estimators to facilitate valid inferences. Moreover, we extend our methods and theory to estimate the multiplicative complier average treatment effect. Our analysis is randomization-based, allowing the working models to be misspecified. Finally, we conduct simulation studies to illustrate the advantages of the model-assisted methods and apply these analysis methods in a randomized experiment to evaluate the effect of academic services or incentives on academic performance.

Unreliable Continuous Treatment Indicators in Propensity Score Analysis

Propensity score analyses (PSA) of continuous treatments often operationalize the treatment as a multi-indicator composite, and its composite reliability is unreported. Latent variables or factor scores accounting for this unreliability are seldom used as alternatives to composites. This study examines the effects of the unreliability of indicators of a latent treatment in PSA using the generalized propensity score (GPS). A Monte Carlo simulation study was conducted varying composite reliability, continuous treatment representation, variability of factor loadings, sample size, and number of treatment indicators to assess whether Average Treatment Effect (ATE) estimates differed in their relative bias, Root Mean Squared Error, and coverage rates. Results indicate that low composite reliability leads to underestimation of the ATE of latent continuous treatments, while the number of treatment indicators and variability of factor loadings show little effect on ATE estimates, after controlling for overall composite reliability. The results also show that, in correctly specified GPS models, the effects of low composite reliability can be somewhat ameliorated by using factor scores that were estimated including covariates. An illustrative example is provided using survey data to estimate the effect of teacher adoption of a workbook related to a virtual learning environment in the classroom.

The Effect: An Introduction to Research Design and CausalityNickHuntington‐KleinChapman &amp; Hall/CRC, 2022, xiv + 620 pages, $39.95, paperback. ISBN: 9781032125787

International Statistical ReviewEarly View Book Review The Effect: An Introduction to Research Design and Causality , Nick Huntington-Klein Chapman & Hall/CRC, 2022, xiv + 620 pages, $39.95, paperback. ISBN: 9781032125787 Brian W. Sloboda, Corresponding Author Brian W. Sloboda [email protected] Department of Economics and Finance, 1616 McCormick Drive, University of Maryland, Global Campus, Largo, 20774 Maryland, USASearch for more papers by this author Brian W. Sloboda, Corresponding Author Brian W. Sloboda [email protected] Department of Economics and Finance, 1616 McCormick Drive, University of Maryland, Global Campus, Largo, 20774 Maryland, USASearch for more papers by this author First published: 21 June 2023 https://doi.org/10.1111/insr.12547Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL No abstract is available for this article. References Angrist, J. D., and Pischke, J. S. (2009). Mostly Harmless Econometrics: An Empiricist's Companion. Princeton: Princeton University Press. Angrist, J. D., and Pischke, J. S. (2014). Mastering' Metrics: The Path from Cause to Effect. Princeton: Princeton University Press. Athey, S., and Imbens, G. W. (2017). The State of Applied Econometrics: Causality and Policy Evaluation. Journal of Economic Perspectives, 31(2), 3– 32. Holland, P. W. (1986). Statistics and Causal Inference. Journal of the American Statistical Association, 81(396), 945– 960. Imbens, G. W., and Angrist, J. D. (1994). Identification and estimation of local average treatment effects. Econometrica: Journal of the Econometric Society, 467– 475 62. Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, 688– 701. Early ViewOnline Version of Record before inclusion in an issue ReferencesRelatedInformation

A new covariate selection strategy for high dimensional data in causal effect estimation with multivariate treatments

Selection of covariates is crucial in the estimation of average treatment effects given observational data with high or even ultra-high dimensional pretreatment variables. Existing methods for this problem typically assume sparse linear models for both outcome and univariate treatment, and cannot handle situations with ultra-high dimensional covariates. In this paper, we propose a new covariate selection strategy called double screening prior adaptive lasso (DSPAL) to select confounders and predictors of the outcome for multivariate treatments, which combines the adaptive lasso method with the marginal conditional (in)dependence prior information to select target covariates, in order to eliminate confounding bias and improve statistical efficiency. The distinctive features of our proposal are that it can be applied to high-dimensional or even ultra-high dimensional covariates for multivariate treatments, and can deal with the cases of both parametric and nonparametric outcome models, which makes it more robust compared to other methods. Our theoretical analyses show that the proposed procedure enjoys the sure screening property, the ranking consistency property and the variable selection consistency. Through a simulation study, we demonstrate that the proposed approach selects all confounders and predictors consistently and estimates the multivariate treatment effects with smaller bias and mean squared error compared to several alternatives under various scenarios. In real data analysis, the method is applied to estimate the causal effect of a three-dimensional continuous environmental treatment on cholesterol level and enlightening results are obtained.

How does a local instrumental variable method perform across settings with instruments of differing strengths? A simulation study and an evaluation of emergency surgery.

Local instrumental variable (LIV) approaches use continuous/multi-valued instrumental variables (IV) to generate consistent estimates of average treatment effects (ATEs) and Conditional Average Treatment Effects (CATEs). There is little evidence on how LIV approaches perform according to the strength of the IV or with different sample sizes. Our simulation study examined the performance of an LIV method, and a two-stage least squares (2SLS) approach across different sample sizes and IV strengths. We considered four 'heterogeneity' scenarios: homogeneity, overt heterogeneity (over measured covariates), essential heterogeneity (unmeasured), and overt and essential heterogeneity combined. In all scenarios, LIV reported estimates with low bias even with the smallest sample size, provided that the instrument was strong. Compared to 2SLS, LIV provided estimates for ATE and CATE with lower levels of bias and Root Mean Squared Error. With smaller sample sizes, both approaches required stronger IVs to ensure low bias. We considered both methods in evaluating emergency surgery (ES) for three acute gastrointestinal conditions. Whereas 2SLS found no differences in the effectiveness of ES according to subgroup, LIV reported that frailer patients had worse outcomes following ES. In settings with continuous IVs of moderate strength, LIV approaches are better suited than 2SLS to estimate policy-relevant treatment effect parameters.

Comparing the effects of four common drug classes on the progression of mild cognitive impairment to dementia using electronic health records

The objective of this study was to investigate the potential association between the use of four frequently prescribed drug classes, namely antihypertensive drugs, statins, selective serotonin reuptake inhibitors, and proton-pump inhibitors, and the likelihood of disease progression from mild cognitive impairment (MCI) to dementia using electronic health records (EHRs). We conducted a retrospective cohort study using observational EHRs from a cohort of approximately 2 million patients seen at a large, multi-specialty urban academic medical center in New York City, USA between 2008 and 2020 to automatically emulate the randomized controlled trials. For each drug class, two exposure groups were identified based on the prescription orders documented in the EHRs following their MCI diagnosis. During follow-up, we measured drug efficacy based on the incidence of dementia and estimated the average treatment effect (ATE) of various drugs. To ensure the robustness of our findings, we confirmed the ATE estimates via bootstrapping and presented associated 95% confidence intervals (CIs). Our analysis identified 14,269 MCI patients, among whom 2501 (17.5%) progressed to dementia. Using average treatment estimation and bootstrapping confirmation, we observed that drugs including rosuvastatin (ATE = − 0.0140 [− 0.0191, − 0.0088], p value < 0.001), citalopram (ATE = − 0.1128 [− 0.125, − 0.1005], p value < 0.001), escitalopram (ATE = − 0.0560 [− 0.0615, − 0.0506], p value < 0.001), and omeprazole (ATE = − 0.0201 [− 0.0299, − 0.0103], p value < 0.001) have a statistically significant association in slowing the progression from MCI to dementia. The findings from this study support the commonly prescribed drugs in altering the progression from MCI to dementia and warrant further investigation.

Inverse Probability Weighting Estimation in Completely Randomized Experiments

In addition to treatment assignments and observed outcomes, covariate information is often available prior to randomization in completely randomized experiments that compare an active treatment versus control. The analysis of covariance (ANCOVA) method is commonly applied to adjust for baseline covariates in order to improve precision. We focus on making propensity score-based adjustment to covariates under the completely randomized design in a finite population of experimental units with two treatment groups. We study inverse probability weighting (IPW) estimation of the finite-population average treatment effect for a general class of working propensity score models, which includes generalized linear models for binary data. We provide randomization-based asymptotic analysis of the propensity score approach and explore the finite-population asymptotic behaviors of two IPW estimators of the average treatment effect. We identify a condition under which propensity score-based covariate adjustment is asymptotically equivalent to an ANCOVA-based covariate adjustment and improves precision compared with a simple unadjusted comparison between treatment and control arms. In particular, when the working propensity score is fitted by a generalized linear model for binary data with an intercept term, the asymptotic variance of the IPW estimators is the same for any link function, including identity link, logit link, probit link, and complementary log-log link. We demonstrate these methods using an HIV clinical trial and a post-traumatic stress disorder study. Finally, we present a simulation study comparing the finite-sample performance of IPW and other methods for both continuous and binary outcomes. Supplementary materials for this article are available online.

Neuroevolutionary representations for learning heterogeneous treatment effects

Within the field of causal inference, we consider the problem of estimating heterogeneous treatment effects from data. We propose and validate a novel approach for learning feature representations to aid the estimation of the conditional average treatment effect or cate. Our method focuses on an intermediate layer in a neural network trained to predict the outcome from the features. In contrast to previous approaches that encourage the distribution of representations to be treatment-invariant, we leverage a genetic algorithm that optimizes over representations useful for predicting the outcome to select those less useful for predicting the treatment. This allows us to retain information within the features useful for predicting the outcome even if that information may be related to treatment assignment. We validate our method on synthetic examples and illustrate its use on a real life dataset. This paper extends work previously presented at the International Conference on Computational Science in London (Burkhart and Ruiz, 2022).

Effect of farm machinery adoption on household income and food security: evidence from a nationwide household survey in India

An increase in farm productivity and income is crucially linked to farm mechanization. Labour shortage in rural areas and frequent extreme weather events have made a resounding pitch for the deepening of farm mechanization in India. Our study examines the effect of agricultural machinery adoption on net agricultural income, household income, and household consumption in India; the latter is considered as an indicator of food security. A nationwide household survey of 10253 households drawn from the majority of states of India has been used. To assess the impact of machinery adoption on the outcome variables, the endogenous switching regression (ESR) model is used. Of the three types of machineries investigated, such as tractors, electric pumps, and diesel pumps, the average adoption rates are found to be low in the country, i.e., 8% for tractors, 24% for electric pumps, and 14% for diesel pumps. The wide heterogeneity in terms of adoption is mostly rooted in the large geographical differences among the states of India, as machinery adoption is concentrated in the north and north-central plains of the country. The first stage of ESR, which is a probit model, reveals the determinants of mechanization as access to credit, climatic shocks, access to irrigation, and farmer cooperative membership. The average treatment effect estimates from the second stage outcome regression reveal that the adoption of machineries has increased net agricultural income by 31%, household income by 19%, and food consumption by 5%. Given these demonstrable positive effects of mechanization in agriculture, there is a need for deepening of low-cost scale appropriate farm machines suitable for small holders. Government support would play a pivotal role in this direction by making farm credits accessible to smallholders, providing timely subsidies on machinery purchase, and providing required extension services.

Inference on subgroups identified based on a heterogeneous treatment effect in a post hoc analysis of a clinical trial.

Due to the many benefits of understanding treatment effect heterogeneity in a clinical trial, an exploratory post hoc subgroup analysis is often performed to find subpopulations of patients with conditional average treatment effect that suggests better treatment efficacy than in the overall population. A naive re-substitution approach uses all available data to identify a subgroup and then proceeds with estimation and inference using the same data set. This approach generally leads to an overly optimistic estimate of conditional average treatment effect. In this article, in a post hoc analysis, we estimate the target optimal subgroup through maximizing a utility function, from candidates systematically identified with a penalized regression. We then compare two resampling-based bias-correction methods, cross-validation and debiasing bootstrap, for obtaining approximately unbiased estimates and valid inference of conditional average treatment effect in the identified subgroup, with either an empirical or an augmented estimator. Our results show that both the cross-validation and the debiasing bootstrap methods reduce the re-substitution bias effectively. The cross-validation method appears to have less biased point estimates, smaller standard error estimates, but poorer coverages than the debiasing bootstrap method when using the empirical estimator and the sample size is moderate. Using the augmented estimator in the debiasing bootstrap method leads to less biased point estimates but poorer coverages. We conclude that bias correction should be a part of every exploratory post hoc subgroup analysis to eliminate re-substitution bias and to obtain a proper confidence interval for the estimated conditional average treatment effect in the selected subgroup.

Overall average treatment effects from clinical trials, one-variable-at-a-time subgroup analyses and predictive approaches to heterogeneous treatment effects: Toward a more patient-centered evidence-based medicine.

Despite the predominance of the evidence-based medicine paradigm, a fundamental incongruity remains: Evidence is derived from groups of people, yet medical decisions are made by and for individuals. Randomization ensures the comparability of treatment groups within a clinical trial, which allows for unbiased estimation of average treatment effects. If we treated groups of patients instead of individuals, or if patients with the same disease were identical to one another in all factors that determined the harms and the benefits of therapy, then these group-level averages would make a perfectly sound foundation for medical decision-making. But patients differ from one another in many ways that determine the likelihood of an outcome, both with and without a treatment. Nevertheless, popular approaches to evidence-based medicine have encouraged a reliance on the average treatment effects estimated from clinical trials and meta-analysis as guides to decision-making for individuals. Here, we discuss the limitations of this approach as well as limitations of conventional, one-variable-at-a-time subgroup analysis; finally, we discuss the rationale for "predictive" approaches to heterogeneous treatment effects. Predictive approaches to heterogeneous treatment effects combine methods for causal inference (e.g. randomization) with methods for prediction that permit inferences about which patients are likely to benefit and which are not, taking into account multiple relevant variables simultaneously to yield "personalized" estimates of benefit-harm trade-offs. We focus on risk modeling approaches, which rely on the mathematical dependence of the absolute treatment effect with the baseline risk, which varies substantially "across patients" in most trials. While there are a number of examples of risk modeling approaches that have been practice-changing, risk modeling does not provide ideal estimates of individual treatment effects, since risk modeling does not account for how individual variables might modify the effects of therapy. In "effect modeling," prediction models are developed directly on clinical trial data, including terms for treatment and treatment effect interactions. These more flexible approaches may better uncover individualized treatment effects, but are also prone to overfitting when dimensionality is high, power is low, and there is limited prior knowledge about effect modifiers.

Properties of least squares estimator in estimation of average treatment effects

Treatment effects are often estimated by the least squares estimator controlling for some covariates. This paper investigates its properties. When the propensity score is constant, it is a consistent estimator of the average treatment effects if it is viewed as a semiparametric partially linear regression estimator, but it is not necessarily more efficient than the simple difference-of-means estimator. If it is literally viewed as a least squares estimator with a finite number of controls, it is equal to the weighted average of conditional average treatment effects with potentially negative weights, although the negative weight issue does not exist under semiparametric interpretation. It is shown that the negative weight issue can be avoided by use of logit specification.

Propensity score analysis with local balance.

Most propensity score (PS) analysis methods rely on a correctly specified parametric PS model, which may result in biased estimation of the average treatment effect (ATE) when the model is misspecified. More flexible nonparametric models for treatment assignment alleviate this issue, but they do not always guarantee covariate balance. Methods that force balance in the means of covariates and their transformations between the treatment groups, termed global balance in this article, do not always lead to unbiased estimation of ATE. Their estimated propensity scores only ensure global balance but not the balancing property, which is defined as the conditional independence between treatment assignment and covariates given the propensity score. The balancing property implies not only global balance but also local balance-the mean balance of covariates in propensity score stratified sub-populations. Local balance implies global balance, but the reverse is false. We propose the propensity score with local balance (PSLB) methodology, which incorporates nonparametric propensity score models and optimizes local balance. Extensive numerical studies showed that the proposed method can substantially outperform existing methods that estimate the propensity score by optimizing global balance, when the model is misspecified. The proposed method is implemented in the R package PSLB.

Estimating heterogeneous causal effects in observational studies using small area predictors

The official statistics produced by National Statistical Institutes are mainly used by policy makers to take decisions. In particular, when policy makers and decision takers would like to know the impact of a given policy, it is important to acknowledge the heterogeneity of the treatment effects for different domains. If the domain of interest is small with regard to its sample size, then the evaluator has entered the small area estimation (SAE) dilemma. Based on the modification of the Inverse Propensity Weighting estimator and the traditional small area predictors, new estimators of area specific average treatment effects are proposed for unplanned domains. A robustified version of the predictor against presence of the outliers is also developed. Analytical Mean Squared Error (MSE) estimators of the proposed predictors are derived. These methods provide a tool to map the policy impacts that can help to better target the treatment group(s). The properties of these small area estimators are illustrated by means of a design-based simulation using a real data set where the aim is to study the effects of permanent versus temporary contracts on the economic insecurity of households in different regions of Italy.

Complex discontinuity designs using covariates: Impact of school grade retention on later life outcomes in Chile

Regression discontinuity designs are extensively used for causal inference in observational studies. However, they are usually confined to settings with simple treatment rules and determined by a single running variable with a single cutoff. Motivated by the problem of estimating the impact of grade retention on educational and juvenile crime outcomes in Chile, we propose a framework and methods for complex discontinuity designs that encompass multiple treatment rules. In this framework the observed covariates play a central role for identification, estimation, and generalization of causal effects. Identification is nonparametric and relies on a local strong ignorability assumption. Estimation proceeds, as in any observational study, under strong ignorability, yet in a neighborhood of the cutoffs of the running variables. We discuss estimation approaches based on matching and weighting, including complementary regression modeling adjustments. We present assumptions for generalization, that is, for identification and estimation of average treatment effects for target populations. We also describe two approaches to select the neighborhood for analysis. We find that grade retention in Chile has a negative impact on future grade retention but is not associated with dropping out of school or committing a juvenile crime.