Estimation Of Average Treatment Effects Research Articles

High-speed rails and knowledge productivity: A global perspective

This study finds empirical macro-economic evidence for the effect of high-speed rails (HSRs) on national innovation through the analyses of 59 high- and upper-middle-income countries. Knowledge productivity (KP) is assumed to be represented by annual patent applications per million capita. The associations between HSR development and KP are analyzed with three methods: (1) estimation of KP function using 10-year panel data, covering 14 countries with HSRs; (2) estimation of average treatment effect using difference-in-differences analysis for a treatment group of 15 and a control group of 44 countries; and (3) estimation of average treatment effect using matching techniques for a dataset covering 29 countries, of which 10 received treatment. The results of all three analyses covering about 40 years of HSRs' history, robustly suggest that HSR development can have positive externalities on KP both in the short and long runs. In addition, it is found that the HSRs’ impacts on countries with upper-middle income and countries with high urban population growth are larger than other countries. They suggest that HSR development can be a part of national innovation policy and that upper-middle-income countries, highly urbanizing countries, and countries with insufficient inter-regional transportation systems can have advantages in HSR introduction for promoting knowledge productivity.

PNS265 Incorporating Context in Observational Real-World DATA Using Targeted Learning

The integration of context into the data-generating process (DGP) in observational studies drives innovation in causal structural frameworks. A common older approach was to pre-specify a DGP and use directed acyclic graphs (DAGs) to guide the minimally sufficient subset of covariates. Advances in Targeted Learning (TL) allow us to estimate DGP via super learners (SL), weighted ensembles of machine learning algorithms. The TL approach supports a larger predictor space to draw upon and reduces potential for model misspecification, improving reproducibility. We compare TL and DAG-directed approaches, using synthetic data, generated from a realistic in silico DGP, and extending examples using Real-World Data (RWD). We simulate data from a given DAG and compare three methods: 1) targeted maximum likelihood estimation (TMLE), 2) traditional treatment weight approaches (stabilized propensity score weights) with a correctly specified model [matching our DGP], and 3) repeating 2) with variations of model misspecification. We then reproduce these on a real-world example. We assess performance by calling additional, more advanced machine-learning libraries into the super learner. In our examples, TMLE improves estimation of propensity scores and reduces biases. This latter effect is particularly apparent in our RWD example, where balancing via inverse treatment weights is problematic due to pathologic distribution of the propensity scores’ domain of common support. The correct model specification, which contains the process that generated the data, is crucial to obtain unbiased estimates of the actual average treatment effects (ATE). This should be incorporated into the model. However, in practice, models often are not selected based on DGP; and modeling choices could result in different answers to the same research question. TMLE provides a rigorous foundation that is less subject to these kinds of manipulations and robust to model misspecification.

Estimation of Conditional Average Treatment Effects With High-Dimensional Data

Given the unconfoundedness assumption, we propose new nonparametric estimators for the reduced dimensional conditional average treatment effect (CATE) function. In the first stage, the nuisance functions necessary for identifying CATE are estimated by machine learning methods, allowing the number of covariates to be comparable to or larger than the sample size. The second stage consists of a low-dimensional local linear regression, reducing CATE to a function of the covariate(s) of interest. We consider two variants of the estimator depending on whether the nuisance functions are estimated over the full sample or over a hold-out sample. Building on Belloni at al. and Chernozhukov et al., we derive functional limit theory for the estimators and provide an easy-to-implement procedure for uniform inference based on the multiplier bootstrap. The empirical application revisits the effect of maternal smoking on a baby’s birth weight as a function of the mother’s age.

The Counterfactual χ-GAN: Finding comparable cohorts in observational health data.

Causal inference often relies on the counterfactual framework, which requires that treatment assignment is independent of the outcome, known as strong ignorability. Approaches to enforcing strong ignorability in causal analyses of observational data include weighting and matching methods. Effect estimates, such as the average treatment effect (ATE), are then estimated as expectations under the re-weighted or matched distribution, P. The choice of P is important and can impact the interpretation of the effect estimate and the variance of effect estimates. In this work, instead of specifying P, we learn a distribution that simultaneously maximizes coverage and minimizes variance of ATE estimates. In order to learn this distribution, this research proposes a generative adversarial network (GAN)-based model called the Counterfactual χ-GAN (cGAN), which also learns feature-balancing weights and supports unbiased causal estimation in the absence of unobserved confounding. Our model minimizes the Pearson χ2-divergence, which we show simultaneously maximizes coverage and minimizes the variance of importance sampling estimates. To our knowledge, this is the first such application of the Pearson χ2-divergence. We demonstrate the effectiveness of cGAN in achieving feature balance relative to established weighting methods in simulation and with real-world medical data.

Impact of individual and neighborhood social capital on the physical and mental health of pregnant women: the Japan Environment and Children\u2019s Study (JECS)

BackgroundPrevious studies revealed positive, negative, and no influence of social capital on the health outcomes of pregnant women. It was considered that such differences were caused by the disparities of outcome measures and sample sizes between studies. Our chief aim was to verify the positive influence of social capital on the health condition of pregnant women using established health outcome measures and large-scale nationwide survey data.MethodsWe employed questionnaire survey data from 79,210 respondents to the Japan Environment and Children’s Study, and physical and mental component summary scores from the 8-Item Short-Form Health Survey as outcome measures. We estimated the effect of individual and neighborhood social capitals on physical and mental component summary scores. To consider the property that the richness of social capital would be generally determined by individual characteristics, and to estimate the causal influence of social capital on health without bias caused by said property, we adopted average treatment effect estimation with inverse probability weighting. Generally, average treatment effects are based on the difference of average outcomes between treated and untreated groups in an intervention. In this research, we reckoned individuals’ different levels of social capital as a kind of non-randomized treatment for respective individuals, and we applied average treatment effect estimation. The analysis regarded pregnant women with the lowest level of social capital as untreated samples and women with other levels of social capitals as treated samples.ResultsFor mental component summary score, the maximum average treatment effects in the comparison between the lowest and highest levels of social capital were approximately 4.4 and 1.6 for individual and neighborhood social capital, respectively. The average treatment effects for the physical component summary score were negligible for both social capital types.ConclusionsSocial capital particularly contributes to improving mental component summary score in pregnant women. The likelihood of a mentally healthy pregnancy may be increased by enhancing social capital.

Does open peer review improve citation count? Evidence from a propensity score matching analysis of PeerJ

This study aims to investigate whether open peer review can improve citation count. Articles published in PeerJ during 2013 and 2015 were chosen as the data set. Two categories of the articles were generated: articles with closed peer review history and articles with open peer review history. A propensity score matching with the radius matching method was performed using 14 confounding variables. The other five common matching methods of propensity score matching, the bias-adjusted matching estimator developed by Abadie and Imbens (Simple and bias-corrected matching estimators for average treatment effects, National Bureau of Economic Research, Cambridge, pp 1–57, 2002), and the data set excluding articles with an extremely high citation count were used to check the robustness of the results. The results of this study showed that articles with open peer review history could be expected to have significantly greater citation counts than articles with closed peer review history. Our results suggest that open peer review can improve citation count, and that the best practice for open peer review might be a compromise open peer review.

Average treatment effects for stayers with correlated random coefficient models of panel data

SummaryCorrelated random coefficient (CRC) models provide a useful framework for estimating average treatment effects (ATE) with panel data by accommodating heterogeneous treatment effects and flexible patterns of selection. In their simplest form, they lead to the well‐known difference‐in‐differences estimator. CRC models yield estimates of ATE for “movers” (i.e., cross‐sectional units whose treatment status changed over time) while ATE for “stayers” (i.e., cross‐sectional units who retained the same treatment status over time) are not identified. We study additional restrictions on selection into treatment that lead to the identification of ATE for stayers by an extrapolation from quantities identified by the CRC model. We discuss estimation and testing of the extrapolation's validity, then use our results to estimate the returns to agricultural technology adoption among maize farmers in Kenya.

Propensity score specification for optimal estimation of average treatment effect with binary response.

Propensity score methods are commonly used in statistical analyses of observational data to reduce the impact of confounding bias in estimations of average treatment effect. While the propensity score is defined as the conditional probability of a subject being in the treatment group given that subject's covariates, the most precise estimation of average treatment effect results from specifying the propensity score as a function of true confounders and predictors only. This property has been demonstrated via simulation in multiple prior research articles. However, we have seen no theoretical explanation as to why this should be so. This paper provides that theoretical proof. Furthermore, this paper presents a method for performing the necessary variable selection by means of elastic net regression, and then estimating the propensity scores so as to obtain optimal estimates of average treatment effect. The proposed method is compared against two other recently introduced methods, outcome-adaptive lasso and covariate balancing propensity score. Extensive simulation analyses are employed to determine the circumstances under which each method appears most effective. We applied the proposed methods to examine the effect of pre-cardiac surgery coagulation indicator on mortality based on a linked dataset from a retrospective review of 1390 patient medical records at Jewish Hospital (Louisville, KY) with the Society of Thoracic Surgeons database.

Estimation and Validation of Ratio-based Conditional Average Treatment Effects Using Observational Data

While sample sizes in randomized clinical trials are large enough to estimate the average treatment effect well, they are often insufficient for estimation of treatment-covariate interactions critical to studying data-driven precision medicine. Observational data from real world practice may play an important role in alleviating this problem. One common approach in trials is to predict the outcome of interest with separate regression models in each treatment arm, and estimate the treatment effect based on the contrast of the predictions. Unfortunately, this simple approach may induce spurious treatment-covariate interaction in observational studies when the regression model is misspecified. Motivated by the need of modeling the number of relapses in multiple sclerosis (MS) patients, where the ratio of relapse rates is a natural choice of the treatment effect, we propose to estimate the conditional average treatment effect (CATE) as the ratio of expected potential outcomes, and derive a doubly robust estimator of this CATE in a semiparametric model of treatment-covariate interactions. We also provide a validation procedure to check the quality of the estimator on an independent sample. We conduct simulations to demonstrate the finite sample performance of the proposed methods, and illustrate their advantages on real data by examining the treatment effect of dimethyl fumarate compared to teriflunomide in MS patients. Supplementary materials for this article are available online.

Doubly robust treatment effect estimation with missing attributes

Missing attributes are ubiquitous in causal inference, as they are in most applied statistical work. In this paper we consider various sets of assumptions under which causal inference is possible despite missing attributes and discuss corresponding approaches to average treatment effect estimation, including generalized propensity score methods and multiple imputation. Across an extensive simulation study, we show that no single method systematically outperforms others. We find, however, that doubly robust modifications of standard methods for average treatment effect estimation with missing data repeatedly perform better than their nondoubly robust baselines; for example, doubly robust generalized propensity score methods beat inverse-weighting with the generalized propensity score. This finding is reinforced in an analysis of an observational study on the effect on mortality of tranexamic acid administration among patients with traumatic brain injury in the context of critical care management. Here, doubly robust estimators recover confidence intervals that are consistent with evidence from randomized trials, whereas nondoubly robust estimators do not.

Regression-adjusted average treatment effect estimates in stratified randomized experiments

Summary Linear regression is often used in the analysis of randomized experiments to improve treatment effect estimation by adjusting for imbalances of covariates in the treatment and control groups. This article proposes a randomization-based inference framework for regression adjustment in stratified randomized experiments. We re-establish, under mild conditions, the finite-population central limit theorem for a stratified experiment, and we prove that both the stratified difference-in-means estimator and the regression-adjusted average treatment effect estimator are consistent and asymptotically normal; the asymptotic variance of the latter is no greater and typically less than that of the former. We also provide conservative variance estimators that can be used to construct large-sample confidence intervals for the average treatment effect.

Does the estimation of the propensity score by machine learning improve matching estimation? The case of Germany's programmes for long term unemployed

Matching-type estimators using the propensity score are the major workhorse in active labour market policy evaluation. This work investigates if machine learning algorithms for estimating the propensity score lead to more credible estimation of average treatment effects on the treated using a radius matching framework. Considering two popular methods, the results are ambiguous: We find that using LASSO based logit models to estimate the propensity score delivers more credible results than conventional methods in small and medium sized high dimensional datasets. However, the usage of Random Forests to estimate the propensity score may lead to a deterioration of the performance in situations with a low treatment share. The application reveals a positive effect of the training programme on days in employment for long-term unemployed. While the choice of the “first stage” is highly relevant for settings with low number of observations and few treated, machine learning and conventional estimation becomes more similar in larger samples and higher treatment shares.

Robust and efficient semi-supervised estimation of average treatment effects with application to electronic health records data.

We consider the problem of estimating the average treatment effect (ATE) in a semi-supervised learning setting, where a very small proportion of the entire set of observations are labeled with the true outcome but features predictive of the outcome are available among all observations. This problem arises, for example, when estimating treatment effects in electronic health records (EHR) data because gold-standard outcomes are often not directly observable from the records but are observed for a limited number of patients through small-scale manual chart review. We develop an imputation-based approach for estimating the ATE that is robust to misspecification of the imputation model. This effectively allows information from the predictive features to be safely leveraged to improve efficiency in estimating the ATE. The estimator is additionally doubly-robust in that it is consistent under correct specification of either an initial propensity score model or a baseline outcome model. It is also locally semiparametric efficient under an ideal semi-supervised model where the distribution of the unlabeled data is known. Simulations exhibit the efficiency and robustness of the proposed method compared to existing approaches in finite samples. We illustrate the method by comparing rates of treatment response to two biologic agents for treatment inflammatory bowel disease using EHR data from Partners'Healthcare.

Rerandomization Strategies for Balancing Covariates Using Pre-Experimental Longitudinal Data

ABSTRACTThis article considers experimental design based on the strategy of rerandomization to increase the efficiency in experiments. Two aspects of rerandomization are addressed. First, we propose a two-stage allocation sample scheme for randomization inference to the units in experiments that guarantees that the difference-in-mean estimator is an unbiased estimator of the sample average treatment effect for any experiment, conserves the exactness of randomization inference, and halves the time consumption of the rerandomization design. Second, we propose a rank-based covariate-balance measure which can take into account the estimated relative weight of each covariate. Several strategies for estimating these weights using pre-experimental data are proposed. Using Monte Carlo simulations, the proposed strategies are compared to complete randomization and Mahalanobis-based rerandomization. An empirical example is given where the power of a mean difference test of electricity consumption of 54 households is increased by 99%, in comparison to complete randomization, using one of the proposed designs based on high frequency longitudinal electricity consumption data. Supplementary materials for this article are available online.

Reducing Interference Bias in Online Marketplace Pricing Experiments

Online marketplace designers frequently run A/B tests to measure the impact of proposed product changes. However, given that marketplaces are inherently connected, total average treatment effect estimates obtained through Bernoulli randomized experiments are often biased due to violations of the stable unit treatment value assumption. This can be particularly problematic for experiments that impact sellers' strategic choices, affect buyers' preferences over items in their consideration set, or change buyers' consideration sets altogether. In this work, we measure and reduce bias due to interference in online marketplace experiments by using observational data to creating clusters of similar listings, and then using those clusters to conduct cluster-randomized field experiments. We provide a lower bound on the magnitude of bias due to interference by conducting a meta-experiment that randomizes over two experiment designs: one Bernoulli randomized, one cluster randomized. In both meta-experiment arms, treatment sellers are subject to a different platform fee policy than control sellers, resulting in different prices for buyers. By conducting a joint analysis of the two meta-experiment arms, we find a large and statistically significant difference between the total average treatment effect estimates obtained with the two designs, and estimate that 32.60% of the Bernoulli-randomized treatment effect estimate is due to interference bias. We also find weak evidence that the magnitude and/or direction of interference bias depends on extent to which a marketplace is supply- or demand-constrained, and analyze a second meta-experiment to highlight the difficulty of detecting interference bias when treatment interventions require intention-to-treat analysis.

Model-assisted inference for treatment effects using regularized calibrated estimation with high-dimensional data

Consider the problem of estimating average treatment effects when a large number of covariates are used to adjust for possible confounding through outcome regression and propensity score models. We develop new methods and theory to obtain not only doubly robust point estimators for average treatment effects, which remain consistent if either the propensity score model or the outcome regression model is correctly specified, but also model-assisted confidence intervals, which are valid when the propensity score model is correctly specified but the outcome model may be misspecified. With a linear outcome model, the confidence intervals are doubly robust, that is, being also valid when the outcome model is correctly specified but the propensity score model may be misspecified. Our methods involve regularized calibrated estimators with Lasso penalties but carefully chosen loss functions, for fitting propensity score and outcome regression models. We provide high-dimensional analysis to establish the desired properties of our methods under comparable sparsity conditions to previous results, which give valid confidence intervals when both the propensity score and outcome models are correctly specified. We present simulation studies and an empirical application which demonstrate advantages of the proposed methods compared with related methods based on regularized maximum likelihood estimation. The methods are implemented in the R package $\mathtt{RCAL}$.

Inference on mean quality-adjusted lifetime using joint models for continuous quality of life process and time to event

The estimated average treatment effect in observational studies is biased if the assumptions of ignorability and overlap are not satisfied. To deal with this potential problem when propensity score weights are used in the estimation of the treatment effects, in this paper we propose a bootstrap bias correction estimator for the average treatment effect (ATE) obtained with the inverse propensity score (BBC-IPS) estimator. We show in simulations that the BBC-IPC performs well when we have misspecifications of the propensity score (PS) due to: omitted variables (ignorability property may not be satisfied), overlap (imbalances in distribution between treatment and control groups) and confounding effects between observables and unobservables (endogeneity). Further refinements in bias reductions of the ATE estimates in smaller samples are attained by iterating the BBC-IPS estimator.

Regression discontinuity design with many thresholds

Numerous empirical studies employ regression discontinuity designs with multiple cutoffs and heterogeneous treatments. A common practice is to normalize all the cutoffs to zero and estimate one effect. This procedure identifies the average treatment effect (ATE) on the observed distribution of individuals local to existing cutoffs. However, researchers often want to make inferences on more meaningful ATEs, computed over general counterfactual distributions of individuals, rather than simply the observed distribution of individuals local to existing cutoffs. This paper proposes a consistent and asymptotically normal estimator for such ATEs when heterogeneity follows a non-parametric function of cutoff characteristics in the sharp case. The proposed estimator converges at the minimax optimal rate of root-n for a specific choice of tuning parameters. Identification in the fuzzy case, with multiple cutoffs, is impossible unless heterogeneity follows a finite-dimensional function of cutoff characteristics. Under parametric heterogeneity, this paper proposes an ATE estimator for the fuzzy case that optimally combines observations to maximize its precision.

Estimation of average treatment effects using panel data when treatment effect heterogeneity depends on unobserved fixed effects

SummaryThis paper proposes a new panel data approach to identify and estimate the time‐varying average treatment effect (ATE). The approach allows for treatment effect heterogeneity that depends on unobserved fixed effects. In the presence of this type of heterogeneity, existing panel data approaches identify the ATE for limited subpopulations only. In contrast, the proposed approach identifies and estimates the ATE for the entire population. The approach relies on the linear fixed effects specification of potential outcome equations and uses exogenous variables that are correlated with the fixed effects. I apply the approach to study the impact of a mother's smoking during pregnancy on her child's birth weight.

On the estimation of average treatment effects with right-censored time to event outcome and competing risks.

We are interested in the estimation of average treatment effects based on right-censored data of an observational study. We focus on causal inference of differences between t-year absolute event risks in a situation with competing risks. We derive doubly robust estimation equations and implement estimators for the nuisance parameters based on working regression models for the outcome, censoring, and treatment distribution conditional on auxiliary baseline covariates. We use the functional delta method to show that these estimators are regular asymptotically linear estimators and estimate their variances based on estimates of their influence functions. In empirical studies, we assess the robustness of the estimators and the coverage of confidence intervals. The methods are further illustrated using data from a Danish registrystudy.