Estimation Of Treatment Effects Research Articles

A practical guide to estimating treatment effects in patients with rheumatic diseases using real-world data.

Randomized controlled trials are considered the gold standard in study methodology. However, due to their study design and inclusion criteria, these studies may not capture the heterogeneity of real-world patient populations. In contrast, the lack of randomization and the presence of both measured and unmeasured confounding factors could bias the estimated treatment effect when using observational data. While causal inference methods allow for the estimation of treatment effects, their mathematical complexity may hinder their application in clinical research. We present a practical, nontechnical guide using a common statistical package (Stata) and a motivational simulated dataset that mirrors real-world observational data from patients with rheumatic diseases. We demonstrate regression analysis, regression adjustment, inverse-probability weighting, propensity score (PS) matching and two robust estimation methods. Although the methods applied to control for confounding factors produced similar results, the commonly used one-to-one PS matching method could yield biased results if not thoroughly assessed. The guide we propose aims to facilitate the use of readily available methods in a common statistical package. It may contribute to robust and transparent epidemiological and statistical methods, thereby enhancing effectiveness research using observational data in rheumatology.

Estimation of average treatment effects for massively unbalanced binary outcomes

. The MLE of the ATE in the logit model for binary outcomes may have a significant second-order bias if the event has a low probability, which is the case we focus on in this article. We derive the second-order bias of the logit ATE estimator, and we propose a bias-corrected estimator of the ATE. We also propose a variation on the logit model with parameters that are elasticities. Finally, we propose a computational trick that avoids numerical instability in the case of estimation for rare events.

Abadie’s Kappa and Weighting Estimators of the Local Average Treatment Effect

Recent research has demonstrated the importance of flexibly controlling for covariates in instrumental variables estimation. In this article we study the finite sample and asymptotic properties of various weighting estimators of the local average treatment effect (LATE), motivated by Abadie’s kappa theorem and offering the requisite flexibility relative to standard practice. We argue that two of the estimators under consideration, which are weight normalized, are generally preferable. Several other estimators, which are unnormalized, do not satisfy the properties of scale invariance with respect to the natural logarithm and translation invariance, thereby exhibiting sensitivity to the units of measurement when estimating the LATE in logs and the centering of the outcome variable more generally. We also demonstrate that, when noncompliance is one sided, certain weighting estimators have the advantage of being based on a denominator that is strictly greater than zero by construction. This is the case for only one of the two normalized estimators, and we recommend this estimator for wider use. We illustrate our findings with a simulation study and three empirical applications, which clearly document the sensitivity of unnormalized estimators to how the outcome variable is coded. We implement the proposed estimators in the Stata package kappalate.

Adaptively leverage multiple real-world data sources for treatment effect estimation based on similarity

ABSTRACT The incorporation of real-world data (RWD) into medical product development and evaluation has exhibited consistent growth. However, there is no universally adopted method of how much information to borrow from external data. This paper proposes a study design methodology called Tree-based Monte Carlo (TMC) that dynamically integrates patients from various RWD sources to calculate the treatment effect based on the similarity between clinical trial and RWD. Initially, a propensity score is developed to gauge the resemblance between clinical trial data and each real-world dataset. Utilizing this similarity metric, we construct a hierarchical clustering tree that delineates varying degrees of similarity between each RWD source and the clinical trial data. Ultimately, a Gaussian process methodology is employed across this hierarchical clustering framework to synthesize the projected treatment effects of the external group. Simulation result shows that our clustering tree could successfully identify similarity. Data sources exhibiting greater similarity with clinical trial are accorded higher weights in treatment estimation process, while less congruent sources receive comparatively lower emphasis. Compared with another Bayesian method, meta-analytic predictive prior (MAP), our proposed method’s estimator is closer to the true value and has smaller bias.

The effect of carbon farming training on food security and development resilience in Northern Ghana

ABSTRACT Carbon farming has recently been advocated for as climate change and variability mitigation and/or adaptation strategy in global agriculture. In this study, we address an important research question of whether carbon farming training can improve household resilience capacity as well as food security by employing internationally standardized indicators. Household resilience capacity and its components are measured using the Food and Agriculture Organisation (FAO)’s resilience capacity index while the food security measures used include household and child food insecurity experience scale (FIES and CFIES), food consumption score (FCS) and household dietary diversity score (HDDS). We relied on doubly robust treatment effect estimators to account for potential selection bias and heterogeneity. We find that carbon farming training has no statistically significant effect on overall household resilience capacity. However, we find a large and statistically significant effect on key components of resilience (specifically, access to basic services, assets and social safety nets) and a marginal improvement in adaptive capacity. We also find statistically significant effect on FCS and HDDS but not for the other food security indicators (FIES and CFIES). Overall, the results suggest that agricultural training programs, particularly climate change adaptation capacity building initiatives, could improve important welfare measures in developing countries.

Evaluating the comparability of osteoporosis treatments using propensity score and negative control outcome methods in UK and Denmark electronic health record databases.

Evidence on the comparative effectiveness of osteoporosis treatments is heterogeneous. This may be attributed to different populations and clinical practice, but also to differing methodologies ensuring comparability of treatment groups before treatment effect estimation and the amount of residual confounding by indication. This study assessed the comparability of denosumab vs oral bisphosphonate (OBP) groups using propensity score (PS) methods and negative control outcome (NCO) analysis. A total of 280 288 women aged ≥50yr initiating denosumab or OBP in 2011-2018 were included from the UK Clinical Practice Research Datalink (CPRD) and the Danish National Registries (DNR). Balance of observed covariates was assessed using absolute standardized mean difference (ASMD) before and after PS weighting, matching, and stratification, with ASMD >0.1 indicating imbalance. Residual confounding was assessed using NCOs with ≥100 events. Hazard ratio (HR) and 95%CI between treatment and NCO were estimated using Cox models. Presence of residual confounding was evaluated with 2 approaches (1) >5% of NCOs with 95% CI excluding 1, (2) >5% of NCOs with an upper CI <0.75 or lower CI >1.3. The number of imbalanced covariates before adjustment (CPRD 22/87; DNR 18/83) decreased, with 2%-11% imbalance remaining after weighting, matching, or stratification. Using approach 1, residual confounding was present for all PS methods in both databases (≥8% of NCOs), except for stratification in DNR (3.8%). Using approach 2, residual confounding was present in CPRD with PS matching (5.3%) and stratification (6.4%), but not with weighting (4.3%). Within DNR, no NCOs had HR estimates with upper or lower CI limits beyond the specified bounds indicating residual confounding for any PS method. Achievement of covariate balance and determination of residual bias were dependent upon several factors including the population under study, PS method, prevalence of NCO, and the threshold indicating residual confounding.

Analyzing adverse effects of subway extension on housing prices in affluent urban neighborhoods

This study examines the effects of subway extension on housing prices in affluent urban neighborhoods, focusing on the Q-line extension in Manhattan, New York City. Utilizing synthetic controls and treatment effects estimators, distinct pricing trends across property types are revealed, particularly condominiums. The observed pricing dynamics deviate from the assumption that increasing supplies are associated with price discounts. Moreover, the connection between price discounts and noise complaints is not observed, highlighting the significant role of demand-side factors, especially neighborhood characteristics, in shaping housing prices. An unexpected demographic shift is observed, suggesting that the Q-line extension may disproportionately benefit non-white and low-income groups, challenging the prevailing gentrification narrative. We use the term “inverse premiumization” to denote the phenomenon where anticipated price increases in affluent neighborhoods due to transit improvements fail to materialize. Furthermore, our analysis of speculative behavior reveals a spike in short-term growth during the public notice period, gradually slowing down during construction and operation phases. These findings offer the nuanced adverse effects of subway extension on housing prices, contributing to our understanding of short-to medium-term price premiums and discounts. These insights are key considerations for city planners and policymakers navigating urban development, housing market, and speculative behavior dynamics.

Variance estimation for average treatment effects estimated by g-computation

AbstractThe average treatment effect is used to evaluate effects of interventions in a population. Under certain causal assumptions, such an effect may be estimated from observational data using the g-computation technique. The asymptotic properties of this estimator appears not to be well-known and hence bootstrapping has become the preferred method for estimating its variance. Bootstrapping is, however, not an optimal choice for multiple reasons; it is a slow procedure and, if based on too few bootstrap samples, results in a highly variable estimator of the variance. In this paper, we consider estimators of potential outcome means and average treatment effects using g-computation. We consider these parameters for the entire population but also in subgroups, for example, the average treatment effect among the treated. We derive their asymptotic distributions in a general framework. An estimator of the asymptotic variance is proposed and shown to be consistent when g-computation is used in conjunction with the M-estimation technique. The proposed estimator is shown to be superior to the bootstrap technique in a simulation study. Robustness against model misspecification is also demonstrated by means of simulations.

Estimating treatment effect in randomized trial after control to treatment crossover using external controls

ABSTRACT In clinical trials, it is common to design a study that permits the administration of an experimental treatment to participants in the placebo or standard of care group post primary endpoint. This is often seen in the open-label extension phase of a phase III, pivotal study of the new medicine, where the focus is on assessing long-term safety and efficacy. With the availability of external controls, proper estimation and inference of long-term treatment effect during the open-label extension phase in the absence of placebo-controlled patients are now feasible. Within the framework of causal inference, we propose several difference-in-differences (DID) type methods and a synthetic control method (SCM) for the combination of randomized controlled trials and external controls. Our realistic simulation studies demonstrate the desirable performance of the proposed estimators in a variety of practical scenarios. In particular, DID methods outperform SCM and are the recommended methods of choice. An empirical application of the methods is demonstrated through a phase III clinical trial in rare disease.

Doubly robust proximal synthetic controls.

To infer the treatment effect for a single treated unit using panel data, synthetic control (SC) methods construct a linear combination of control units' outcomes that mimics the treated unit's pre-treatment outcome trajectory. This linear combination is subsequently used to impute the counterfactual outcomes of the treated unit had it not been treated in the post-treatment period, and used to estimate the treatment effect. Existing SC methods rely on correctly modeling certain aspects of the counterfactual outcome generating mechanism and may require near-perfect matching of the pre-treatment trajectory. Inspired by proximal causal inference, we obtain two novel nonparametric identifying formulas for the average treatment effect for the treated unit: one is based on weighting, and the other combines models for the counterfactual outcome and the weighting function. We introduce the concept of covariate shift to SCs to obtain these identification results conditional on the treatment assignment. We also develop two treatment effect estimators based on these two formulas and generalized method of moments. One new estimator is doubly robust: it is consistent and asymptotically normal if at least one of the outcome and weighting models is correctly specified. We demonstrate the performance of the methods via simulations and apply them to evaluate the effectiveness of a pneumococcal conjugate vaccine on the risk of all-cause pneumonia in Brazil.

Sequential covariate-adjusted randomization via hierarchically minimizing Mahalanobis distance and marginal imbalance.

In comparative studies, covariate balance and sequential allocation schemes have attracted growing academic interest. Although many theoretically justified adaptive randomization methods achieve the covariate balance, they often allocate patients in pairs or groups. To better meet the practical requirements where the clinicians cannot wait for other participants to assign the current patient for some economic or ethical reasons, we propose a method that randomizes patients individually and sequentially. The proposed method conceptually separates the covariate imbalance, measured by the newly proposed modified Mahalanobis distance, and the marginal imbalance, that is the sample size difference between the 2 groups, and it minimizes them with an explicit priority order. Compared with the existing sequential randomization methods, the proposed method achieves the best possible covariate balance while maintaining the marginal balance directly, offering us more control of the randomization process. We demonstrate the superior performance of the proposed method through a wide range of simulation studies and real data analysis, and also establish theoretical guarantees for the proposed method in terms of both the convergence of the imbalance measure and the subsequent treatment effect estimation.

Robustness of response-adaptive randomization.

Doubly adaptive biased coin design (DBCD), a response-adaptive randomization scheme, aims to skew subject assignment probabilities based on accrued responses for ethical considerations. Recent years have seen substantial advances in understanding DBCD's theoretical properties, assuming correct model specification for the responses. However, concerns have been raised about the impact of model misspecification on its design and analysis. In this paper, we assess the robustness to both design model misspecification and analysis model misspecification under DBCD. On one hand, we confirm that the consistency and asymptotic normality of the allocation proportions can be preserved, even when the responses follow a distribution other than the one imposed by the design model during the implementation of DBCD. On the other hand, we extensively investigate three commonly used linear regression models for estimating and inferring the treatment effect, namely difference-in-means, analysis of covariance (ANCOVA) I, and ANCOVA II. By allowing these regression models to be arbitrarily misspecified, thereby not reflecting the true data generating process, we derive the consistency and asymptotic normality of the treatment effect estimators evaluated from the three models. The asymptotic properties show that the ANCOVA II model, which takes covariate-by-treatment interaction terms into account, yields the most efficient estimator. These results can provide theoretical support for using DBCD in scenarios involving model misspecification, thereby promoting the widespread application of this randomization procedure.

KG-TREAT: Pre-training for Treatment Effect Estimation by Synergizing Patient Data with Knowledge Graphs.

Treatment effect estimation (TEE) is the task of determining the impact of various treatments on patient outcomes. Current TEE methods fall short due to reliance on limited labeled data and challenges posed by sparse and high-dimensional observational patient data. To address the challenges, we introduce a novel pre-training and fine-tuning framework, KG-TREAT, which synergizes large-scale observational patient data with biomedical knowledge graphs (KGs) to enhance TEE. Unlike previous approaches, KG-TREAT constructs dual-focus KGs and integrates a deep bi-level attention synergy method for in-depth information fusion, enabling distinct encoding of treatment-covariate and outcome-covariate relationships. KG-TREAT also incorporates two pre-training tasks to ensure a thorough grounding and contextualization of patient data and KGs. Evaluation on four downstream TEE tasks shows KG-TREAT's superiority over existing methods, with an average improvement of 7% in Area under the ROC Curve (AUC) and 9% in Influence Function-based Precision of Estimating Heterogeneous Effects (IF-PEHE). The effectiveness of our estimated treatment effects is further affirmed by alignment with established randomized clinical trial findings.

Exploiting Geometry for Treatment Effect Estimation via Optimal Transport

Estimating treatment effects from observational data suffers from the issue of confounding bias, which is induced by the imbalanced confounder distributions between the treated and control groups. As an effective approach, re-weighting learns a group of sample weights to balance the confounder distributions. Existing methods of re-weighting highly rely on a propensity score model or moment alignment. However, for complex real-world data, it is difficult to obtain an accurate propensity score prediction. Although moment alignment is free of learning a propensity score model, accurate estimation for high-order moments is computationally difficult and still remains an open challenge, and first and second-order moments are insufficient to align the distributions and easy to be misled by outliers. In this paper, we exploit geometry to capture the intrinsic structure involved in data for balancing the confounder distributions, so that confounding bias can be reduced even with outliers. To achieve this, we construct a connection between treatment effect estimation and optimal transport, a powerful tool to capture geometric information. After that, we propose an optimal transport model to learn sample weights by extracting geometry from confounders, in which geometric information between groups and within groups is leveraged for better confounder balancing. A projected mirror descent algorithm is employed to solve the derived optimization problem. Experimental studies on both synthetic and real-world datasets demonstrate the effectiveness of our proposed method.

Adversarially Balanced Representation for Continuous Treatment Effect Estimation

Individual treatment effect (ITE) estimation requires adjusting for the covariate shift between populations with different treatments, and deep representation learning has shown great promise in learning a balanced representation of covariates. However the existing methods mostly consider the scenario of binary treatments. In this paper, we consider the more practical and challenging scenario in which the treatment is a continuous variable (e.g. dosage of a medication), and we address the two main challenges of this setup. We propose the adversarial counterfactual regression network (ACFR) that adversarially minimizes the representation imbalance in terms of KL divergence, and also maintains the impact of the treatment value on the outcome prediction by leveraging an attention mechanism. Theoretically we demonstrate that ACFR objective function is grounded in an upper bound on counterfactual outcome prediction error. Our experimental evaluation on semi-synthetic datasets demonstrates the empirical superiority of ACFR over a range of state-of-the-art methods.

Uncertainty Quantification in Heterogeneous Treatment Effect Estimation with Gaussian-Process-Based Partially Linear Model

Estimating heterogeneous treatment effects across individuals has attracted growing attention as a statistical tool for performing critical decision-making. We propose a Bayesian inference framework that quantifies the uncertainty in treatment effect estimation to support decision-making in a relatively small sample size setting. Our proposed model places Gaussian process priors on the nonparametric components of a semiparametric model called a partially linear model. This model formulation has three advantages. First, we can analytically compute the posterior distribution of a treatment effect without relying on the computationally demanding posterior approximation. Second, we can guarantee that the posterior distribution concentrates around the true one as the sample size goes to infinity. Third, we can incorporate prior knowledge about a treatment effect into the prior distribution, improving the estimation efficiency. Our experimental results show that even in the small sample size setting, our method can accurately estimate the heterogeneous treatment effects and effectively quantify its estimation uncertainty.

Continuous Treatment Effect Estimation Using Gradient Interpolation and Kernel Smoothing

We address the Individualized continuous treatment effect (ICTE) estimation problem where we predict the effect of any continuous valued treatment on an individual using ob- servational data. The main challenge in this estimation task is the potential confounding of treatment assignment with in- dividual’s covariates in the training data, whereas during in- ference ICTE requires prediction on independently sampled treatments. In contrast to prior work that relied on regularizers or unstable GAN training, we advocate the direct approach of augmenting training individuals with independently sam- pled treatments and inferred counterfactual outcomes. We in- fer counterfactual outcomes using a two-pronged strategy: a Gradient Interpolation for close-to-observed treatments, and a Gaussian Process based Kernel Smoothing which allows us to down weigh high variance inferences. We evaluate our method on five benchmarks and show that our method out- performs six state-of-the-art methods on the counterfactual estimation error. We analyze the superior performance of our method by showing that (1) our inferred counterfactual re- sponses are more accurate, and (2) adding them to the train- ing data reduces the distributional distance between the con- founded training distribution and test distribution where treat- ment is independent of covariates. Our proposed method is model-agnostic and we show that it improves ICTE accuracy of several existing models.

Simultaneous treatment effect estimation and variable selection for observational data

Due to the difficulties inherent in conducting controlled experiments, recent causal inference studies have focused on developing treatment effect estimation using observational data. One major difficulty in causal inference from observational data is that the underlying causal structure is unknown. This may result in the misidentification of potential sources of causal estimation bias, such as confounders, which must be controlled for accurate estimation. To consider all possible confounders and other relevant information, conventional methods predominantly use all observed covariates indiscriminately. However, previous studies have shown that including all covariates without considering their causal relationships may deteriorate the estimation performance. Although several data-driven variable selection methods have been proposed for treatment effect estimation, they cannot distinguish the confounders from other outcome-related covariates and are limited to simple regression forms. In this study, we propose a method called the Variable Selection Causal Estimation Network (VSCEN) that performs treatment effect estimation and causal variable selection simultaneously. Through end-to-end differentiable training, the VSCEN selects only the covariates beneficial for effect estimation and uses only those selected for effect estimation. Experimental evaluations on fully synthetic, semi-synthetic, and real datasets demonstrate the VSCEN’s superior performance in conditional average treatment effect estimation and competitive performance in average treatment effect estimation along with its accurate variable selection capabilities.

Contrast-specific propensity scores for causal inference with multiple interventions.

Existing methods that use propensity scores for heterogeneous treatment effect estimation on non-experimental data do not readily extend to the case of more than two treatment options. In this work, we develop a new propensity score-based method for heterogeneous treatment effect estimation when there are three or more treatment options, and prove that it generates unbiased estimates. We demonstrate our method on a real patient registry of patients in Singapore with diabetic dyslipidemia. On this dataset, our method generates heterogeneous treatment recommendations for patients among three options: Statins, fibrates, and non-pharmacological treatment to control patients' lipid ratios (total cholesterol divided by high-density lipoprotein level). In our numerical study, our proposed method generated more stable estimates compared to a benchmark method based on a multi-dimensional propensity score.

No star is good news: A unified look at rerandomization based on [formula omitted]-values from covariate balance tests

Randomized experiments balance all covariates on average and are considered the gold standard for estimating treatment effects. Chance imbalances are nonetheless common in realized treatment allocations. To inform readers of the comparability of treatment groups at baseline, contemporary scientific publications often report covariate balance tables with not only covariate means by treatment group but also the associated p-values from significance tests of their differences. The practical need to avoid small p-values as indicators of poor balance motivates balance check and rerandomization based on these p-values from covariate balance tests (ReP) as an attractive tool for improving covariate balance in designing randomized experiments. Despite the intuitiveness of such strategy and its possibly already widespread use in practice, the literature lacks results about its implications on subsequent inference, subjecting many effectively rerandomized experiments to possibly inefficient analyses. To fill this gap, we examine a variety of potentially useful schemes for ReP and quantify their impact on subsequent inference. Specifically, we focus on three estimators of the average treatment effect from the unadjusted, additive, and interacted linear regressions of the outcome on treatment, respectively, and derive their asymptotic sampling properties under ReP. The main findings are threefold. First, the estimator from the interacted regression is asymptotically the most efficient under all ReP schemes examined, and permits convenient regression-assisted inference identical to that under complete randomization. Second, ReP, in contrast to complete randomization, improves the asymptotic efficiency of the estimators from the unadjusted and additive regressions. Standard regression analyses are accordingly still valid but in general overconservative. Third, ReP reduces the asymptotic conditional biases of the three estimators and improves their coherence in terms of mean squared difference. These results establish ReP as a convenient tool for improving covariate balance in designing randomized experiments, and we recommend using the interacted regression for analyzing data from ReP designs.