Estimation Of Treatment Effects Research Articles

Does livestock-crop diversification improve food and nutrition security? Evidence from Eastern Ethiopia

ABSTRACT Pastoral and agro-pastoral households are the most vulnerable parts of the population to climate change-induced shocks and they are increasingly subject to acute food and nutrition insecurity. We identified drivers of livestock-crop diversification and examined its effects on improving households’ food and nutrition security using cross sectional data collected from 258 households across five districts in the East Hararghe zone in Ethiopia. Endogenous Switching Regression (ESR) analysis was used to identify factors associated with the households’ food and nutrition security outcomes and estimate the causal effects of livestock-crop diversification on the households’ food and nutrition security. Our results show that land certification, mobile phone ownership, and access to pastoral training centres were positively and significantly associated with livestock-crop diversification. In contrast, being located in Meyu Muluke and Kumbi districts were negatively and significantly associated with livestock-crop diversification, compared to Chinaksen district. The average treatment effect estimations suggest that livestock-crop diversification has positively and significantly improved diversified households’ nutritional security outcome. The findings also reveal that non-diversified households’ food insecurity would have reduced had they diversified. Nonetheless, livestock-crop diversification is neither enough to pull farm households fully out of food and nutrition insecurity nor the only factor influencing farm households’ food and nutrition security outcomes. Hence, there is also a need to strengthen access to physical infrastructure, credit facilities, extension services, provision of farm inputs, and pastoral training centres to reduce food and nutrition insecurity in the study area.

Re-weighting the randomized controlled trial for generalization: finite-sample error and variable selection

Abstract Randomized controlled trials (RCTs) may suffer from limited scope. In particular, samples may be unrepresentative: some RCTs over- or under-sample individuals with certain characteristics compared to the target population, for which one wants conclusions on treatment effectiveness. Re-weighting trial individuals to match the target population can improve the treatment effect estimation. In this work, we establish the expressions of the bias and variance of such re-weighting procedures—also called inverse propensity of sampling weighting (IPSW)—in presence of categorical covariates for any sample size. Such results allow us to compare the theoretical performance of different versions of IPSW estimates. Besides, our results show how the performance (bias, variance, and quadratic risk) of IPSW estimates depends on the two sample sizes (RCT and target population). A by-product of our work is the proof of consistency of IPSW estimates. In addition, we analyse how including covariates that are not necessary for identifiability of the causal effect may impact the asymptotic variance. Including covariates that are shifted between the two samples but not treatment-effect modifiers increases the variance while non-shifted but treatment-effect modifiers do not. We illustrate all the takeaways in a didactic example, and on a semi-synthetic simulation inspired from critical care medicine.

Efficient designs for three-sequence stepped wedge trials with continuous recruitment.

The standard approach to designing stepped wedge trials that recruit participants in a continuous stream is to divide time into periods of equal length. But the choice of design in such cases is infinitely more flexible: each cluster could cross from the control to the intervention at any point on the continuous time-scale. We consider the case of a stepped wedge design with clusters randomised to just three sequences (designs with small numbers of sequences may be preferred for their simplicity and practicality) and investigate the choice of design that minimises the variance of the treatment effect estimator under different assumptions about the intra-cluster correlation. We make some simplifying assumptions in order to calculate the variance: in particular that we recruit the same number of participants, , from each cluster over the course of the trial, and that participants present at regularly spaced intervals. We consider an intra-cluster correlation that decays exponentially with separation in time between the presentation of two individuals from the same cluster, from a value of for two individuals who present at the same time, to a value of for individuals presenting at the start and end of the trial recruitment interval. We restrict attention to three-sequence designs with centrosymmetry - the property that if we reverse time and swap the intervention and control conditions then the design looks the same. We obtain an expression for the variance of the treatment effect estimator adjusted for effects of time, using methods for generalised least squares estimation, and we evaluate this expression numerically for different designs, and for different parameter values. There is a two-dimensional space of possible three-sequence, centrosymmetric stepped wedge designs with continuous recruitment. The variance of the treatment effect estimator for given and can be plotted as a contour map over this space. The shape of this variance surface depends on and on the parameter , but typically indicates a broad, flat region of close-to-optimal designs. The 'standard' design with equally spaced periods and 1:1:1 allocation rarely performs well, however. In many different settings, a relatively simple design can be found (e.g. one based on simple fractions) that offers close-to-optimal efficiency in that setting. There may also be designs that are robustly efficient over a wide range of settings. Contour maps of the kind we illustrate can help guide this choice. If efficiency is offered as one of the justifications for using a stepped wedge design, then it is worth designing with optimal efficiency in mind.

Matched vs Nonmatched Placebos in a Randomized Trial of COVID-19 Treatments

Matched placebo interventions are complex and resource intensive. Recent evidence suggests matched placebos may not always be necessary. Previous studies have predominantly evaluated potential bias of nonmatched placebos (ie, differing on dose, frequency of administration, or formulation) in pain and mental health, but to date no systematic examination has been conducted in infectious disease. To test for differences between nonmatched and matched placebo arms with respect to clinical outcome measures across multiple therapeutics for COVID-19. In a comparative effectiveness research study, a post hoc analysis was conducted of data on individual patients enrolled in a large, multiarm, platform randomized clinical trial in symptomatic adult outpatients with COVID-19 between January 15, 2021, to September 28, 2023, in which the outcomes of both matched and nonmatched placebo groups were reported. Bayesian and frequentist covariate-adjusted techniques were compared with 7 intervention-placebo pairs. Seven matched and nonmatched placebo pairs (for a total of 7 comparisons) were evaluated throughout the primary platform trial. Comparisons were made between treatment and its associated matched (concurrent) placebo, as well as with nonmatched placebo (alone and in combination) assessed at a similar time point. Outcomes assessed included hospitalizations, EuroQol 5-Dimension 5-level scores, and PROMIS Global-10 scores. A total of 7 intervention-control pairs (N = 2684) were assessed, including 1620 (60.4%) women, with mean (SD) age, 47 (15.2) years; the most common comorbidities were obesity (41.9%) and hypertension (37.9%). In a meta-analysis with decoupled SEs, accounting for overlapping placebo patients, the overall odds ratio (OR) of nonmatched compared with matched placebo was 1.01 (95% credible interval, 0.77-1.32), with posterior probability of equivalence, defined as 0.8 ≤ OR ≤ 1.2 (a deviation from perfect equivalence ie, OR = 1, by no more than 0.2) of 85.4%, implying no significant difference. Unadjusted analysis of the event rate difference between all nonmatched and matched placebo groups did not identify any notable differences across all 7 treatment-placebo combinations assessed. Similar analysis that was conducted for patient-reported quality of life outcomes did not yield statistically significant differences. In this post hoc study of a randomized clinical platform trial, pooling matched and nonmatched placebo patient data did not lead to inconsistencies in treatment effect estimation for any of the investigational drugs. These findings may have significant implications for future platform trials, as the use of nonmatched placebo may improve statistical power, or reduce barriers to placebo implementation.

Estimation of the Local Conditional Tail Average Treatment Effect*

The conditional tail average treatment effect (CTATE) is defined as a difference between the conditional tail expectations (CTE’s) of potential outcomes, which can capture heterogeneity and deliver aggregated local information on treatment effects over different quantile levels and is closely related to the notion of second-order stochastic dominance and the Lorenz curve. These properties render it a valuable tool for policy evaluation. In this paper, we study estimation of the CTATE locally for a group of compliers (local CTATE or LCTATE) under the two-sided noncompliance framework. We consider a semiparametric treatment effect framework under endogeneity for the LCTATE estimation using a newly introduced class of consistent loss functions jointly for the CTE and quantile. We establish the asymptotic theory of our proposed LCTATE estimator and provide an efficient algorithm for its implementation. We then apply the method to evaluate the effects of participating in programs under the Job Training Partnership Act in the US.

Hierarchical modelling of variance components makes analysis of resolvable incomplete block designs more efficient

The standard approach to variance component estimation in linear mixed models for alpha designs is the residual maximum likelihood (REML) method. One drawback of the REML method in the context of incomplete block designs is that the block variance may be estimated as zero, which can compromise the recovery of inter-block information and hence reduce the accuracy of treatment effects estimation. Due to the development of statistical and computational methods, there is an increasing interest in adopting hierarchical approaches to analysis. In order to increase the precision of the analysis of individual trials laid out as alpha designs, we here make a proposal to create an objectively informed prior distribution for variance components for replicates, blocks and plots, based on the results of previous (historical) trials. We propose different modelling approaches for the prior distributions and evaluate the effectiveness of the hierarchical approach compared to the REML method, which is classically used for analysing individual trials in two-stage approaches for multi-environment trials.

TESTING LIMITED OVERLAP

Extreme propensity scores arise in observational studies when treated and control units have very different characteristics. This is commonly referred to as limited overlap. In this paper, we propose a formal statistical test that helps assess the degree of limited overlap. Rejecting the null hypothesis in our test indicates either no or very mild degree of limited overlap and hence reassures that standard treatment effect estimators will be well behaved. One distinguishing feature of our test is that it only requires the use of a few extreme propensity scores, which is in stark contrast to other methods that require consistent estimates of some tail index. Without the need to extrapolate using observations far away from the tail, our procedure is expected to exhibit excellent size properties, a result that is also borne out in our simulation study.

Learning optimal biomarker-guided treatment policy for chronic disorders.

Electroencephalogram (EEG) provides noninvasive measures of brain activity and is found to be valuable for the diagnosis of some chronic disorders. Specifically, pre-treatment EEG signals in the alpha and theta frequency bands have demonstrated some association with antidepressant response, which is well-known to have a low response rate. We aim to design an integrated pipeline that improves the response rate of patients with major depressive disorder by developing a treatment policy guided by the resting state pre-treatment EEG recordings and other treatment effects modifiers. First, we design an innovative automatic site-specific EEG preprocessing pipeline to extract features with stronger signals than raw data. We then estimate the conditional average treatment effect (CATE) using causal forests and use a doubly robust technique to improve efficiency in the estimation of the average treatment effect. We present evidence of heterogeneity in the treatment effect and the modifying power of the EEG features, as well as a significant average treatment effect, a result that cannot be obtained with conventional methods. Finally, we employ an efficient policy learning algorithm to learn an optimal depth-2 treatment assignment decision tree and compare its performance with Q-Learning and outcome-weighted learning via simulation studies and an application to a large multi-site, double-blind, randomized controlled clinical trial, EMBARC.

Online causal inference with application to near real-time post-market vaccine safety surveillance.

Streaming data routinely generated by social networks, mobile or web applications, e-commerce, and electronic health records present new opportunities to monitor the impact of an intervention on an outcome via causal inference methods. However, most existing causal inference methods have been focused on and applied to static data, that is, a fixed data set in which observations are pooled and stored before performing statistical analysis. There is thus a pressing need to turn static causal inference into online causal learning to support near real-time monitoring of treatment effects. In this paper, we present a framework for online estimation and inference of treatment effects that can incorporate new information as it becomes available without revisiting prior observations. We show that, under mild regularity conditions, the proposed online estimator is asymptotically equivalent to the offline oracle estimator obtained by pooling all data. Our proposal is motivated by the need for near real-time vaccine effectiveness and safety monitoring, and our proposed method is applied to a case study on COVID-19 vaccine safety surveillance.

CURE: A deep learning framework pre-trained on large-scale patient data for treatment effect estimation

CURE: A deep learning framework pre-trained on large-scale patient data for treatment effect estimation

A simple method to construct circular repeated measurement design classes and efficiently control carry over effects

Repeated measurement designs (RMDs) are economical and commonly utilized in pharmacology, psychology, and animal science experiments. However, RMDs might have carryover effects, which become the main source of bias in estimation of treatment effects. Carry over effects can economically be controlled with the use of minimal balanced RMDs, strongly balanced RMDs or their generalized classes. No general procedure is available in literature to obtain these important classes of RMDs. An R-Package has been developed in this article to check the given values v and p for the possible existence of minimal circular balanced and strongly balanced RMDs (along with their generalized classes), generate the sets of shifts for possible designs, and to calculate the efficiency of separability and carry over effects for the generated design.

A practical guide to estimating treatment effects in patients with rheumatic diseases using real-world data.

Randomized controlled trials are considered the gold standard in study methodology. However, due to their study design and inclusion criteria, these studies may not capture the heterogeneity of real-world patient populations. In contrast, the lack of randomization and the presence of both measured and unmeasured confounding factors could bias the estimated treatment effect when using observational data. While causal inference methods allow for the estimation of treatment effects, their mathematical complexity may hinder their application in clinical research. We present a practical, nontechnical guide using a common statistical package (Stata) and a motivational simulated dataset that mirrors real-world observational data from patients with rheumatic diseases. We demonstrate regression analysis, regression adjustment, inverse-probability weighting, propensity score (PS) matching and two robust estimation methods. Although the methods applied to control for confounding factors produced similar results, the commonly used one-to-one PS matching method could yield biased results if not thoroughly assessed. The guide we propose aims to facilitate the use of readily available methods in a common statistical package. It may contribute to robust and transparent epidemiological and statistical methods, thereby enhancing effectiveness research using observational data in rheumatology.

Estimation of average treatment effects for massively unbalanced binary outcomes

The MLE of the ATE in the logit model for binary outcomes may have a significant second-order bias if the event has a low probability, which is the case we focus on in this article. We derive the second-order bias of the logit ATE estimator, and we propose a bias-corrected estimator of the ATE. We also propose a variation on the logit model with parameters that are elasticities. Finally, we propose a computational trick that avoids numerical instability in the case of estimation for rare events.

Abadie’s Kappa and Weighting Estimators of the Local Average Treatment Effect

Recent research has demonstrated the importance of flexibly controlling for covariates in instrumental variables estimation. In this article we study the finite sample and asymptotic properties of various weighting estimators of the local average treatment effect (LATE), motivated by Abadie’s kappa theorem and offering the requisite flexibility relative to standard practice. We argue that two of the estimators under consideration, which are weight normalized, are generally preferable. Several other estimators, which are unnormalized, do not satisfy the properties of scale invariance with respect to the natural logarithm and translation invariance, thereby exhibiting sensitivity to the units of measurement when estimating the LATE in logs and the centering of the outcome variable more generally. We also demonstrate that, when noncompliance is one sided, certain weighting estimators have the advantage of being based on a denominator that is strictly greater than zero by construction. This is the case for only one of the two normalized estimators, and we recommend this estimator for wider use. We illustrate our findings with a simulation study and three empirical applications, which clearly document the sensitivity of unnormalized estimators to how the outcome variable is coded. We implement the proposed estimators in the Stata package kappalate.

Adaptively leverage multiple real-world data sources for treatment effect estimation based on similarity

ABSTRACT The incorporation of real-world data (RWD) into medical product development and evaluation has exhibited consistent growth. However, there is no universally adopted method of how much information to borrow from external data. This paper proposes a study design methodology called Tree-based Monte Carlo (TMC) that dynamically integrates patients from various RWD sources to calculate the treatment effect based on the similarity between clinical trial and RWD. Initially, a propensity score is developed to gauge the resemblance between clinical trial data and each real-world dataset. Utilizing this similarity metric, we construct a hierarchical clustering tree that delineates varying degrees of similarity between each RWD source and the clinical trial data. Ultimately, a Gaussian process methodology is employed across this hierarchical clustering framework to synthesize the projected treatment effects of the external group. Simulation result shows that our clustering tree could successfully identify similarity. Data sources exhibiting greater similarity with clinical trial are accorded higher weights in treatment estimation process, while less congruent sources receive comparatively lower emphasis. Compared with another Bayesian method, meta-analytic predictive prior (MAP), our proposed method’s estimator is closer to the true value and has smaller bias.

Causal Effect Estimation on Imaging and Clinical Data for Treatment Decision Support of Aneurysmal Subarachnoid Hemorrhage.

Aneurysmal subarachnoid hemorrhage is a serious medical emergency of brain that has high mortality and poor prognosis. Treatment effect estimation is of high clinical significance to support the treatment decision-making for aneurysmal subarachnoid hemorrhage. However, most existing studies on treatment decision support of this disease are unable to simultaneously compare the potential outcomes of different treatments for a patient. Furthermore, these studies fail to harmoniously integrate the imaging data with non-imaging clinical data, both of which are significant in clinical scenarios. In this paper, the key challenges we address are: how to effectively estimate the treatment effect for aneurysmal subarachnoid hemorrhage; and how to utilize multi-modality data to perform this estimation. Specifically, we first propose a novel scheme that uses multi-modality confounders distillation architecture to predict the treatment outcome and treatment assignment simultaneously. Notably, with these distilled confounder features, we design an imaging and non-imaging interaction representation learning strategy to use the complementary information extracted from different modalities to balance the feature distribution of different treatment groups. We have conducted extensive experiments using a clinical dataset of 656 subarachnoid hemorrhage cases, which was collected from the Hospital Authority of Hong Kong. Our method shows consistent improvements on the evaluation metrics of treatment effect estimation, achieving state-of-the-art results over strong competitors. Code is released at https://github.com/med-air/TOP-aSAH.

The effect of carbon farming training on food security and development resilience in Northern Ghana

ABSTRACT Carbon farming has recently been advocated for as climate change and variability mitigation and/or adaptation strategy in global agriculture. In this study, we address an important research question of whether carbon farming training can improve household resilience capacity as well as food security by employing internationally standardized indicators. Household resilience capacity and its components are measured using the Food and Agriculture Organisation (FAO)’s resilience capacity index while the food security measures used include household and child food insecurity experience scale (FIES and CFIES), food consumption score (FCS) and household dietary diversity score (HDDS). We relied on doubly robust treatment effect estimators to account for potential selection bias and heterogeneity. We find that carbon farming training has no statistically significant effect on overall household resilience capacity. However, we find a large and statistically significant effect on key components of resilience (specifically, access to basic services, assets and social safety nets) and a marginal improvement in adaptive capacity. We also find statistically significant effect on FCS and HDDS but not for the other food security indicators (FIES and CFIES). Overall, the results suggest that agricultural training programs, particularly climate change adaptation capacity building initiatives, could improve important welfare measures in developing countries.

Evaluating the comparability of osteoporosis treatments using propensity score and negative control outcome methods in UK and Denmark electronic health record databases.

Evidence on the comparative effectiveness of osteoporosis treatments is heterogeneous. This may be attributed to different populations and clinical practice, but also to differing methodologies ensuring comparability of treatment groups before treatment effect estimation and the amount of residual confounding by indication. This study assessed the comparability of denosumab vs oral bisphosphonate (OBP) groups using propensity score (PS) methods and negative control outcome (NCO) analysis. A total of 280 288 women aged ≥50years initiating denosumab or OBP in 2011-2018 were included from the UK Clinical Practice Research Datalink (CPRD) and the Danish National Registries (DNR). Balance of observed covariates was assessed using absolute standardised mean difference (ASMD) before and after PS weighting, matching, and stratification, with ASMD >0.1 indicating imbalance. Residual confounding was assessed using NCOs with ≥100 events. Hazard ratio (HR) and 95% confidence interval (CI) between treatment and NCO was estimated using Cox models. Presence of residual confounding was evaluated with two approaches1: >5% of NCOs with 95% CI excluding 1,2 >5% of NCOs with an upper CI <0.75 or lower CI >1.3. The number of imbalanced covariates before adjustment (CPRD 22/87; DNR 18/83) decreased, with 2-11% imbalance remaining after weighting, matching or stratification. Using approach 1, residual confounding was present for all PS methods in both databases (≥8% of NCOs), except for stratification in DNR (3.8%). Using approach 2, residual confounding was present in CPRD with PS matching (5.3%) and stratification (6.4%), but not with weighting (4.3%). Within DNR, no NCOs had HR estimates with upper or lower CI limits beyond the specified bounds indicating residual confounding for any PS method. Achievement of covariate balance and determination of residual bias were dependent upon several factors including the population under study, PS method, prevalence of NCO, and the threshold indicating residual confounding.

Analyzing adverse effects of subway extension on housing prices in affluent urban neighborhoods

This study examines the effects of subway extension on housing prices in affluent urban neighborhoods, focusing on the Q-line extension in Manhattan, New York City. Utilizing synthetic controls and treatment effects estimators, distinct pricing trends across property types are revealed, particularly condominiums. The observed pricing dynamics deviate from the assumption that increasing supplies are associated with price discounts. Moreover, the connection between price discounts and noise complaints is not observed, highlighting the significant role of demand-side factors, especially neighborhood characteristics, in shaping housing prices. An unexpected demographic shift is observed, suggesting that the Q-line extension may disproportionately benefit non-white and low-income groups, challenging the prevailing gentrification narrative. We use the term “inverse premiumization” to denote the phenomenon where anticipated price increases in affluent neighborhoods due to transit improvements fail to materialize. Furthermore, our analysis of speculative behavior reveals a spike in short-term growth during the public notice period, gradually slowing down during construction and operation phases. These findings offer the nuanced adverse effects of subway extension on housing prices, contributing to our understanding of short-to medium-term price premiums and discounts. These insights are key considerations for city planners and policymakers navigating urban development, housing market, and speculative behavior dynamics.

Variance estimation for average treatment effects estimated by g-computation

AbstractThe average treatment effect is used to evaluate effects of interventions in a population. Under certain causal assumptions, such an effect may be estimated from observational data using the g-computation technique. The asymptotic properties of this estimator appears not to be well-known and hence bootstrapping has become the preferred method for estimating its variance. Bootstrapping is, however, not an optimal choice for multiple reasons; it is a slow procedure and, if based on too few bootstrap samples, results in a highly variable estimator of the variance. In this paper, we consider estimators of potential outcome means and average treatment effects using g-computation. We consider these parameters for the entire population but also in subgroups, for example, the average treatment effect among the treated. We derive their asymptotic distributions in a general framework. An estimator of the asymptotic variance is proposed and shown to be consistent when g-computation is used in conjunction with the M-estimation technique. The proposed estimator is shown to be superior to the bootstrap technique in a simulation study. Robustness against model misspecification is also demonstrated by means of simulations.