Marginal Treatment Effect Research Articles

Unveiling the causal effects of China’s minimum living standard guarantee on household transportation expenditures: A causal forest analysis

We investigate the causal effects of China’s Minimum Living Standard Guarantee (MLSG, also called Dibao) subsidy on household transportation expenditures. Utilizing data from the Chinese Social Survey, we employ a combination of the propensity score matching (PSM), causal forest (CF) and marginal treatment effect model (MTE) methodologies to rigorously estimate the subsidy’s effects. The PSM allows us to mitigate selection bias by matching MLSG recipients with comparable non-recipients. While the causal forest captures the heterogeneity of treatment effects across various household profiles. The result of MTE indicate that observable and essential heterogeneity are present to influence the effect of their subsidies, which present the consistent with PSM and CF. The causal mediation analysis indicates the mediating mechanism that MLSG impacts on household transportation expenditures, while also revealing significant variations among different regions. The study not only refines our understanding of the MLSG’s effects on household spending but also offers novel insights into applicational advancements by incorporating machine-learning techniques for policy evaluation. These results have important implications for policy formulation and refinement, particularly in the urban-rural differences.

How to achieve model-robust inference in stepped wedge trials with model-based methods?

A stepped wedge design is an unidirectional crossover design where clusters are randomized to distinct treatment sequences. While model-based analysis of stepped wedge designs is a standard practice to evaluate treatment effects accounting for clustering and adjusting for covariates, their properties under misspecification have not been systematically explored. In this article, we focus on model-based methods, including linear mixed models and generalized estimating equationswith an independence, simple exchangeable, or nested exchangeable working correlation structure. We study when a potentially misspecified working model can offer consistent estimation of the marginal treatment effect estimands, which are defined nonparametrically with potential outcomes and may be functions of calendar time and/or exposure time. We prove a central result that consistency for nonparametric estimands usually requires a correctly specified treatment effect structure, but generally not the remaining aspects of the working model (functional form of covariates, random effects, and error distribution), and valid inference is obtained via the sandwich variance estimator. Furthermore, an additional g-computation step is required to achieve model-robust inference under non-identity link functions or for ratio estimands. The theoretical results are illustrated via several simulation experiments and re-analysis of a completed stepped wedge cluster randomized trial.

WELFARE ANALYSIS VIA MARGINAL TREATMENT EFFECTS

We consider a causal structure with endogeneity, i.e., unobserved confoundedness, where an instrumental variable is available. In this setting, we show that the mean social welfare function can be identified and represented via the marginal treatment effect as the operator kernel. This representation result can be applied to a variety of statistical decision rules for treatment choice, including plug-in rules, Bayes rules, and empirical welfare maximization rules. Focusing on the application of the empirical welfare maximization framework, we provide convergence rates of the worst-case average welfare loss (regret).

Covariate adjustment in randomized controlled trials: General concepts and practical considerations.

There has been a growing interest in covariate adjustment in the analysis of randomized controlled trials in past years. For instance, the US Food and Drug Administration recently issued guidance that emphasizes the importance of distinguishing between conditional and marginal treatment effects. Although these effects may sometimes coincide in the context of linear models, this is not typically the case in other settings, and this distinction is often overlooked in clinical trial practice. Considering these developments, this article provides a review of when and how to use covariate adjustment to enhance precision in randomized controlled trials. We describe the differences between conditional and marginal estimands and stress the necessity of aligning statistical analysis methods with the chosen estimand. In addition, we highlight the potential misalignment of commonly used methods in estimating marginal treatment effects. We hereby advocate for the use of the standardization approach, as it can improve efficiency by leveraging the information contained in baseline covariates while remaining robust to model misspecification. Finally, we present practical considerations that have arisen in our respective consultations to further clarify the advantages and limitations of covariate adjustment.

Impact of apparent temperatures on park visitation behavior: A comprehensive analysis using large-scale mobility data

As many regions continue to show record temperatures and high frequencies of extreme climate events, climate change is disrupting human health and prosperity. This study focuses on how temperature changes and extreme heat as key climate components can impact park visitations (key physical activities indicator). Using the large mobility dataset SafeGraph, this study introduced a causal machine learning approach to estimate marginal heterogenous treatment effects (HTE) of daily apparent temperatures for different park properties and activity types. Our results revealed the declining effects of park visitations as apparent temperature increases and the significant disturbances under extreme heat. Different temperature thresholds that may harm park visitations were identified. Nearby green and public spaces (e.g. neighborhood parks) are the most susceptible category to summer heat. Dog parks, Playgrounds and Ball Field/Sports were well used during the heat. Pool/Splash pad and Community center have shown capabilities of mitigating heat stress. Effectiveness of other park facilities, including Restroom/Drinking Fountain, Water Body, Shade Area Facility were also presented. This study estimates climate impact on park behaviors for multiple usage and environmental situations. Our findings may help future planners, policymakers, landscape architects and park managers better understand climate-related risks when making investment and development decisions.

Advancing unanchored simulated treatment comparisons: A novel implementation and simulation study.

Population-adjusted indirect comparisons, developed in the 2010s, enable comparisons between two treatments in different studies by balancing patient characteristics in the case where individual patient-level data (IPD) are available for only one study. Health technology assessment (HTA) bodies increasingly rely on these methods to inform funding decisions, typically using unanchored indirect comparisons (i.e., without a common comparator), due to the need to evaluate comparative efficacy and safety for single-arm trials. Unanchored matching-adjusted indirect comparison (MAIC) and unanchored simulated treatment comparison (STC) are currently the only two approaches available for population-adjusted indirect comparisons based on single-arm trials. However, there is a notable underutilisation of unanchored STC in HTA, largely due to a lack of understanding of its implementation. We therefore develop a novel way to implement unanchored STC by incorporating standardisation/marginalisation and the NORmal To Anything (NORTA) algorithm for sampling covariates. This methodology aims to derive a suitable marginal treatment effect without aggregation bias for HTA evaluations. We use a non-parametric bootstrap and propose separately calculating the standard error for the IPD study and the comparator study to ensure the appropriate quantification of the uncertainty associated with the estimated treatment effect. The performance of our proposed unanchored STC approach is evaluated through a comprehensive simulation study focused on binary outcomes. Our findings demonstrate that the proposed approach is asymptotically unbiased. We argue that unanchored STC should be considered when conducting unanchored indirect comparisons with single-arm studies, presenting a robust approach for HTA decision-making.

The interpretation of 2SLS with a continuous instrument: A weighted LATE representation

This note introduces a novel weighted local average treatment effect representation for the two-stages least-squares (2SLS) estimand in the continuous instrument with binary treatment case. Under standard conditions, we obtain weights that are nonnegative, integrate to unity, and assign larger values to instrument support points that deviate from their average. Our representation does not require instruments to be discretized nor relies on limiting arguments, such as those used in the definition of the marginal treatment effect (MTE). The pattern of the weights also has a clear interpretation. We believe these features of the representation to be useful for applied researchers when communicating their results. As a direct byproduct of our approach, we also obtain a representation of the 2SLS estimand as a weighted average of treatment effects among “marginal compliance” groups, without having to resort to the threshold-crossing representation underlying the MTE construction. The latter representation has an intuitive interpretation as well.

Policy Tourism and Economic Collaboration Among Local Governments: A Nonparametric Matching Model

Policy tourism is an important, widespread, but understudied aspect of the networking efforts of public managers. Few studies have investigated how the official and purposeful interactions between public managers in the context of policy tourism affect collaboration. To fill this gap, this article aims to explore the question: To what extent does policy tourism channel economic collaboration? We argue that leaders’ visit paths provide a direct channel for collaboration through information exchange, trust building, and integration of regional policies, but the consequences vary between jurisdictions at different levels of economic development. We adopt a newly developed nonparametric matching method to estimate the marginal average treatment effects of policy tourism on interprovincial trade flows in China from 2000 to 2013. Results demonstrate that policy tourism channels interprovincial trade flow significantly and that the impacts decrease over time. Also, policy tourism affects interprovincial trade flow more significantly when initiated by less-developed jurisdictions.

Heterogeneity, selection, and the policy effect of educational expansion on college graduate earnings in China, 1981–2015

This article examines the impact of social origin on the returns to college education in the context of China’s higher education expansion since 1999. Utilizing a double-treatment setting, a marginal treatment effect framework is adopted to estimate the causal effect of college education on earnings, while a difference-in-differences methodology identifies the policy effect of educational expansion on the college premium. Analysis of data from a series of nationally representative Chinese surveys reveals that the “true” college earnings premium is rather small for the post-expansion cohort, and that much of the observed earnings gap between college and high school graduates after the expansion is due to returns to unobserved abilities. Further analysis shows that the college earnings premium after the expansion declines more for rural-origin children with schooling probabilities in the top percentiles than for their urban counterparts.

Model-based standardization using multiple imputation

BackgroundWhen studying the association between treatment and a clinical outcome, a parametric multivariable model of the conditional outcome expectation is often used to adjust for covariates. The treatment coefficient of the outcome model targets a conditional treatment effect. Model-based standardization is typically applied to average the model predictions over the target covariate distribution, and generate a covariate-adjusted estimate of the marginal treatment effect.MethodsThe standard approach to model-based standardization involves maximum-likelihood estimation and use of the non-parametric bootstrap. We introduce a novel, general-purpose, model-based standardization method based on multiple imputation that is easily applicable when the outcome model is a generalized linear model. We term our proposed approach multiple imputation marginalization (MIM). MIM consists of two main stages: the generation of synthetic datasets and their analysis. MIM accommodates a Bayesian statistical framework, which naturally allows for the principled propagation of uncertainty, integrates the analysis into a probabilistic framework, and allows for the incorporation of prior evidence.ResultsWe conduct a simulation study to benchmark the finite-sample performance of MIM in conjunction with a parametric outcome model. The simulations provide proof-of-principle in scenarios with binary outcomes, continuous-valued covariates, a logistic outcome model and the marginal log odds ratio as the target effect measure. When parametric modeling assumptions hold, MIM yields unbiased estimation in the target covariate distribution, valid coverage rates, and similar precision and efficiency than the standard approach to model-based standardization.ConclusionWe demonstrate that multiple imputation can be used to marginalize over a target covariate distribution, providing appropriate inference with a correctly specified parametric outcome model and offering statistical performance comparable to that of the standard approach to model-based standardization.

A computational approach to identification of treatment effects for policy evaluation

For counterfactual policy evaluation, it is important to ensure that treatment parameters are relevant to policies in question. This is especially challenging under unobserved heterogeneity, as is well featured in the definition of the local average treatment effect (LATE). Being intrinsically local, the LATE is known to lack external validity in counterfactual environments. This paper investigates the possibility of extrapolating local treatment effects to different counterfactual settings when instrumental variables can be only binary. We propose a novel framework to systematically calculate sharp nonparametric bounds on various policy-relevant treatment parameters that are defined as weighted averages of the marginal treatment effect (MTE). Our framework is flexible enough to fully incorporate statistical independence (rather than mean independence) of instruments and a large menu of identifying assumptions beyond the shape restrictions on the MTE that have been considered in prior studies. We apply our method to understand the effects of medical insurance policies on the use of medical services.

Green technology innovation and regional carbon emissions: analysis based on heterogeneous treatment effect modeling.

The impact of green technology innovation on regional carbon emissions has been a contentious issue in academic research. In this study, we attempt to analyze the influence of green technology innovation on regional carbon emissions using panel data from 28 Chinese provinces for the period of 2007-2020. Utilizing a heterogeneous treatment effect model, we systematically examine the effects of green technology innovation on regional carbon emissions. Firstly, we conduct a feature selection analysis on the factors influencing regional carbon emissions using causal inference methods based on machine learning. Subsequently, we explore the conditional and marginal treatment effects of green technology innovation on regional carbon emissions using the heterogeneous treatment effect model. Finally, we investigate the dynamic effects of green technology innovation on regional carbon emissions across different periods. Empirical results indicate that firstly, green technology innovation indirectly reduces regional carbon emissions by promoting energy efficiency improvement; secondly, the impact of green technology innovation on carbon emissions exhibits significant regional heterogeneity, with the largest effect observed in the eastern region, followed by the western region and the smallest effect in the central region; thirdly, at a significance level of 5%, green technology innovation has a direct inhibitory effect on carbon emissions in certain regions.

The Marginal Returns to Distance Education: Evidence from Mexico’s Telesecundarias

This paper analyzes a large-scale and long-running distance education program in Mexico. We estimate marginal treatment effects (MTEs) for learning in math and Spanish in telesecundarias relative to traditional Mexican secondary schools using an empirical framework that allows for unobserved sorting on gains. The estimated MTEs reveal that school choice is not random and that the average student experiences significant improvements in both math and Spanish after just one year of attendance in telesecundarias. We find that the existing policy reduces educational inequality, and our policy-relevant treatment effects show that expanding telesecundarias would yield significant improvements in academic performance. (JEL I21, I24, I26, I28, O15)

Don't bet the Farm on Crop Insurance Subsidies: A Marginal Treatment Effect Analysis of French Farms 

Don't bet the Farm on Crop Insurance Subsidies: A Marginal Treatment Effect Analysis of French Farms 

Identification of Marginal Treatment Effects Using Subjective Expectations

Identification of Marginal Treatment Effects Using Subjective Expectations

Perils of Randomized Controlled Trial Survival Extrapolation Assuming Treatment Effect Waning: Why the Distinction Between Marginal and Conditional Estimates Matters

ObjectivesA long-term, constant, protective treatment effect is a strong assumption when extrapolating survival beyond clinical trial follow-up; hence, sensitivity to treatment effect waning is commonly assessed for economic evaluations. Forcing a hazard ratio (HR) to 1 does not necessarily estimate loss of individual-level treatment effect accurately because of HR selection bias. A simulation study was designed to explore the behavior of marginal HRs under a waning conditional (individual-level) treatment effect and demonstrate bias in forcing a marginal HR to 1 when the estimand is “survival difference with individual-level waning”. MethodsData were simulated under 4 parameter combinations (varying prognostic strength of heterogeneity and treatment effect). Time-varying marginal HRs were estimated in scenarios where the true conditional HR attenuated to 1. Restricted mean survival time differences, estimated having constrained the marginal HR to 1, were compared with true values to assess bias induced by marginal constraints. ResultsUnder loss of conditional treatment effect, the marginal HR took a value >1 because of covariate imbalances. Constraining this value to 1 lead to restricted mean survival time difference bias of up to 0.8 years (57% increase). Inflation of effect size estimates also increased with the magnitude of initial protective treatment effect. ConclusionsImportant differences exist between survival extrapolations assuming marginal versus conditional treatment effect waning. When a marginal HR is constrained to 1 to assess efficacy under individual-level treatment effect waning, the survival benefits associated with the new treatment will be overestimated, and incremental cost-effectiveness ratios will be underestimated.

Marginal versus conditional rate estimation for count and recurrent event data with an estimand framework

Count and recurrent event endpoints are common key efficacy endpoints in clinical research. For example, in clinical research of pulmonary diseases such as chronic obstructive pulmonary disease (COPD) or asthma, the reduction of the occurrence of a recurrent event, pulmonary exacerbation (PEx) caused by acute respiratory symptoms, is often used to measure the treatment effect. The occurrence of PEx is often analyzed with nonlinear models, such as Poisson regression or Negative Binomial regression. It is observed that model-estimated within-group PEx rates are often lower than the descriptive statistics of within-group PEx rates. Motivated by this observation, we explore their relationship mathematically and demonstrate that it is due to the difference between conditional PEx rates and population-level PEx rates (marginal rates). Our findings corroborate the recent FDA guidance (2023) [1], which discusses considerations for covariate adjustment in nonlinear models, and that conditional or subgroup treatment effects with covariate adjustment may differ from marginal treatment effects. In this article, we demonstrate how covariate adjustment impacts the estimation of event rates and rate ratios with both closed form and simulation studies. Additionally, following the ICH E9 addendum on the estimand framework [2], we discuss the estimand framework for count and recurrent event data.

Partial Identification of Marginal Treatment Effects with Discrete Instruments and Misreported Treatment*

This paper provides partial identification results for the marginal treatment effect (MTE) when the binary treatment variable is potentially misreported and the instrumental variable is discrete. Identification results are derived under smoothness assumptions. Bounds for both the case of misreported treatment and the case of no misreported treatment are derived. The identification results are illustrated by identifying the marginal treatment effects of food stamps on health.

Firm Productivity and Matching Frictions in th Labor Markets: Is This an Unending Curse to Employers?

The productivity behavior of firms is significantly influenced by labor market frictions in both emerging and established economies. Kenya persistently advocates for enhanced strategies to bolster productivity. The precise impact of labor market friction remains ambiguous. This paper focuses on assessing the influence of qualification and skills mismatch on firm productivity within the context of Kenya. The study utilized secondary crosssectional data obtained from the World Bank database, specifically from the 2016-2017 Skills Toward Employment Productivity (STEP) Household Survey conducted in Kenya. The full specification of the maximum likelihood model under the endogenous switching regression (ESR) was estimated. The results of the study indicate that insufficient education and a mismatch between skills and job requirements have a significantly negative impact on firm productivity. The impact of excessive education on firm production was found to be minimal. The essential finding regarding the marginal treatment effect, which holds significant implications for policymaking, indicates a strong positive association between over-education and firm productivity. A negative association is observed between education and skills mismatch and firm productivity. The policy implications underscore the necessity of aligning graduates with employment opportunities that correspond to their educational background and level of expertise.

Sharp bounds in the latent index selection model

A fundamental question underlying the literature on partial identification is: what can we learn about parameters that are relevant for policy but not necessarily point-identified by the exogenous variation we observe? This paper provides an answer in terms of sharp, analytic characterizations and bounds for an important class of policy-relevant treatment effects, consisting of marginal treatment effects and linear functionals thereof, in the latent index selection model as formalized in Vytlacil (2002). The sharp bounds use the full content of identified marginal distributions, and analytic derivations rely on the theory of stochastic orders. The proposed methods also make it possible to sharply incorporate new auxiliary assumptions on distributions into the latent index selection framework. Empirically, I apply the methods to study the effects of Medicaid on emergency room utilization in the Oregon Health Insurance Experiment, showing that the predictions from extrapolations based on a distribution assumption (rank similarity) differ substantively and consistently from existing extrapolations based on a parametric mean assumption (linearity). This underscores the value of utilizing the model’s full empirical content in combination with auxiliary assumptions.