Marginal Treatment Effects in the Absence of Instrumental Variables

Heterogeneous Treatment Effects in the Presence of Self-Selection: A Propensity Score Perspective

Chapter 71 Econometric Evaluation of Social Programs, Part II: Using the Marginal Treatment Effect to Organize Alternative Econometric Estimators to Evaluate Social Programs, and to Forecast their Effects in New Environments

An essential feature common to all empirical social research is variability across units of analysis. Individuals differ not only in background characteristics, but also in how they respond to a particular treatment, intervention, or stimulation. Moreover, individuals may self-select into treatment on the basis of their anticipated treatment effects. To study heterogeneous treatment effects in the presence of self-selection, Heckman and Vytlacil (1999, 2001a, 2005, 2007b) have developed a structural approach that builds on the marginal treatment effect (MTE). In this paper, we extend the MTE-based approach through a redefinition of MTE. Specifically, we redefine MTE as the expected treatment effect conditional on the propensity score (rather than all observed covariates) as well as a latent variable representing unobserved resistance to treatment. As with the original MTE, the new MTE can also be used as a building block for evaluating standard causal estimands. However, the weights associated with the new MTE are simpler, more intuitive, and easier to compute. Moreover, the new MTE is a bivariate function, and thus is easier to visualize than the original MTE. Finally, the redefined MTE immediately reveals treatment effect heterogeneity among individuals who are at the margin of treatment. As a result, it can be used to evaluate a wide range of policy changes with little analytical twist, and to design policy interventions that optimize the marginal benefits of treatment. We illustrate the proposed method by estimating heterogeneous economic returns to college with National Longitudinal Study of Youth 1979 (NLSY79) data.

Matching-adjusted indirect comparisons (MAIC) and simulated treatment comparisons (STC) are commonly used for indirect treatment comparisons when patient-level data are available for some treatments but not others. However, MAIC can become inefficient when covariate overlap across trials is limited, and STC may fail to estimate marginal (population average) treatment effects required for health technology assessment when non-linear outcome models, such as logistic or survival models, are used. We propose a new STC approach to address these limitations. For settings where a linear model for the treatment effect can be assumed (e.g. log odds ratios or log hazard ratios), we propose to apply STC with jackknife pseudo values of the marginal treatment effect estimate as the dependent variable, rather than the observed outcomes. These pseudo values decompose the marginal treatment effect estimate into individual patient contributions, which can then be regressed on patient-level characteristics to predict marginal treatment effects in alternative target populations. We illustrate the approach using two worked examples involving non-linear outcome models. In simulated trials of continuous, binary, and time-to-event outcomes, the proposed approach showed less bias than traditional STC for non-linear outcome models, and had lower mean squared error than MAIC, both under correct and incorrect specification of baseline covariate functional forms.

Estimating Marginal Treatment Effects under Unobserved Group Heterogeneity

Covariate adjustment in RCTs results in increased power to detect conditional effects compared with the power to detect unadjusted or marginal effects

Heterogeneity is an important issue in the study of causal reasoning.It is shown by whether or not individuals receive treatment and how they respond to treatment differently. In contrast to the conventional treatment effect, which ignores the effect of heterogeneity, a marginal treatment effect (MTE) was introduced to represent the marginal benefit of treatment, which is heterogeneously dependent on observed and unobserved factors. However, traditional methods prefer the curse of dimensionality in calculating MTE, leading to bias in empirical studies. In view of this, this paper proposes a nonparametric framework based on machine learning algorithms and theoretically validates the consistency of the approach. In our framework, we first consistently generate propensity scores from random forests, and then apply the propensity scores to the classical identification of MTEs. The innovative nonparametric MTE approach in this paper shows reliable consistency in the estimation of high-dimensional causal inferences and allows for a more efficient assessment of policy interventions.

This paper uses official Italian micro data and different methods to estimate, in the framework of potential outcomes, the marginal return to college education allowing for heterogeneous returns and for self-selection into higher education. Specifically, the paper is focused on the estimation of heterogeneity of average treatment effect (ATE) on a cohort of college and high school graduates using the 2008 survey on household, income and wealth of the Bank of Italy. Methodologically, this study was carried out by using both propensity-score-based (PS-based) methods and a new approach based on marginal treatment effects (MTE), recently proposed by Heckman and his associates as a useful strategy when the ignorability assumption may be violated. In the PS-based approach, heterogeneous treatment effects are estimated in three different manners: the traditional stratification approach (propensity score strata), the regression adjustment within propensity score strata and, finally, a non-parametric smoothing approach. In the MTE approach, the treatment effect heterogeneity across individuals is estimated in a parametric as well as a semi-parametric strategy. Our empirical analysis shows that the estimated heterogeneity is substantial: following MTE based results (quite representative of other methods) the return to college graduation for a randomly selected individual varies from as high as 20 % (for persons who would add one fifth of wage from graduating college) to as low as −22 % (for persons who would lose from college graduation), suggesting that returns are higher for individuals more likely to attend college. Furthermore, the results of different methods show very low (point) estimates of ATE: average college returns vary from 3.5 % by the PS-smoothing method to 1.8 % by the parametric MTE method, which also leads a greater treatment effect on treated (5.5 %), a moderate, but significant sorting gain and a negligible selection bias.

Introdução: Em Portugal denota-se a existência de poucos instrumentos devidamente validados para avaliar o desenvolvimento motor da criança, no âmbito da fisioterapia pediátrica. Tendo em conta a necessidade de basear a prática clinica do fisioterapeuta cada vez mais na evidência científica, revela-se de extrema importância o desenvolvimento de instrumentos de avaliação válidos e fiáveis, que permitam ao fisioterapeuta uma avaliação objetiva e padronizada dos seus resultados. O presente estudo pretende dar um contributo para a validação da subescala de Motricidade Global da Peabody Developmental Motor Scale – 2 (PDMS-2), na sua versão portuguesa, e analisar sua validade e eficácia quando aplicados em crianças com e sem atraso do desenvolvimento motor, com idade dos 0 aos 71 meses. Pretende-se assim verificar algumas das propriedades psicométricas da versão portuguesa da escala (apenas na componente de Motricidade Global), nomeadamente a coerência interna, da sua fiabilidade teste-reteste e a sensibilidade face às diferentes faixas etárias abrangidas e a crianças com atraso no desenvolvimento motor. Método: Após obter as devidas autorizações aplicou-se a Subescala de Motricidade Global da PDMS-2 e um questionário de caracterização da condição da amostra de 68 crianças residentes na localidade de Moura. Foi realizado o teste-reteste, e verificada a consistência interna de cada dimensão da escala na componente QMG de forma a serem analisados estatisticamente. Resultados: Verificaram-se níveis elevados de coerência interna em todas as dimensões com o alpha de Cronbach a variar entre os 0,87 para a subescala dos reflexos, 0,93 para a subescala da Postura, 0,96 para as Habilidades Manipulativas e 0,99 para a subescala de Locomoção, o que indica a existência de uma elevada consistência interna. A fiabilidade teste-resteste também se revelou elevada com valores de CCI acima dos 0,88. Ao nível da validade de constructo verificaram-se as diferenças entre as pontuações médias das crianças com e sem atraso de desenvolvimento motor não foram estatisticamente significativas. Conclusão: A Subescala de Motricidade Global da versão portuguesa da PSMS-2 mostrou possuir boas propriedades psicométricas, quer a nível de coerência interna, que ao nível da fiabilidade teste-reteste. Não mostrou conseguir discriminar entre crianças com atraso de desenvolvimento motor. Considerou-se uma escala abrangente, útil e clara mas a sua aplicação mostrou-se longa.

Monte Carlo simulation methods are increasingly being used to evaluate the property of statistical estimators in a variety of settings. The utility of these methods depends upon the existence of an appropriate data-generating process. Observational studies are increasingly being used to estimate the effects of exposures and interventions on outcomes. Conventional regression models allow for the estimation of conditional or adjusted estimates of treatment effects. There is an increasing interest in statistical methods for estimating marginal or average treatment effects. However, in many settings, conditional treatment effects can differ from marginal treatment effects. Therefore, existing data-generating processes for conditional treatment effects are of little use in assessing the performance of methods for estimating marginal treatment effects. In the current study, we describe and evaluate the performance of two different data-generation processes for generating data with a specified marginal odds ratio. The first process is based upon computing Taylor Series expansions of the probabilities of success for treated and untreated subjects. The expansions are then integrated over the distribution of the random variables to determine the marginal probabilities of success for treated and untreated subjects. The second process is based upon an iterative process of evaluating marginal odds ratios using Monte Carlo integration. The second method was found to be computationally simpler and to have superior performance compared to the first method.

We offer a propensity score perspective to interpret and analyze the marginal treatment effect (MTE). Specifically, we redefine MTE as the expected treatment effect conditional on the propensity score and a latent variable representing unobserved resistance to treatment. As with the original MTE, the redefined MTE can be used as a building block for constructing standard causal estimands. The weights associated with the new MTE, however, are simpler, more intuitive, and easier to compute. Moreover, the redefined MTE immediately reveals treatment effect heterogeneity among individuals at the margin of treatment, enabling us to evaluate a wide range of policy effects.

This paper provides partial identification results for the marginal treatment effect (MTE) when the binary treatment variable is potentially misreported and the instrumental variable is discrete. Identification results are derived under smoothness assumptions. Bounds for both the case of misreported treatment and the case of no misreported treatment are derived. The identification results are illustrated by identifying the marginal treatment effects of food stamps on health.

I partially identify the marginal treatment effect (MTE) when the treatment is misclassified. I explore two restrictions, allowing for dependence between the instrument and the misclassification decision. If the signs of the propensity scores’ derivatives are equal, I identify the MTE sign. If those derivatives are similar, I bound the MTE. To illustrate, I analyze the impact of alternative sentences (fines and community service versus no punishment) on recidivism in Brazil, where court appeals processes generate misclassification. The estimated misclassification bias may be as large as 10% of the largest possible MTE, and the bounds contain the correctly estimated MTE.

Since the seminal introduction of the propensity score by Rosenbaum and Rubin, propensity-score-based (PS-based) methods have been widely used for drawing causal inferences in the behavioral and social sciences. However, the propensity score approach depends on the ignorability assumption: there are no unobserved confounders once observed covariates are taken into account. For situations where this assumption may be violated, Heckman and his associates have recently developed a novel approach based on marginal treatment effects (MTE). In this paper, we (1) explicate consequences for PS-based methods when aspects of the ignorability assumption are violated; (2) compare PS-based methods and MTE-based methods by making a close examination of their identification assumptions and estimation performances; (3) apply these two approaches in estimating the economic return to college using data from NLSY 1979 and discuss their discrepancies in results. When there is a sorting gain but no systematic baseline difference between treated and untreated units given observed covariates, PS-based methods can identify the treatment effect of the treated (TT). The MTE approach performs best when there is a valid and strong instrumental variable (IV). In addition, this paper introduces the "smoothing-difference PS-based method," which enables us to uncover heterogeneity across people of different propensity scores in both counterfactual outcomes and treatment effects.

A number of statistical approaches have been proposed for incorporating supplemental information in randomized clinical trials. Existing methods often compare the marginal treatment effects to evaluate the degree of consistency between sources. Dissimilar marginal treatment effects would either lead to increased bias or down-weighting of the supplemental data. This represents a limitation in the presence of treatment effect heterogeneity, in which case the marginal treatment effect may differ between the sources solely due to differences between the study populations. We introduce the concept of covariate-adjusted exchangeability, in which differences in the marginal treatment effect can be explained by differences in the distributions of the effect modifiers. The potential outcomes framework is used to conceptualize covariate-adjusted and marginal exchangeability. We utilize a linear model and the existing multisource exchangeability models framework to facilitate borrowing when marginal treatment effects are dissimilar but covariate-adjusted exchangeability holds. We investigate the operating characteristics of our method using simulations. We also illustrate our method using data from two clinical trials of very low nicotine content cigarettes. Our method has the ability to incorporate supplemental information in a wider variety of situations than when only marginal exchangeability is considered.