Distribution Of Potential Outcomes Research Articles

Sharp Bounds on Functionals of the Joint Distribution in the Analysis of Treatment Effects

This article proposes an identification and estimation method that allows researchers to bound continuous functionals of the joint distribution of potential outcomes from the literature on treatment effects. The focus is on a model where no restrictions are imposed on treatment selection. The method can sharply bound interesting parameters when analytical bounds are difficult to derive, can be used in settings in which instruments are available, and can easily accommodate additional model constraints. However, computational considerations for the method are found to be important and are discussed in detail. Supplementary materials for this article are available online.

Setting a baseline for Integrated Landscape Design: Cost and risk assessment in herbaceous feedstock supply chains

The literature on bioenergy has pointed out that financial risks and limited performance benchmarks are leading barriers to wide-spread industry expansion. The purpose of this paper is to document a baseline cost and risk assessment from the perspective of biorefinery management for two herbaceous feedstocks, switchgrass and corn stover, in an application of Integrated Landscape Design (ILD). Integrating energy crops into row crops has attracted considerable attention leading to a variety of ways to implement ILD. However, variation in production conditions, different methods of implementing unit operations, and technology performance variability are among many factors leading to financial risk facing biorefineries from feedstock supply based on ILD. Utilizing data from field-based trials on switchgrass and corn stover in Virginia and Iowa in the United States, we estimate feedstock logistics cost and characterize it by a distribution of potential outcomes. The results show that, across fields included in the study, logistics cost for switchgrass vary from $73 per Mg to $171 per Mg. For corn stover, the inter-field variation in cost ranges from $83 per Mg to $105 per Mg. Results for intra-field uncertainty show that the cost for switchgrass ranges from $58 per Mg to $74 per Mg, and for corn stover they range from $50 per Mg to $65 per Mg. These results inform a baseline from which improvements in ILD can be compared, and provide biorefinery management with a starting point for cost and risk estimates of feedstock from ILD.

Partial identification of the treatment effect distribution and its functionals

In the treatment effect problem, the available information is on the marginal distributions of potential outcomes, but not on their joint distribution. The only point identified functional of the treatment effect distribution is its average, the average treatment effect (ATE). Quantiles and other functionals of the distribution of treatment effect are only partially identified. Bounds on a single quantile and on the cumulative distribution function (c.d.f.) in a single point are sharp (Makarov bounds). We show that bounds on functionals of the quantile process that use Makarov bounds are not sharp, because the Makarov bounds are pointwise, but not uniformly sharp. This allows us to propose improved bounds on functionals of the c.d.f. As an intermediate result, we find that the Makarov bounds on the region that contains the c.d.f. of the treatment effect distribution in a finite number of points can be improved. We provide numerical illustrations throughout the paper permitting a clear visualization of how the method works.

Financial Market Assumptions & Models for Pension Plans: A Technical Comment on the PIMS Model Assumptions for Asset Markets

The financial market assumptions of the PBGC’s PIMS model are critical inputs to simulations for most apparent uses of the system. They currently appear to be based on a reduced form, classical approach to assessing and forecasting the distribution of returns on various classes of input assets, allowing for a fairly sophisticated and useful approach to understanding simulated distributions of potential pension insurance outcomes as well as the net financial status of the PBGC. This technical note discusses some of the capital market side assumptions utilized in the model. It also comments on important related assumptions including the assumed asset allocations of insured plans, making suggestion for possible modification of input assumptions of the model to reflect time variation in financial market return behavior as well as time variation in observed plan allocations.

Flexible Sensitivity Analysis for Observational Studies Without Observable Implications

A fundamental challenge in observational causal inference is that assumptions about unconfoundedness are not testable from data. Assessing sensitivity to such assumptions is therefore important in practice. Unfortunately, some existing sensitivity analysis approaches inadvertently impose restrictions that are at odds with modern causal inference methods, which emphasize flexible models for observed data. To address this issue, we propose a framework that allows (1) flexible models for the observed data and (2) clean separation of the identified and unidentified parts of the sensitivity model. Our framework extends an approach from the missing data literature, known as Tukey’s factorization, to the causal inference setting. Under this factorization, we can represent the distributions of unobserved potential outcomes in terms of unidentified selection functions that posit a relationship between treatment assignment and unobserved potential outcomes. The sensitivity parameters in this framework are easily interpreted, and we provide heuristics for calibrating these parameters against observable quantities. We demonstrate the flexibility of this approach in two examples, where we estimate both average treatment effects and quantile treatment effects using Bayesian nonparametric models for the observed data.

The isotonic regression approach for an instrumental variable estimation of the potential outcome distributions for compliers

This paper discusses an instrumental variable estimation of the potential outcome distributions for compliers. The existing nonparametric estimators have a limitation in that they give non-proper cumulative distribution functions that violate the non-decreasing property. Using the least squares representation of the standard nonparametric estimators, a simple isotonic regression approach has been developed. A nonparametric bootstrap method is proposed as an appropriate method used to estimate the variances of the isotonic regression estimators. A simulation study demonstrates that the isotonic regression estimators provide more proper and efficient cumulative distribution functions, with much smaller standard errors than those of the standard nonparametric estimators when the proportion of compliers is small. The methods are illustrated with a study to estimate the distributional causal effect of a veteran status on future earnings.

Different temperature thresholds for antipyretic intervention in critically ill children with fever due to infection: the FEVER feasibility RCT.

Fever accelerates host immune system control of pathogens but at a high metabolic cost. The optimal approach to fever management and the optimal temperature thresholds used for treatment in critically ill children are unknown. To determine the feasibility of conducting a definitive randomised controlled trial (RCT) to evaluate the clinical effectiveness and cost-effectiveness of different temperature thresholds for antipyretic management. A mixed-methods feasibility study comprising three linked studies - (1) a qualitative study exploring parent and clinician views, (2) an observational study of the epidemiology of fever in children with infection in paediatric intensive care units (PICUs) and (3) a pilot RCT with an integrated-perspectives study. Participants were recruited from (1) four hospitals in England via social media (for the FEVER qualitative study), (2) 22 PICUs in the UK (for the FEVER observational study) and (3) four PICUs in England (for the FEVER pilot RCT). (1) Parents of children with relevant experience were recruited to the FEVER qualitative study, (2) patients who were unplanned admissions to PICUs were recruited to the FEVER observational study and (3) children admitted with infection requiring mechanical ventilation were recruited to the FEVER pilot RCT. Parents of children and clinicians involved in the pilot RCT. The FEVER qualitative study and the FEVER observational study had no interventions. In the FEVER pilot RCT, children were randomly allocated (1 : 1) using research without prior consent (RWPC) to permissive (39.5 °C) or restrictive (37.5 °C) temperature thresholds for antipyretics during their PICU stay while mechanically ventilated. (1) The acceptability of FEVER, RWPC and potential outcomes (in the FEVER qualitative study), (2) the size of the potentially eligible population and the temperature thresholds used (in the FEVER observational study) and (3) recruitment and retention rates, protocol adherence and separation between groups and distribution of potential outcomes (in the FEVER pilot RCT). In the FEVER qualitative study, 25 parents were interviewed and 56 clinicians took part in focus groups. Both the parents and the clinicians found the study acceptable. Clinicians raised concerns regarding temperature thresholds and not using paracetamol for pain/discomfort. In the FEVER observational study, 1853 children with unplanned admissions and infection were admitted to 22 PICUs between March and August 2017. The recruitment rate was 10.9 per site per month. The majority of critically ill children with a maximum temperature of > 37.5 °C received antipyretics. In the FEVER pilot RCT, 100 eligible patients were randomised between September and December 2017 at a recruitment rate of 11.1 per site per month. Consent was provided for 49 out of 51 participants in the restrictive temperature group, but only for 38 out of 49 participants in the permissive temperature group. A separation of 0.5 °C (95% confidence interval 0.2 °C to 0.8 °C) between groups was achieved. A high completeness of outcome measures was achieved. Sixty parents of 57 children took part in interviews and/or completed questionnaires and 98 clinicians took part in focus groups or completed a survey. Parents and clinicians found the pilot RCT and RWPC acceptable. Concerns about children being in pain/discomfort were cited as reasons for withdrawal and non-consent by parents and non-adherence to the protocol by clinicians. Different recruitment periods for observational and pilot studies may not fully reflect the population that is eligible for a definitive RCT. The results identified barriers to delivering the definitive FEVER RCT, including acceptability of the permissive temperature threshold. The findings also provided insight into how these barriers may be overcome, such as by limiting the patient inclusion criteria to invasive ventilation only and by improved site training. A definitive FEVER RCT using a modified protocol should be conducted, but further work is required to agree important outcome measures for clinical trials among critically ill children. The FEVER observational study is registered as NCT03028818 and the FEVER pilot RCT is registered as Current Controlled Trials ISRCTN16022198. This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 23, No. 5. See the NIHR Journals Library website for further project information.

Partial Mean Processes with Generated Regressors: Continuous Treatment Effects and Nonseparable Models

Partial mean with generated regressors arises in several econometric problems, such as the distribution of potential outcomes with continuous treatments and the quantile structural function in a nonseparable triangular model. This paper proposes a nonparametric estimator for the partial mean process, where the second step consists of a kernel regression on regressors that are estimated in the first step. The main contribution is a uniform expansion that characterizes in detail how the estimation error associated with the generated regressor affects the limiting distribution of the marginal integration estimator. The general results are illustrated with two examples: the generalized propensity score for a continuous treatment (Hirano and Imbens, 2004) and control variables in triangular models (Newey, Powell, and Vella, 1999; Imbens and Newey, 2009). An empirical application to the Job Corps program evaluation demonstrates the usefulness of the method.

Political economy of planned relocation: A model of action and inaction in government responses

Abstract Planned relocation has been shown to have significant impacts on the livelihoods and wellbeing of people and communities, whether the resettlement process is inclusive or coercive. For states, planned relocation represents risks to those communities but also to government investments and political legitimacy. Evaluations of relocations commonly focus on the risks and benefits of government interventions while overlooking the consequences of not intervening. Here we develop a conceptual framework to examine the factors that influence government decision-making about whether or not to undertake planned relocation of populations in the context of environmental change. The study examines planned relocation decisions and non-decisions by government agencies in West Bengal in India for communities seeking relocation due to coastal flooding. It focuses on three localities facing river erosion losing significant land areas in small islands and communities where populations recognize the need for public intervention, but where there has been a diversity of responses from the state authorities. Data are derived from interviews with key respondents involved in planning and implementing relocation and with residents affected by those government decisions (n = 26). These data show that government action is explained by a combination of risk aversion within political systems to avoid perceived negative consequences, and a lack of government accountability. The empirical cases demonstrate the uneven application of action and inaction and the consequent uneven distribution of potential outcomes on populations. The study suggests that while there may be a growing demand for planned relocation in places affected by environmental change, its implementation is likely to be uneven, with profound socioeconomic implications for those living in such localities.

A Bayesian nonparametric approach to causal inference on quantiles.

We propose a Bayesian nonparametric approach (BNP) for causal inference on quantiles in the presence of many confounders. In particular, we define relevant causal quantities and specify BNP models to avoid bias from restrictive parametric assumptions. We first use Bayesian additive regression trees (BART) to model the propensity score and then construct the distribution of potential outcomes given the propensity score using a Dirichlet process mixture (DPM) of normals model. We thoroughly evaluate the operating characteristics of our approach and compare it to Bayesian and frequentist competitors. We use our approach to answer an important clinical question involving acute kidney injury using electronic health records.

Regression discontinuity with categorical outcomes

We consider the regression discontinuity (RD) design with categorical outcomes, and exploit the possibility of adapting well-developed microeconometric models to the RD setting. The channels through which the forcing variable affects the potential outcome distributions are constrained to be minimal, to preserve the nonparametric feature of the RD design. Focusing on general categorical outcomes (nominal or ordinal), we develop a new RD estimator based on a nonparametric extension of the well-known multinomial logit model. The key issues of selecting the optimal bandwidth and constructing confidence regions robust to bias correction, of which the solutions only exist so far for the local linear estimator and a single treatment effect, are addressed through the general approach of local likelihood. The proposed estimator and associated inference are easy to implement, and the codes in MATLAB and an R package are available as a supplement to the paper. They are demonstrated by two empirical applications and simulation experiments.

Testing for Rank Invariance or Similarity in Program Evaluation

This paper discusses testable implications of rank invariance or rank similarity, assumptions that are common in program evaluation and quantile treatment eect (QTE) models. We nonparametrically identify and test the counterfactual distribution of individual potential ranks, or features of the distribution. The tests allow treatment to be endogenous, while essentially not requiring any additional assumptions other than those used to identify and estimate QTEs. We focus on testing ranks in the unconditional distribution of potential outcomes, and briey discuss testing for invariance or similarity of conditional ranks. The proposed tests are applied to the JTPA training program and Project STAR. For the former, we investigate whether job training causes individuals to systematically change their ranks in the earnings distribution; for the latter, we analyze the impacts of small classes and teacher aides on the gender performance gap in early childhood education. Also investigated is how these educational treatments interact with teacher experience in aecting students’ relative performance.

A probabilistic asteroid impact risk model: assessment of sub-300 m impacts

A comprehensive asteroid threat assessment requires the quantification of both the impact likelihood and resulting consequence across the range of possible events. This paper presents a probabilistic asteroid impact risk (PAIR) assessment model developed for this purpose. The model incorporates published impact frequency rates with state-of-the-art consequence assessment tools, applied within a Monte Carlo framework that generates sets of impact scenarios from uncertain input parameter distributions. Explicit treatment of atmospheric entry is included to produce energy deposition rates that account for the effects of thermal ablation and object fragmentation. These energy deposition rates are used to model the resulting ground damage, and affected populations are computed for the sampled impact locations. The results for each scenario are aggregated into a distribution of potential outcomes that reflect the range of uncertain impact parameters, population densities, and strike probabilities. As an illustration of the utility of the PAIR model, the results are used to address the question of what minimum size asteroid constitutes a threat to the population. To answer this question, complete distributions of results are combined with a hypothetical risk tolerance posture to provide the minimum size, given sets of initial assumptions for objects up to 300m in diameter. Model outputs demonstrate how such questions can be answered and provide a means for interpreting the effect that input assumptions and uncertainty can have on final risk-based decisions. Model results can be used to prioritize investments to gain knowledge in critical areas or, conversely, to identify areas where additional data have little effect on the metrics of interest.

Job Displacement during the Great Recession: Tight Bounds on Distributional Treatment Effect Parameters Using Panel Data

This paper develops new techniques to bound distributional treatment effect parameters that depend on the joint distribution of potential outcomes -- an object not identified by standard identifying assumptions such as selection on observables or even when treatment is randomly assigned. I show that panel data and an additional assumption on the dependence between untreated potential outcomes for the treated group over time (i) provide more identifying power for distributional treatment effect parameters than existing bounds and (ii) provide a more plausible set of conditions than existing methods that obtain point identification. I apply these bounds to study heterogeneity in the effect of job displacement during the Great Recession. Using standard techniques, I find that workers who were displaced during the Great Recession lost on average 34% of their earnings relative to their counterfactual earnings had they not been displaced. Using the methods developed in the current paper, I also show that the average effect masks substantial heterogeneity across workers.

Partial identification of functionals of the joint distribution of “potential outcomes”

In this paper, we present a systematic study of partial identification of two general classes of functionals of the joint distribution of two “potential outcomes” when a bivariate sample from the joint distribution is not available to the econometrician. Assuming the identification of the conditional marginal distributions of potential outcomes and the distribution of the covariate vector, we show that the identified sets for functionals in both classes are intervals and provide conditions under which the identified sets point identify the true value of the functionals. In addition, we establish sufficient and necessary conditions for the covariate information to be informative in the sense of shrinking the identified sets. We focus on the application of our general results to evaluating distributional treatment effects of a binary treatment in two commonly used frameworks in the literature for evaluating average treatment effects: the selection on observables framework and a latent threshold-crossing model. We characterize the role of the propensity score in the selection-on-observables framework and the role of endogenous selection in the latent threshold-crossing model. Examples of policy parameters that our results apply include the correlation coefficient between the potential outcomes, many inequality measures of the distribution of treatment effects, and median of the distribution of the individual treatment effect.

Treatment Effects With Unobserved Heterogeneity: A Set Identification Approach

We propose the sharp identifiable bounds of the potential outcome distributions using panel data. We allow for the possibility that statistical randomization of treatment assignments is not achieved until unobserved heterogeneity is properly controlled for. We use certain stationarity assumptions to obtain the sharp bounds. Our approach allows for dynamic treatment decisions, where the current treatment decisions may depend on the past treatments or the past observed outcomes. As an empirical illustration, we study the effect of smoking during pregnancy on infant birthweight. We find that for the group of switchers the infant birthweight of a smoking mother is first-order stochastically dominated by that of a nonsmoking mother.

Regression Discontinuity with Categorical Outcomes

We consider the regression discontinuity (RD) design with categorical outcomes, and exploit the possibility of adapting well-developed microeconometric models to the RD setting. The channels through which the forcing variable affects the potential outcome distributions are constrained to a least extent, to preserve the nonparametric feature of the RD design. Focusing on general categorical outcomes (nominal or ordinal), we develop a new RD estimator based on a nonparametric extension of the well-known multinomial logit model. The key issues of selecting the optimal bandwidth and constructing confidence regions robust to bias correction, of which the solutions only exist so far for the local linear estimator and a single treatment effect, are addressed under the general approach of local likelihood. The proposed estimator and associated inference are easy to implement and the codes in MATLAB are available as a supplement to the paper. They are demonstrated by two empirical applications.

Estimating Conditional Average Treatment Effects

We consider a functional parameter called the conditional average treatment effect (CATE), designed to capture the heterogeneity of a treatment effect across subpopulations when the unconfoundedness assumption applies. In contrast to quantile regressions, the subpopulations of interest are defined in terms of the possible values of a set of continuous covariates rather than the quantiles of the potential outcome distributions. We show that the CATE parameter is nonparametrically identified under unconfoundedness and propose inverse probability weighted estimators for it. Under regularity conditions, some of which are standard and some are new in the literature, we show (pointwise) consistency and asymptotic normality of a fully nonparametric and a semiparametric estimator. We apply our methods to estimate the average effect of a first-time mother’s smoking during pregnancy on the baby’s birth weight as a function of the mother’s age. A robust qualitative finding is that the expected effect becomes stronger (more negative) for older mothers.

(Partially) Identifying Potential Outcome Distributions in Triangular Systems

In this paper we propose a new unifying approach to (partially) identify potential outcome distributions in a non-separable triangular model with a binary endogenous variable and a binary instrument. Our identification strategy provides a testable condition under which the objects of interest are point identified. When point identification is not achieved, we provide sharp bounds on the potential outcome distributions and the difference of marginal distributions.

Financial market assumptions and pension plan models: a comment on PIMS model asset markets assumptions

AbstractThe financial market assumptions of the Pension Benefit Guaranty Corporation (PBGC)'s Pension Insurance Modeling System model are critical inputs to simulations for most apparent uses of the system. They currently appear to be based on a reduced form, ‘classical’ approach to assessing and forecasting the distribution of returns on various classes of input assets, allowing for a fairly sophisticated and useful approach to understanding simulated distributions of potential pension insurance outcomes as well as the net financial status of the PBGC. This technical note discusses some of the capital market side assumptions utilized in the model. It also comments on important related assumptions including the assumed asset allocations of insured plans, making suggestion for possible modification of input assumptions of the model to reflect time variation in financial market return behavior as well as time variation in observed plan allocations.