Counterfactual Outcomes Research Articles

Algeria’s Trade with GAFTA Countries: A Synthetic Control Approach

Arab countries signed the Greater Arab Free Trade Area (GAFTA) agreement in 1998, while Algeria only joined in 2005. In this study, I investigate the counterfactual trade that Algeria would have had with its GAFTA trade partners had it joined the agreement in 1998. I employ synthetic control methods (SCM) in the context of the gravity approach to international trade to estimate this counterfactual outcome. The SCM creates the synthetic control unit based on a weighted average of the past characteristics and trade realizations of the countries that signed the GAFTA agreement and compares the counterfactual trade outcome against the actual outcome of the treated unit, Algeria in our case. Estimation results indicate that Algeria’s trade with nine GAFTA countries, covering more than 96 % of Algeria’s pre-treatment trade, would have improved if Algeria had signed the agreement in 1998.

The impact of NERICA adoption on productivity and poverty of the small-scale rice farmers in the Gambia

The study assessed the causal effects of NERICA adoption on rice yields and income, using country-wide data of 600 rice farmers in The Gambia. To control for differences in socio-demographic and environmental characteristics of adopters and non-adopters and enable a causal interpretation of NERICA adoption on the variables of interest, the study used the counterfactual outcome framework to control for such differences. The results, based on observed sample estimates, showed significant differences in rice yields and income between the NERICA adopters and non-adopters. Further, the results of the framework, based on the Local Average Treatment Effect (LATE) estimates, which allowed a causal interpretation, showed that NERICA adoption significantly increased average rice yields and annual income of small-scale rice farmers by 157 kg per hectare and $148, respectively.

Partial identification of distributional and quantile treatment effects in difference-in-differences models

In this paper, we supplement the identification results for the mean treatment effect for the treated in the difference-in-differences framework studied by Abadie (2005) by establishing partial identification results for the distribution and quantile of the counterfactual outcome and of the treatment effect for the treated. Unlike the mean treatment effect which is identifiable from either the panel or the repeated cross sectional data, we show that the distribution/quantile of either the counterfactual outcome or of the treatment effect is only partially identified and in general the identified interval is tighter with panel than with the repeated cross sectional data. We further illustrate this gain from the panel via a numerical example.

Additive Decompositions with Interaction Effects

This paper proposes a comprehensive, path-independent decomposition formula of changes into ceteris paribus effects and interaction effects. The formula implies a reassessment of sequential decomposition methods that are widely used in the literature and that are restrictive in how they treat interaction effects. If counterfactual outcomes are correctly specified, it may also be viewed as a description of certain aspects of causality in the situation where more than one causal influence is present.

Stochastic counterfactuals and stochastic sufficient causes.

Most work in causal inference concerns deterministic counterfactuals; the literature on stochastic counterfactuals is small. In the stochastic counterfactual setting, the outcome for each individual under each possible set of exposures follows a probability distribution so that for any given exposure combination, outcomes vary not only between individuals but also probabilistically for each particular individual. The deterministic sufficient cause framework supplements the deterministic counterfactual framework by allowing for the representation of counterfactual outcomes in terms of sufficient causes or causal mechanisms. In the deterministic sufficient cause framework it is possible to test for the joint presence of two causes in the same causal mechanism, referred to as a sufficient cause interaction. In this paper, these ideas are extended to the setting of stochastic counterfactuals and stochastic sufficient causes. Formal definitions are given for a stochastic sufficient cause framework. It is shown that the empirical conditions that suffice to conclude the presence of a sufficient cause interaction in the deterministic sufficient cause framework suffice also to conclude the presence of a sufficient cause interaction in the stochastic sufficient cause framework. Two examples from the genetics literature, in which there is evidence that sufficient cause interactions are present, are discussed in light of the results in this paper.

Introduction to the Virtual Issue: Past and Future Research Agenda on Causal Inference

If you ask political scientists whether a major goal of their empirical research is to test causal relationships, I suspect that most of them answer “yes.” In contrast, many statisticians would say that their methods may improve predictions but have little to do with causal inference. As a result, even as an increasing number of social scientists use statistical methods to infer causal effects in their empirical research, many of these methods were not originally designed for causal inference. This intellectual gap has been rapidly filled by two “revolutions.” In the community of empirical researchers, the identification revolution has occurred where the key assumptions for identifying causal quantities are taken much more seriously than before. Problems related to measurement and selection bias are no longer brushed under the table. The discipline has recently witnessed a rise in the use of randomized experiments, and an increasing number of political scientists now choose research designs to improve the credibility of identification assumptions (for example, see the 2009 Political Analysis special issue on “Natural Experiments in Political Science”). Simultaneously, among methodologists in various disciplines, the potential outcomes revolution has taken place where traditional statistical methods are reframed within the formal framework of causal inference. The potential outcomes framework, which originates in the classic works of Neyman and Rubin, makes explicit the fundamental problem of causal inference: to infer causal effects one must estimate counterfactual outcomes from observed data. Along with the development of new methodologies, this revolution has led to re-interpretations of familiar statistical methods such as regression and instrumental variables methods, and clarified the exact conditions under which they can be used to infer causal relationships from observed data. Many political methodologists contributed to these new developments. Here, I selected six articles published in Political Analysis between 2005 and 2010 that represent the leadership taken by political methodologists to contribute to advancement of causal inference. These articles have influenced the way in which political scientists and other social scientists conduct their empirical research, and three of them won the Warren Miller best paper award for 2005, 2007, and 2009. After briefly discussing each article with references to other relevant Political Analysis articles, I identify further research challenges where unresolved and yet important methodological problems await the contributions from political methodologists.

Augmented Designs to Assess Principal Strata Direct Effects

SummaryMany studies involving causal questions are often concerned with understanding the causal pathways by which a treatment affects an outcome. Thus, the concept of ‘direct’versus‘indirect’ effects comes into play. We tackle the problem of disentangling direct and indirect effects by investigating new augmented experimental designs, where the treatment is randomized, and the mediating variable is not forced, but only randomly encouraged. There are two key features of our framework: we adopt a principal stratification approach, and we mainly focus on principal strata effects, avoiding involving a priori counterfactual outcomes. Using non-parametric identification strategies, we provide a set of assumptions, which allow us to identify partially the causal estimands of interest: the principal strata direct effects. Some examples are shown to illustrate our design and causal estimands of interest. Large sample bounds for the principal strata average direct effects are provided, and a simple hypothetical example is used to show how our augmented design can be implemented and how the bounds can be calculated. Finally our augmented design is compared and contrasted with a standard randomized design.

Decomposition of Inequality among Groups by Counterfactual Modeling: An Analysis of the Gender Wage Gap in Japan

This paper introduces a new method for decomposing group differences in the mean of a variable into various within-group and between-group components with respect to group categories of intermediary variables. This is accomplished by considering counterfactual outcomes that would be realized by social interventions that change the relationship among variables. Because such a change does not by itself determine the counterfactual outcome, the paper introduces and juxtaposes two different mechanisms—the mechanism of realizing the counterfactual state that deviates least from the existing state, and the mechanism of holding relations among variables other than those that are modified by a given intervention unchanged—and demonstrates that despite the large difference in the mechanisms, they yield highly congruent outcomes. As an illustrative example, the paper analyzes gender inequality in hourly wages in Japan and thereby demonstrates the usefulness of the new method for deriving policy implications.

Learning: Not Just the Facts, Ma'am, but the Counterfactuals as Well

Our brains allow us to consider rewards and other scenarios that could have happened but did not. Such counterfactual outcomes can influence our choices and hasten learning. A series of recent studies has begun to untangle the neural circuitry responsible for monitoring counterfactual outcomes. Here, we summarize several recent complementary discoveries, including a new article in the current issue of PLoS Biology. Neurons in several brain areas that process directly experienced rewards respond to counterfactual information about rewards as well. Among these brain regions, the frontal pole appears to be most specialized, and carries a decision variable representing the value of the best alternative option. Together, these findings suggest that counterfactual learning and thinking build upon scaffolding circuits that evolved to learn from direct experience.

Counterfactual Choice and Learning in a Neural Network Centered on Human Lateral Frontopolar Cortex

Decision making and learning in a real-world context require organisms to track not only the choices they make and the outcomes that follow but also other untaken, or counterfactual, choices and their outcomes. Although the neural system responsible for tracking the value of choices actually taken is increasingly well understood, whether a neural system tracks counterfactual information is currently unclear. Using a three-alternative decision-making task, a Bayesian reinforcement-learning algorithm, and fMRI, we investigated the coding of counterfactual choices and prediction errors in the human brain. Rather than representing evidence favoring multiple counterfactual choices, lateral frontal polar cortex (lFPC), dorsomedial frontal cortex (DMFC), and posteromedial cortex (PMC) encode the reward-based evidence favoring the best counterfactual option at future decisions. In addition to encoding counterfactual reward expectations, the network carries a signal for learning about counterfactual options when feedback is available—a counterfactual prediction error. Unlike other brain regions that have been associated with the processing of counterfactual outcomes, counterfactual prediction errors within the identified network cannot be related to regret theory. Furthermore, individual variation in counterfactual choice-related activity and prediction error-related activity, respectively, predicts variation in the propensity to switch to profitable choices in the future and the ability to learn from hypothetical feedback. Taken together, these data provide both neural and behavioral evidence to support the existence of a previously unidentified neural system responsible for tracking both counterfactual choice options and their outcomes.

How comparing decision outcomes affects subsequent decisions: The carry-over of a comparative mind-set

AbstractIn the current paper we investigate how feedback over decision outcomes may affect future decisions. In an experimental study we demonstrate that if people receive feedback over the outcomes they obtained (“factual outcomes”) and the outcomes they would have obtained had they decided differently (“counterfactual outcomes”), they become regret-averse in subsequent decisions. This effect is not only observed when this feedback evoked regret (with counterfactual outcomes being higher than factual outcomes), but even when the feedback evoked no regret (with factual outcomes being equal to counterfactual outcomes). The findings suggest that this effect on subsequent decisions is at least partly due to the transfer of a comparison mind-set triggered in the prior choice.

How to use marginal structural models in randomized trials to estimate the natural direct and indirect effects of therapies mediated by causal intermediates

Although intention-to-treat analysis is a standard approach, additional supplemental analyses are often required to evaluate the biological relationship among interventions, intermediates, and outcomes. Therefore, we need to evaluate whether the effect of an intervention on a particular outcome is mediated by a hypothesized intermediate variable. To evaluate the size of the direct effect in the total effect, we applied the marginal structural model to estimate the average natural direct and indirect effects in a large-scale randomized controlled trial (RCT). Method The average natural direct effect is defined as the difference in the probability of a counterfactual outcome between the experimental and control arms, with the intermediate set to what it would have been, had the intervention been a control treatment. We considered two marginal structural models to estimate the average natural direct and indirect effects introduced by VanderWeele (Epidemiology 2009) and applied them in a large-scale RCT - the Candesartan Antihypertensive Survival Evaluation in Japan (CASE-J trial) - that compared angiotensin receptor blockers and calcium-channel blockers in high-risk hypertensive patients. There were no strong blood pressure-independent or dependent effects; however, a systolic blood pressure reduction of about 1.9 mmHg suppressed all events. Compared to the blood pressure-independent effects of calcium channel blockers, those of angiotensin receptor blockers contributed positively to cardiovascular and cardiac events, but negatively to cerebrovascular events. There is a particular condition for estimating the average natural direct effect. It is impossible to check whether this condition is satisfied with the available data. We estimated the average natural direct and indirect effects through the achieved systolic blood pressure in the CASE-J trial. This first application of estimating the average natural effects in an RCT can be useful for obtaining an in-depth understanding of the results and further development of similar interventions.

“If only” counterfactual thoughts about exceptional actions

People create counterfactual alternatives that change an exceptional action to be like a usual one (e.g., "if he had placed his usual small bet he would have lost less"), as shown in Experiment 1. Experiments 2 and 3 eliminated and reversed this well-known effect: An exceptional action is instead changed to an exceptional alternative when it leads to a better outcome. Experiments 4 and 5 show that the reversal occurs whether or not the exceptional alternative is a justified action. The results indicate that the tendency to change an exceptional action to be like a usual one is guided by the optimality of the counterfactual outcome more than the exceptionality or justifiability of the action. The implications for theories of the counterfactual imagination are discussed.

Counterfactual closeness and predicted affect

Empirical research on counterfactual thinking has found a closeness effect: people report higher negative affect if an actual outcome is close to a better counterfactual outcome. However, it remains unclear what actually is a “close” miss. In three experiments that manipulate close counterfactuals, closeness effects were found only when closeness was unambiguously defined either with respect to a contrasted alternative, or with respect to a categorical boundary. In a real task people failed to report greater negative affect when encountering a close numerical miss, while they predicted greater negative affect hypothetically. These results show that counterfactual closeness effects on affect depend on closeness being accessible and unambiguously defined.

Discrete Games with Flexible Information Structures: An Application to Local Grocery Markets

Game-theoretic models are frequently employed to study strategic interaction between agents. Empirical research has focused on estimating payoff functions while maintaining strong assumptions regarding the information structure of the game. I show how to relax informational assumptions to enhance the credibility of empirical analysis in discrete games. I propose the use of a flexible information structure that nests the most common assumptions used in the discrete games literature. Despite the added flexibility, the parameters of the players’ payoff functions remain point identified under standard rich support assumptions. As is common in discrete-choice games, multiple equilibria complicate identification and inference in empirical applications. I introduce an approach to equilibrium selection that avoids parametric assumptions and lets selection depend on public information that is not observed by the econometrician. I apply the method to data on the entry and exit patterns of grocery stores. The model provides useful bounds to equilibrium outcomes. In addition, the empirical analysis indicates that more restrictive informational assumptions commonly used in empirical applications models can generate qualitatively misleading counterfactual outcomes.

Striatal sensitivity to personal responsibility in a regret-based decision-making task

Regret and relief are complex emotional states associated with the counterfactual processing of nonobtained outcomes in a decision-making situation. In the "actor effect," a sense of agency and personal responsibility is thought to heighten these emotions. Using fMRI, we scanned volunteers (n = 22) as they played a task involving choices between two wheel-of-fortune gambles. We examined how neural responses to counterfactual outcomes were modulated by giving subjects the opportunity to change their minds, as a manipulation of personal responsibility. Satisfaction ratings to the outcomes were highly sensitive to the difference between the obtained and nonobtained outcome, and ratings following losses were lower on trials with the opportunity to change one's mind. Outcome-related activity in the striatum and orbitofrontal cortex was positively related to the satisfaction ratings. The striatal response was modulated by the agency manipulation: Following losses, the striatal signal was significantly lower when the subject had the opportunity to change his/her mind. These results support the involvement of frontostriatal mechanisms in counterfactual thinking and highlight the sensitivity of the striatum to the effects of personal responsibility.

A structural mean model to allow for noncompliance in a randomized trial comparing 2 active treatments

We propose a structural mean modeling approach to obtain compliance-adjusted estimates for treatment effects in a randomized-controlled trial comparing 2 active treatments. The model relates an individual's observed outcome to his or her counterfactual untreated outcome through the observed receipt of active treatments. Our proposed estimation procedure exploits baseline covariates that predict compliance levels on each arm. We give a closed-form estimator which allows for differential and unexplained selectivity (i.e. noncausal compliance-outcome association due to unobserved confounding) as well as a nonparametric error distribution. In a simple linear model for a 2-arm trial, we show that the distinct causal parameters are identified unless covariate-specific expected compliance levels are proportional on both treatment arms. In the latter case, only a linear contrast between the 2 treatment effects is estimable and may well be of key interest. We demonstrate the method in a clinical trial comparing 2 antidepressants.

Identification of Stochastic Sequential Bargaining Models, Second Version

Stochastic sequential bargaining games (Merlo and Wilson (1995, 1998)) have found wide applications in various fields including political economy and macroeconomics due to their flexibility in explaining delays in reaching an agreement. In this paper, we present new results in nonparametric identification of such models under different scenarios of data availability. First, we give conditions for an observed distribution of players’ decisions and agreed allocations of the surplus, or the to be rationalized by a sequential bargaining model. We show the common discount rate is identified, provided the surplus is monotonic in unobservable states (USV) given observed ones (OSV). Then the mapping from states to surplus, or the function, is also recovered under appropriate normalizations. Second, when the cake is only observed under agreements, the discount rate and the impact of observable states on the cake can be identified, if the distribution of USV satisfies some exclusion restrictions and the cake is additively separable in OSV and USV. Third, if data only report when an agreement is reached but never report the size of the cake, we propose a simple algorithm that exploits shape restrictions on the cake function and the independence of USV to recover all rationalizable probabilities for agreements under counterfactual state transitions. Numerical examples show the set of rationalizable counterfactual outcomes so recovered can be informative.

Targeted Maximum Likelihood Based Causal Inference: Part I

Given causal graph assumptions, intervention-specific counterfactual distributions of the data can be defined by the so called G-computation formula, which is obtained by carrying out these interventions on the likelihood of the data factorized according to the causal graph. The obtained G-computation formula represents the counterfactual distribution the data would have had if this intervention would have been enforced on the system generating the data. A causal effect of interest can now be defined as some difference between these counterfactual distributions indexed by different interventions. For example, the interventions can represent static treatment regimens or individualized treatment rules that assign treatment in response to time-dependent covariates, and the causal effects could be defined in terms of features of the mean of the treatment-regimen specific counterfactual outcome of interest as a function of the corresponding treatment regimens. Such features could be defined nonparametrically in terms of so called (nonparametric) marginal structural models for static or individualized treatment rules, whose parameters can be thought of as (smooth) summary measures of differences between the treatment regimen specific counterfactual distributions. In this article, we develop a particular targeted maximum likelihood estimator of causal effects of multiple time point interventions. This involves the use of loss-based super-learning to obtain an initial estimate of the unknown factors of the G-computation formula, and subsequently, applying a target-parameter specific optimal fluctuation function (least favorable parametric submodel) to each estimated factor, estimating the fluctuation parameter(s) with maximum likelihood estimation, and iterating this updating step of the initial factor till convergence. This iterative targeted maximum likelihood updating step makes the resulting estimator of the causal effect double robust in the sense that it is consistent if either the initial estimator is consistent, or the estimator of the optimal fluctuation function is consistent. The optimal fluctuation function is correctly specified if the conditional distributions of the nodes in the causal graph one intervenes upon are correctly specified. The latter conditional distributions often comprise the so called treatment and censoring mechanism. Selection among different targeted maximum likelihood estimators (e.g., indexed by different initial estimators) can be based on loss-based cross-validation such as likelihood based cross-validation or cross-validation based on another appropriate loss function for the distribution of the data. Some specific loss functions are mentioned in this article. Subsequently, a variety of interesting observations about this targeted maximum likelihood estimation procedure are made. This article provides the basis for the subsequent companion Part II-article in which concrete demonstrations for the implementation of the targeted MLE in complex causal effect estimation problems are provided.

Fungibility and the impact of development assistance: Evidence from Vietnam's health sector

The apparent fungibility of aid is a challenge to the evaluation of donor-funded development projects, requiring a comparison of the observed outcomes with the outcomes that would have occurred if the project had not gone ahead. Where projects are targeted on specific geographic areas, counterfactual outcomes in each can differ from observed outcomes because the amount of government spending (gross of aid) differs, the productivity of government spending differs, or both. This paper estimates the benefits of two concurrent World Bank health projects in Vietnam targeted on specific provinces. Estimates are derived from a model linking outcomes (under-five mortality) to government spending before and after the project and in project and nonproject provinces, and are presented for different assumptions regarding fungibility of funds (zero and full fungibility) and the impacts of the project on the productivity of government spending (the project modifies productivity in both sectors equally and in neither sector). The estimated mortality reductions are highly insensitive to the assumed degree of fungibility, but highly sensitive to the assumed productivity effects (the estimates range from 1 to 25%). The wide range reflects the uncertainty due to the lack of a genuine control group of provinces.