Synthetic Control Approach Research Articles

Right-To-Carry Laws and Violent Crime: A Comprehensive Assessment Using Panel Data and a State-Level Synthetic Controls Analysis

The 2005 report of the National Research Council (NRC) on Firearms and Violence recognized that violent crime was higher in the post-passage period (relative to national crime patterns) for states adopting right-to-carry (RTC) concealed handgun laws, but because of model dependence the panel was unable to identify the true causal effect of these laws from the then-existing panel data evidence. This study uses 14 additional years of panel data (through 2014) capturing an additional 11 RTC adoptions and new statistical techniques to see if more convincing and robust conclusions can emerge. Our preferred panel data regression specification (the “DAW model”) and the Brennan Center (BC) model, as well as other statistical models by Lott and Mustard (LM) and Moody and Marvell (MM) that had previously been offered as evidence of crime-reducing RTC laws, now consistently generate estimates showing RTC laws increase overall violent crime and/or murder when run on the most complete data. We then use the synthetic control approach of Alberto Abadie and Javier Gardeazabal (2003) to generate state-specific estimates of the impact of RTC laws on crime. Our major finding is that under all four specifications (DAW, BC, LM, and MM), RTC laws are associated with higher aggregate violent crime rates, and the size of the deleterious effects that are associated with the passage of RTC laws climbs over time. Ten years after the adoption of RTC laws, violent crime is estimated to be 13-15% percent higher than it would have been without the RTC law. Unlike the panel data setting, these results are not sensitive to the covariates included as predictors. The magnitude of the estimated increase in violent crime from RTC laws is substantial in that, using a consensus estimate for the elasticity of crime with respect to incarceration of .15, the average RTC state would have to double its prison population to counteract the RTC-induced increase in violent crime.

FX interventions in Brazil: A synthetic control approach

In the aftermath of the taper tantrum, the Central Bank of Brazil announced a major program of sterilized foreign exchange intervention. We use a synthetic control approach to estimate its impact on the level and volatility of the exchange rate. Our counterfactual results, based on the experience of other emerging markets, indicate the program led to an appreciation of the Brazilian real in excess of 10%. Some of our estimates also point to a decline in the option-implied volatility. A second announcement extending the program had more muted effects, and subsequent extensions had little or no impact.

Evaluating the Health Impact of Large-Scale Public Policy Changes: Classical and Novel Approaches.

Large-scale public policy changes are often recommended to improve public health. Despite varying widely—from tobacco taxes to poverty-relief programs—such policies present a common dilemma to public health researchers: how to evaluate their health effects when randomized controlled trials are not possible. Here, we review the state of knowledge and experience of public health researchers who rigorously evaluate the health consequences of large-scale public policy changes. We organize our discussion by detailing approaches to address three common challenges of conducting policy evaluations: distinguishing a policy effect from time trends in health outcomes or preexisting differences between policy-affected and -unaffected communities (using difference-in-differences approaches); constructing a comparison population when a policy affects a population for whom a well-matched comparator is not immediately available (using propensity score or synthetic control approaches); and addressing unobserved confounders by utilizing quasi-random variations in policy exposure (using regression discontinuity, instrumental variables, or near-far matching approaches).

Criminal Prosecution and Physician Supply

While there are many evidences of the effect of medical malpractice tort, research on the effect of medical malpractice criminal sanctions are rare. This paper tries to identify the causal effect of criminal prosecution utilizing exogenous variations over the likelihood of criminal prosecution. In 2004, a medical accident occurred in Fukushima prefecture, Japan and an obstetrician was prosecuted one year after. This prosecution exogenously changed the likelihood of criminal prosecution in Fukushima prefecture. Using synthetic control approach, we estimate the causal effect of criminal prosecution. The prosecution decreased the number of obstetricians by 13% and some of them changed their business to gynecology, which involves lower risk. However, the effect is concentrated on obstetricians, not all physicians. While the relatively weak estimates of tort liability in the literature might suggest strengthening the present sanctions, we need to be cautious about such policy. In addition, the paper shows that the sentence of acquittal did not resolve the effect caused by the initial prosecution. This illuminates the importance of criminal prosecution itself and its social sanctions, not subsequent criminal sanctions. It also highlights the importance of risk perception of physicians.

Trade to aid: EU's temporary tariff waivers for flood-hit Pakistan

In this paper, we study the effectiveness of the first large-scale unilateral trade concessions as foreign aid for disaster relief, i.e., EU tariff waivers on goods heavily exported by Pakistan, which was severely hit by the 2010 floods. Using a triple-difference approach and a synthetic control approach, we find that the tariff waivers substantially increased Pakistan's exports to the EU. The export hike occurred within a few months after the waivers became effective, and did not significantly depress exports by competing countries. While the export boost brought greater employment opportunities in the tariff-waived industries, we find little evidence that the greater labor demands from trade were particularly beneficial to the areas most affected by the floods. Our findings suggest that trade policy may complement traditional means of foreign aid—but trade concessions alone may be inadequate, as the areas most affected by natural disasters may be poorly targeted.

On the heterogeneous consequences of civil war

We show how the occurrence of a civil war has heterogeneous effects on the level of GDP, using case-study, synthetic control and large-N panel-data approaches. We first discuss the relation between these methods and then provide lower and upper estimates of the economic effect of civil war. Although, on average, the incidence of internal conflicts has a negative effect on the GDP level, it is very often insignificant. More importantly, however, both methods display a wide variety of individual separate effects, and in a large number of countries civil war has either no effect or a positive and significant impact on the prospect for economic growth.

Have Pay Equity Laws in Canada Helped Women? A Synthetic-Control Approach

ABSTRACTOntario passed a very aggressive Pay Equity Act in 1988, and in 1996 Quebec passed a similar Pay Equity Act. We use synthetic-control methods to examine what has happened to the gender pay gap (female–male earnings ratio) in Ontario since 2005 and to see whether Quebec’s Pay Equity Act has had any effect on its pay gap. Ontario and Quebec are chosen simply because they are the provinces with the most comprehensive and “aggressive” pay equity laws. We also use synthetic-control methods to investigate whether these acts may have had an adverse effect on the female–male employment ratios in Ontario and Quebec. We find that Ontario’s act has had a negligible effect on that province’s gender pay gap and employment ratio. However, Quebec’s act appears to have reduced the gender pay gap, although at a cost of a somewhat smaller employment ratio for women in that province.

Regional Policy Evaluation: Interactive Fixed Effects and Synthetic Controls

In this paper, we investigate the use of interactive eect or linear factor models in regional policy evaluation. We contrast treatment eect estimates obtained by Bai (2009)'s least squares method with the popular dierence in dierence estimates as well as with estimates obtained using synthetic control approaches as developed by Abadie and coauthors. We show that dierence in dierences are generically biased and we derive the support conditions that are required for the application of synthetic controls. We construct an extensive set of Monte Carlo experiments to compare the performance of these estimation methods in small samples. As an empirical illustration, we also apply them to the evaluation of the impact on local unemployment of an enterprise zone policy implemented in France in the 1990s.

Thirty Years of Conflict and Economic Growth in Turkey: A Synthetic Control Approach

This study seeks to estimate the economic effects of PKK terrorism in Turkey in a causal framework. We create a synthetic control group that reproduces the Turkish real per capita Gross Domestic Product (GDP) before PKK terrorism emerged in the second half of the 1980s. We compare the GDP of the synthetic Turkey without terrorism to the actual Turkey with terrorism for the period 1955-2008. Covering the period of 1988-2008, we find that the Turkish per capita GDP would have been higher by an average of about $1,585 per year had it not been exposed to PKK terrorism. This translates into an average of 13.8 percent higher per capita GDP or a 0.62 percentage points higher annual growth over a period of 21 years. Our estimate is robust to country exclusion, sparse controls, various non-outcome characteristics as predictors of GDP, alternative specifications of the in-space placebo experiments and to other potentially confounding interventions to the sample units in the pre-terrorism period.

Boulevard of broken dreams. The end of EU funding (1997: Abruzzi, Italy)

EU regional policies aim to push regions into self-sustaining growth. Successful interventions would imply a higher growth rate, not only during the treatment (when the region benefits from the transfers), but also after the expiry of the program (when the financing terminates). We investigate to what extent this happened in the case of Italy's Abruzzi region, which entered into the Objective 1 (Convergence) program in 1989 and exited it in 1996 (without a transitional regime). More specifically we focus upon the post expiry period by implementing a synthetic control approach. Our results indicate that exiting the program had a negative effect on regional per-capita GDP growth. This result is a confirmation of the widespread evidence that during their implementation EU regional policies help boost the economic performance of the treated regions. However, additional evidence suggests that the permanent effect of the treatment is negligible: the policies fail to transfer the treated regions to a permanently higher GDP growth path.

Estimating the effect of the EMU on current account balances: A synthetic control approach

The European sovereign debt crisis wrought major political and economic damage on the European Monetary Union (EMU). This led to a reassessment of the pre-crisis period of economic growth and stability in the EMU, shifting attention to the macroeconomic imbalances that emerged between member states, especially those in current account balances. This paper uses macroeconomic data on OECD economies and a new statistical approach for causal inference in observational studies—the synthetic control method—to estimate the effect of the EMU on the current account balances of individual member states. This ‘counterfactuals’ approach provides strong evidence that the introduction of the EMU was responsible for the divergence in current account balances among member states in the run-up to the euro crisis. The results suggest that the EMU effect operated through multiple channels and that fundamental changes to the institutional framework of the EMU may be required to safeguard the currency union against a reemergence of dangerous external imbalances in the future.

The price of the euro: evidence from sovereign debt markets

ABSTRACTThe objective of this article is to ascertain how the Economic and Monetary Union (EMU) in Europe has affected its members’ long-term government bond yields. In order to estimate the effect, this article utilizes a synthetic control approach. The main finding is that the majority of the member countries did not receive economic gains from the EMU in sovereign debt markets. Synthetic counterfactual analysis finds strong evidence that Austria, Belgium, Finland, France and the Netherlands have paid a positive and substantial euro-premium in their 10-year government bonds since the adoption of the single currency. After the most recent financial crisis, government bond yields have been higher in all member countries compared to the situation that would have occurred without the monetary unification. This article concludes that from the viewpoint of sovereign borrowing, it would be beneficial for a country to maintain its own currency and monetary policy.

Measuring the Vice-Presidential Home State Advantage With Synthetic Controls

Measuring the effect strategic choices have on electoral outcomes is problematic, because this requires an assessment of the outcome under a counterfactual that is not observed. To overcome this problem, we extend the synthetic control approach for causal inference to circumstances with multiple treated cases and use it to estimate the effect of vice-presidential candidates on their home states’ vote. Existing research has concluded that vice-presidential candidates have little effect on the outcome of elections in their home states. However, our results from elections spanning 1884-2012 suggest that vice-presidential candidates increase their tickets’ performance in their home states by 2.67 percentage points on average—considerably higher than previous studies have found. In addition, our results suggest that the vice-presidential home state advantage (HSA) could have swung four presidential elections since 1960, if presidential candidates had chosen running mates from strategically optimal states.

Health Behaviors, Mental Health, and Health Care Utilization Among Single Mothers After Welfare Reforms in the 1990s.

We studied the health of low-income US women affected by the largest social policy change in recent US history: the 1996 welfare reforms. Using the Behavioral Risk Factor Surveillance System (1993-2012), we performed 2 types of analysis. First, we used difference-in-difference-in-differences analyses to estimate associations between welfare reforms and health outcomes among the most affected women (single mothers aged 18-64 years in 1997; n = 219,469) compared with less affected women (married mothers, single nonmothers, and married nonmothers of the same age range in 1997; n = 2,422,265). We also used a synthetic control approach in which we constructed a more ideal control group for single mothers by weighting outcomes among the less affected groups to match pre-reform outcomes among single mothers. In both specifications, the group most affected by welfare reforms (single mothers) experienced worse health outcomes than comparison groups less affected by the reforms. For example, the reforms were associated with at least a 4.0-percentage-point increase in binge drinking (95% confidence interval: 0.9, 7.0) and a 2.4-percentage-point decrease in the probability of being able to afford medical care (95% confidence interval: 0.1, 4.8) after controlling for age, educational level, and health care insurance status. Although the reforms were applauded for reducing welfare dependency, they may have adversely affected health.

Invited Commentary: An Interdisciplinary Approach for Policy Evaluation.

Evidence-based policymaking is becoming the norm, but how do we acquire the evidence to inform policies? In their article in the present issue of the Journal, Basu et al. (Am J Epidemiol. 2016;183(6):531-538) used difference-in-difference-in-differences models and a synthetic control approach to examine the effects of the 1996 welfare reforms on health outcomes among single mothers. In the present commentary, we discuss the limitations of observational studies for policy evaluation. Difference-in-differences models, from the field of economics, offer a rigorous approach to cope with those limitations.

Boulevard of Broken Dreams. The End of the EU Funding (1997: Abruzzi, Italy)

EU regional policies aim to push regions into self-sustaining growth. Successful interventions would imply a higher growth rate, not only during the treatment (when the region benefits from the transfers), but also after the expiry of the program (when the financing terminates). We investigate to what extent this happened in the case of Italy’s Abruzzi region, which entered into the Objective 1 (Convergence) program in 1989 and exited it in 1996 (without a transitional regime). More specifically we focus upon the post expiry period by implementing a synthetic control approach. Our results indicate that exiting the program had a negative effect on regional per-capita GDP growth. This result is a confirmation of the widespread evidence that during their implementation EU regional policies help boost the economic performance of the treated regions. However, additional evidence suggests that the permanent effect of the treatment is negligible: the policies fail to transfer the treated regions to a permanently higher GDP growth path.

Assessing Revenue Authority Performance in Developing Countries: A Synthetic Control Approach

ABSTRACTIn many developing countries, comprehensive reforms have been undertaken in tax administration during recent years and the implementation of semiautonomous revenue authorities (SARAs) constitutes one of their most visible expressions. To date, no comprehensive assessment has been undertaken to investigate how successful these SARAs have been in improving revenue collection in these countries. Therefore, this article attempts to fill this gap by assessing revenue collection trends in 20 developing countries after the SARA has been implemented. Using the synthetic control method, we show that in most countries, the implementation of a SARA did not significantly improve government revenues.

Assessing Revenue Authority Performance in Developing Countries: A Synthetic Control Approach

In many developing countries, comprehensive reforms have been undertaken in tax administration during recent years and the implementation of semi-autonomous Revenue Authorities (SARAs) constitutes one of their most visible expressions. To date, no comprehensive assessment has been undertaken to investigate how successful these SARAs have been in improving revenue collection in these countries. Therefore this paper attempts to fill this gap by assessing revenue collection trends in 20 developing countries after the SARA has been implemented. Using the Synthetic Control Method, we show that show that in most countries, the implementation of a SARA did not significantly improve government revenues.

Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units.

This paper examines the synthetic control method in contrast to commonly used difference‐in‐differences (DiD) estimation, in the context of a re‐evaluation of a pay‐for‐performance (P4P) initiative, the Advancing Quality scheme. The synthetic control method aims to estimate treatment effects by constructing a weighted combination of control units, which represents what the treated group would have experienced in the absence of receiving the treatment. While DiD estimation assumes that the effects of unobserved confounders are constant over time, the synthetic control method allows for these effects to change over time, by re‐weighting the control group so that it has similar pre‐intervention characteristics to the treated group.We extend the synthetic control approach to a setting of evaluation of a health policy where there are multiple treated units. We re‐analyse a recent study evaluating the effects of a hospital P4P scheme on risk‐adjusted hospital mortality. In contrast to the original DiD analysis, the synthetic control method reports that, for the incentivised conditions, the P4P scheme did not significantly reduce mortality and that there is a statistically significant increase in mortality for non‐incentivised conditions. This result was robust to alternative specifications of the synthetic control method. © 2015 The Authors. Health Economics published by John Wiley & Sons Ltd.

Is Mississippi's prescription-only precursor control law a prescription to decrease the production and raise the price of methamphetamine?

BackgroundIn 2010, Mississippi became the second state to require a prescription to purchase pseudoephedrine-based medications. Proponents of “prescription-only” laws argue that they are necessary to disrupt methamphetamine markets, but critics note the costs to legal consumers of cold medications may offset some of the laws’ intended benefits. ObjectiveWe evaluated the effect of prescription-only restrictions for methamphetamine precursors on state-level methamphetamine lab seizures and methamphetamine prices. MethodsWe used a synthetic control approach to create a control state comparable to Mississippi and then used permutation testing to determine if the resulting difference was statistically significant. ResultsWe found that Mississippi's prescription-only law removed 2637 small methamphetamine labs in the two years after the law became effective, which represents a 77% reduction in small labs relative to the synthetic counterfactual. We found no evidence that the law impacted methamphetamine prices. ConclusionWe conclude that while prescription-only laws can reduce the number of domestic small methamphetamine labs in operation, methamphetamine availability is unlikely to be materially impacted.