Synthetic Control Estimation Research Articles

More doctors, better health? A generalised synthetic control approach to estimating impacts of increasing doctors under Brazil's Mais Medicos programme

Worldwide, there are an insufficient number of primary care physicians to provide accessible, high-quality primary care services. Better knowledge on the health impacts of policies aimed at improving access to primary care physicians is important for informing future policies.Using a generalised synthetic control estimator (GSC), we estimate the effect of the increase in primary care physicians from the Programa Mais Médicos in Brazil. The GSC allows us to estimates a continuous treatment effects which are heterogenous by region. We exploit the variation in physicians allocated to each Brazilian microregion to identify the impact of an increasing Mais Médicos primary care physicians. We explore hospitalisations and mortality rates (both total and from ambulatory care sensitive conditions) as outcomes. Our analysis differs from previous work by estimating the impact of the increase in physician numbers, as opposed to the overall impact of programme participation. We examine the impact on hospitalisations and mortality rates and employ a panel dataset with monthly observations of all Brazilian microregion over the period 2008–2017.We find limited effects of an increase in primary care physicians impacting health outcomes - with no significant impact of the Programa Mais Médicos on hospitalisations or mortality rates. Potential explanations include substitution of other health professionals, impacts materialising over the longer-term, and poor within-region allocation of Mais Médicos physicians.

A method of moments approach to asymptotically unbiased Synthetic Controls

A common approach to constructing a Synthetic Control unit is to fit on the outcome variable and covariates in pre-treatment time periods, but it has been shown by Ferman and Pinto (2021) that this approach does not provide asymptotic unbiasedness when the fit is imperfect and the number of controls is fixed. Many related panel methods have a similar limitation when the number of units is fixed. I introduce and evaluate a new method in which the Synthetic Control is constructed using a General Method of Moments approach where units not being included in the Synthetic Control are used as instruments. I show that a Synthetic Control Estimator of this form will be asymptotically unbiased as the number of pre-treatment time periods goes to infinity, even when pre-treatment fit is imperfect and the number of units is fixed. Furthermore, if both the number of pre-treatment and post-treatment time periods go to infinity, then averages of treatment effects can be consistently estimated. I provide a model selection procedure for deciding whether a unit should be used as an instrument or as a control. I also conduct simulations and an empirical application to compare the performance of this method with existing approaches in the literature.

Empowered or impoverished: the impact of panic buttons on domestic violence

AbstractThis paper estimates the causal effect of a targeted panic button program—implemented in two Turkish provinces between 2012 and 2016—on domestic violence. Difference-in-differences and synthetic control estimates suggest that the program notably increased physical violence against women both at the extensive and intensive margins. The likelihood of physical violence in the treated provinces increased by more than 5 percentage points relative to the control provinces, and the number of physical violence incidents increased by about 10 percent. The increase comes almost entirely from the increase in violence against less-educated women. Employment rates and economic independence indicators improved for women in the treated provinces, which suggests that the program empowered vulnerable women. However, male partners increased physical violence in response to female empowerment. The results are consistent with the male backlash theories and a class of non-cooperative models incorporating violence as a vehicle for enhancing males’ bargaining power, but inconsistent with the models predicting that economic empowerment of women reduces violence by balancing bargaining power within the household. We also develop a method based on retrospective violence information to understand whether the increase is attributable to actual or self-reported violence and conclude that the estimates are entirely driven by the increase in actual violence.

Using synthetic control method to estimate the growth effects of economic liberalisation: Evidence from transition economies

AbstractWe examine the contribution of institutional reforms to economic growth. To this end, we distinguish between several classes of institutional reform in the approach to economic liberalisation. Based on a sample of 24 current and former transition economies for the period 1980–2016, we estimate the counterfactual scenarios related to each distinctive institutional approach to the economic liberalisation by making use of the synthetic control method. Our evidence uncovers notable contrasts in the long‐term effectiveness of designated policy approaches in recovering from the transitional recession. A variety of synthetic control estimates suggests that a sustained big bang approach (rapid reforms) appears to be the most effective approach, while an abortive and gradualist approach tends to produce a permanent breakdown of the growth trajectories relative to the estimated counterfactuals. The point estimates survive an extensive battery of placebo tests. By employing the same methodological tool kit, future research could examine the impact of transition reforms on other socio‐economic variables, such as the poverty rates, income inequalities, health and environmental outcomes.

Is Democracy the Answer to Intractable Climate Change?

Abstract Climate change is the greatest governance challenge humanity has ever faced. Understanding why some governments successfully reduce greenhouse gas emissions and others fail is thus imperative. While regime type is often hypothesized to be a source of variation in greenhouse gas emissions, empirical findings about the effects of democracy and autocracy on climate action are contradictory. This research note reconciles these inconsistencies and adopts a quasi-experimental approach to investigate the relationship between democratization and greenhouse gas emissions. A fixed effects model with a synthetic control estimator is used to construct appropriate counterfactuals and evaluate the effect of regime type on emissions with data from the World Bank and Varieties of Democracy Project. The analysis shows that movement toward democracy does not have a significant effect on emissions, suggesting that research on the politics of emissions reduction should focus on factors other than regime type.

Matching on Noise: Finite Sample Bias in the Synthetic Control Estimator

Abstract We investigate the properties of a systematic bias that arises in the synthetic control estimator in panel data settings with finite pre-treatment periods, offering intuition and guidance to practitioners. The bias comes from matching to idiosyncratic error terms (noise) in the treated unit and the donor units’ pre-treatment outcome values. This in turn leads to a biased counterfactual for the post-treatment periods. We use Monte Carlo simulations to evaluate the determinants of the bias in terms of error term variance, sample characteristics and DGP complexity, providing guidance as to which situations are likely to yield more bias. We also offer a procedure to reduce the bias using a direct computational bias-correction procedure based on re-sampling from a pilot model that can reduce the bias in empirically feasible implementations. As a final potential solution, we compare the performance of our corrections to that of an Interactive Fixed Effects model. An empirical application focused on trade liberalization indicates that the magnitude of the bias may be economically meaningful in a real world setting.

A Design-Based Perspective on Synthetic Control Methods

Since their introduction by Abadie and Gardeazabal, Synthetic Control (SC) methods have quickly become one of the leading methods for estimating causal effects in observational studies in settings with panel data. Formal discussions often motivate SC methods by the assumption that the potential outcomes were generated by a factor model. Here we study SC methods from a design-based perspective, assuming a model for the selection of the treated unit(s) and period(s). We show that the standard SC estimator is generally biased under random assignment. We propose a Modified Unbiased Synthetic Control (MUSC) estimator that guarantees unbiasedness under random assignment and derive its exact, randomization-based, finite-sample variance. We also propose an unbiased estimator for this variance. We document in settings with real data that under random assignment, SC-type estimators can have root mean-squared errors that are substantially lower than that of other common estimators. We show that such an improvement is weakly guaranteed if the treated period is similar to the other periods, for example, if the treated period was randomly selected. While our results only directly apply in settings where treatment is assigned randomly, we believe that they can complement model-based approaches even for observational studies.

Estimating the Effects of Political Instability in Nascent Democracies

Abstract In this paper, we revisit and examine the effect of political instability on institutional quality. To this end, we exploit the crackdown of government cabinet in 2008 that led to almost two decades of pervasive political instability using the unique case of Slovenia and estimate the average treatment effect of political instability and the missing counterfactual scenario for the period 1996–2020. Based on large-scale Bayesian analysis with Metropolis–Hastings algorithm, we extract the residual component of institutional quality from the variation in predetermined unobserved characteristics. Across more than 1.5 million sequenced random samples, we construct novel time-varying estimates of institutional quality at the local and regional level. Our difference-in-differences and synthetic control estimates emphasize a rampant deterioration of institutional quality in response to political instability with evidence of weakened rule of law, less effective public sector, decreased political accountability, and more widespread prevalence of power abuses. The estimated negative effects of political instability are robust to a variety of placebo checks across more than 12 billion placebo averages for institutional quality outcome variable using random sampling method of treatment permutation.

Long-term effects of sectarian politics: evidence from Lebanon

Abstract We examine the effect of sectarian politics in the presence of weak state capacity on long-term economic growth. To this end, we exploit the 1956 civil uprising between Maronite Christian and Sunni Muslim factions in Lebanon to estimate the impact of sectarian political tensions on long-term growth. To isolate the impact of the uprising, we use synthetic control estimator and match Lebanon’s pre-1956 growth and development trajectory with the rest of the world where such uprising did not occur, and estimate the counterfactual growth trajectory in the hypothetical absence of the sectarian conflict. Our evidence indicates large and pervasive negative growth effects of factionalism. Our estimates imply that Lebanon’s per capita income down to the present day is on average 57% lower than that of its pre-1956 synthetic control group without sectarian clashes, and does not seem to be driven by preexisting or subsequent trends and shocks. The negative long-term growth effect of sectarian conflict is robust to a battery of spatial and temporal placebo checks, choice of samples and is not sensitive to the composition of control groups.

Combining process tracing and synthetic control method: Bridging two ways of making causal inference in evaluation research

This article discusses potential ways of combining two methods of evaluation in single-case studies: the synthetic control method and the process tracing method. Both are designed to examine certain events/programmes that take place in given cases but view these events/programmes from different causal perspectives. Seeing an event/programme as a cause, synthetic control estimates its impact on one or more outcomes. Conversely, starting from a certain outcome, process tracing uncovers the causes responsible. One can start from the causal explanation reached via one of the two methods and then proceed to examine that explanation through the other method. Once the causes of an outcome are traced via a process tracing analysis, that account can be validated by estimating the effects of those causes via synthetic control. Equally, once the impact of a certain event is estimated through synthetic control, causal mechanisms traceable via process tracing can be exploited to refine that impact evaluation.

Freedom of choice and health services’ performance: Evidence from a national health system

Public policies fostering the freedom of choice of provider in the healthcare sector are increasingly common in many countries and regions, where policymakers wish to empower patients and improve health service performance. However, in the literature there is not clear consensus about the impact of expanded patient choice on healthcare quality yet. This study investigates whether increasing patients' freedom of choice influences health system outcomes in terms of various non-clinical aspects of care, a dimension often overlooked by researchers in this field. Our study considers a “natural experiment” that took place within the Spanish National Health System in 2009 under which citizens of the Community (region) of Madrid were allowed to freely choose among any GP and/or specialist in their region. The empirical analysis was conducted by using Spanish microdata for the period 2002–2016 and used synthetic control estimation techniques. The key findings show the reform had a strong and long-lasting impact, reducing average waiting times and increasing patients' satisfaction with the specialist attention received. We did not detect any statistically significant impact of the reform on the other responsiveness domains analysed. Our analysis shows that freedom of choice policies could improve health system performance if they are combined with appropriate economic incentives for health providers.

Long-Term Economic Effects of Populist Legal Reforms: Evidence from Argentina.

In this paper, we examine the consequences of populist government for long-term economic growth and development. To this end, we estimate the long-term growth impact of the Juan Péron’s political rule in Argentina, which led to a comprehensive overhaul of the institutional framework laid by the Argentine founding fathers in the 1853 Constitution. Our hypothesis is that the progressive substitution of a growth-enhancing institutional framework by exclusionary growth-distorting frameworks explains Argentina’s economic decline from one the world’s richest countries on the eve of World War I to an underdeveloped nation in the present day. We emphasize the erosion of the rule of law and restraint of economic freedom during Perón’s first government (1943–1955) as two fundamental coadjutant causes of Argentina’s decline. The populist legal reforms of Perón had long-lasting adverse economic effects. By comparing Argentina’s pre-Péron growth trajectory with a donor pool of 58 countries for the period 1860–2015, we estimate the counterfactual scenario without Péron’s reforms. A variety of synthetic control estimates uncover substantial negative effects of the weakening of the rule of law and the populist reforms that began in 1940s on the trajectory of economic growth and development. The populist overhaul negated the economic growth advantages inherent in the 1853 Alberdian constitution. Without the short-sighted populist Peron episodes, Argentina would be a rich country down to the present day with per capita income comparable to southern European countries. We also perform a series of randomization inferences and a battery of placebo analyses, which confirm our results.Supplementary InformationThe online version contains supplementary material available at 10.1057/s41294-022-00193-4.

How local partnerships to improve urgent and emergency care have impacted delayed transfers of care from hospitals in England: an analysis based on a synthetic control estimation method

ObjectivesPatients should be discharged from hospital when they are medically fit. However, discharges are often delayed for non-medical reasons including access to social care. One aim of local health and...

Combining Matching and Synthetic Control to Tradeoff Biases From Extrapolation and Interpolation

The synthetic control (SC) method is widely used in comparative case studies to adjust for differences in pretreatment characteristics. SC limits extrapolation bias at the potential expense of interpolation bias, whereas traditional matching estimators have the opposite properties. This complementarity motives us to propose a matching and synthetic control (or MASC) estimator as a model averaging estimator that combines the standard SC and matching estimators. We show how to use a rolling-origin cross-validation procedure to train the MASC to resolve tradeoffs between interpolation and extrapolation bias. We use a series of empirically based placebo and Monte Carlo simulations to shed light on when the SC, matching, MASC and penalized SC estimators do (and do not) perform well. Then, we apply these estimators to examine the economic costs of conflicts in the context of Spain.

The economics of state fragmentation: Assessing the economic impact of secession

AbstractThis paper presents estimates of the economic effects of secession for a large panel of countries that gained independence between 1940 and 2016. It relies on a semi‐parametric identification strategy that controls for the confounding effects of past gross domestic product (GDP) dynamics, anticipation effects, unobserved heterogeneity, model uncertainty, and effect heterogeneity. Our findings indicate that secession lowered per capita GDP, on average by around 24% in the 10th post‐independence year. Nevertheless, the associated cross‐sectional standard deviation of 25% emphasizes that the economic impact of secession also varied widely across countries. A novel procedure to produce confidence intervals around synthetic control estimates of treatment effects demonstrates the statistical significance of the findings. We document the implications for several historical independence waves and connect them to the existing literature. We find tentative evidence that, in decreasing order of relative importance, the adverse effects of independence are disproportionally present in non‐oil producing landlocked countries and are mitigated when newly independent states use their new‐found political autonomy to democratize or to liberalize their trade regime.

On the Properties of the Synthetic Control Estimator with Many Periods and Many Controls

We consider the asymptotic properties of the synthetic control (SC) estimator when both the number of pretreatment periods and control units are large. If potential outcomes follow a linear factor model, we provide conditions under which the SC unit asymptotically recovers the factor structure of the treated unit, even when the pretreatment fit is imperfect. This happens when there are weights diluted among an increasing number of control units such that a weighted average of the factor structure of the control units asymptotically reconstructs the factor structure of the treated unit. In this case, the SC estimator is asymptotically unbiased even when treatment assignment is correlated with time-varying unobservables. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

New Evidence on the Effects of Mandatory Waiting Periods for Abortion

Beyond a handful of studies examining early-adopting states in the early 1990s, little is known about the causal effects of mandatory waiting periods for abortion. In this study we evaluate the effects of a Tennessee law enacted in 2015 that requires women to make an additional trip to abortion providers for state-directed counseling at least 48 hours before they can obtain an abortion. Our difference-in-differences and synthetic-control estimates indicate that the introduction of the mandatory waiting period caused a 53–69 percent increase in the share of abortions obtained during the second trimester. Our analysis examining overall abortion rates is less conclusive but suggests a reduction caused by the waiting period. To put these estimates into context, we provide back-of-the-envelope calculations on the additional monetary costs that Tennessee’s MWP imposes on women seeking abortions.

A Penalized Synthetic Control Estimator for Disaggregated Data

Synthetic control methods are commonly applied in empirical research to estimate the effects of treatments or interventions on aggregate outcomes. A synthetic control estimator compares the outcome of a treated unit to the outcome of a weighted average of untreated units that best resembles the characteristics of the treated unit before the intervention. When disaggregated data are available, constructing separate synthetic controls for each treated unit may help avoid interpolation biases. However, the problem of finding a synthetic control that best reproduces the characteristics of a treated unit may not have a unique solution. Multiplicity of solutions is a particularly daunting challenge when the data includes many treated and untreated units. To address this challenge, we propose a synthetic control estimator that penalizes the pairwise discrepancies between the characteristics of the treated units and the characteristics of the units that contribute to their synthetic controls. The penalization parameter trades off pairwise matching discrepancies with respect to the characteristics of each unit in the synthetic control against matching discrepancies with respect to the characteristics of the synthetic control unit as a whole. We study the properties of this estimator and propose data-driven choices of the penalization parameter.

Synthetic Control Estimation Beyond Comparative Case Studies: Does the Minimum Wage Reduce Employment?

Panel data are often used in empirical work to account for fixed additive time and unit effects. The synthetic control estimator relaxes the assumption of additive fixed effects for comparative case studies in which a treated unit adopts a single policy. This article generalizes the synthetic control estimator to estimate parameters associated with multiple discrete or continuous explanatory variables, jointly estimating the parameters and synthetic controls for each unit. I apply the estimator to study the disemployment effects of the minimum wage, estimating that increases in the minimum wage reduce employment.

Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects

Probably because of their interpretability and transparent nature, synthetic controls have become widely applied in empirical research in economics and the social sciences. This article aims to provide practical guidance to researchers employing synthetic control methods. The article starts with an overview and an introduction to synthetic control estimation. The main sections discuss the advantages of the synthetic control framework as a research design, and describe the settings where synthetic controls provide reliable estimates and those where they may fail. The article closes with a discussion of recent extensions, related methods, and avenues for future research. (JEL B41, C32, C54, E23, F15, O47)