P13 Synthetic control methodology as a tool for evaluating public health interventions – a literature review

Abstract

Background The synthetic control method (SCM) improves causal inference in non-randomised studies by building a counterfactual using a weighted combination of potential control units. Although it has been widely used in other disciplines it is not widely used in public health research. Our objectives were to identify the use of SCM studies in health and to summarise strengths and limitations identified in the literature. Methods We included studies that used a SCM design to investigate a health outcome of any intervention in any population. We searched for the term ‘synthetic control method’ in 26 health, social science and grey literature databases as well as checking for additional studies by key authors. No restrictions were placed on language or date. Searches were completed in February 2016. We summarised key information about the studies including setting, number of treated and control units, intervention and outcome, number of pre- and post-intervention data-points available and other methods used in the same study. Results Searches identified 35 health-related studies of which 23 were from US settings and investigated a single treated unit. Most studies had at least 10 control units. Interventions investigated included health finance and health systems reform, industry regulation and taxation policies. Common outcomes were mortality rates and insurance rates/health care access. Most studies had more than 4 pre- and post-implementation data-points. SCM is most commonly used alongside difference-in-difference methods. Advantages of SCM are that it does not rely on parallel pre-implementation trends and that it allows for time-varying unmeasured confounders. Limitations include the need for suitable data on both the treated unit and a pool of potential controls, difficulties if the treated unit is an outlier and the inapplicability of traditional statistical tests due to the small number of treated and control units and the fact that they have not been randomly allocated. Falsification tests are generally used as an alternative. Conclusion This comprehensive literature review suggests thatSCM has been little used in health despite some advantages over existing methods. Future research incorporating the method, ideally in combination with other methods, would be of value.