Abstract
Policymakers seeking to target health policies efficiently towards specific population groups need to know which individuals stand to benefit the most from each of these policies. While traditional approaches for subgroup analyses are constrained to only consider a small number of pre-defined subgroups, recently proposed causal machine learning (CML) approaches help explore treatment-effect heterogeneity in a more flexible yet principled way. Causal forests use a generalisation of the random forest algorithm to estimate heterogenous treatment effects both at the individual and the subgroup level. Our paper aims to explore this approach in the setting of health policy evaluation with strong observed confounding, applied specifically to the context of mothers’ health insurance enrolment in Indonesia. Comparing two health insurance schemes (subsidised and contributory) against no insurance, we find beneficial average impacts of enrolment in contributory health insurance on maternal health care utilisation and infant mortality, but no impact of subsidised health insurance. The causal forest algorithm identified significant heterogeneity in the impacts of contributory insurance, not just along socioeconomic variables that we pre-specified (indicating higher benefits for poorer, less educated, and rural women), but also according to some other characteristics not foreseen prior to the analysis, suggesting in particular important geographical impact heterogeneity. Our study demonstrates the power of CML approaches to uncover unexpected heterogeneity in policy impacts. The findings from our evaluation of past health insurance expansions can potentially guide the re-design of the eligibility criteria for subsidised health insurance in Indonesia.
Highlights
Policymakers around the world are implementing health system policies to promote access to essential health care and to meet the health-related Sustainable Development Goals (Sachs 2012)
While the majority of births recorded in our dataset were not covered by any insurance scheme (Appendix B Table 2), subsidised health insurance saw a steep increase from 2005, while infant mortality decreased and the proportion of births attended by a midwife or physician demonstrated a clear upwards trend (Appendix C Fig. 1)
Most variables display large differences, with births under subsidised insurance being more likely to be from a rural household and from mothers who are older at birth, less likely to have studied at university and more likely to have only elementary school education, belong to lower wealth quintiles, and receive social assistance programmes, compared to those without subsidised insurance
Summary
Policymakers around the world are implementing health system policies to promote access to essential health care and to meet the health-related Sustainable Development Goals (Sachs 2012). A key focus for health policies— and for the present paper—is heterogeneity in terms of observed effect modifiers, i.e. measured covariates that can modify the causal effect of a policy. In the typical setting where health policy impact evaluation is based on observational data, some evidence about effect modifiers can be obtained from subgroup analyses, by comparing the effects of interventions across different population groups, characterised, for instance, by their socio-economic status (Mackenbach 2003). Impact evaluations of health policies tend not to present such comparisons, due to concerns that subgroup analysis, unless pre-specified, may produce spurious findings (Petticrew et al 2012). Even when a treatment effect heterogeneity estimation is implemented, it typically involves including ad-hoc interaction terms in the models, necessitating parametric assumptions that are unlikely to hold (Hainmueller and Mummolo 2019)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
