Designing and planning natural experimental evaluations with good practice recommendations

Abstract

Abstract Background There have been significant developments in methods, infrastructure, and data availability for natural experimental evaluations. The rise in the use of natural experimental approaches means it is important that researchers, public health practitioners, funders, and users of intervention research understand the phases involved when designing and planning a natural experimental evaluation and are aware of good practice considerations. Methods Guidance was developed from three international workshops and an online consultation with researchers, journal editors and end users of natural experimental evaluations. The project team comprised researchers with expertise in natural experimental evaluations. The project included an oversight group and an advisory group of independent experts. Results The design and planning of an evaluation involves understanding the definition of a natural experiment, being aware of opportunities for a natural experimental evaluation and the relevance of a systems perspective and mixed methods approaches. The practicalities of planning an evaluation involve assessing the evaluability and feasibility of the evaluation and identifying and involving stakeholders throughout the process. Good practice considerations include understanding the evaluation design and planning processes, engaging stakeholders, and recognising the respective strengths of different analytical approaches. Recommendations for policymakers, public health practitioners and commissioners of evaluations include recognising and using high quality evaluations and incorporating evaluation plans into the implementation of new policies and programmes. Conclusions Understanding the processes involved in designing and planning a natural experimental evaluation are important to successfully conducting and evaluating natural experiment interventions.