Abstract
Background: The importance of evaluating policy processes to achieve health equity is well recognised but such evaluation encounters methodological, theoretical and political challenges. This paper describes how a program theorybased evaluation framework can be developed and tested, using the example of an evaluation of the South Australian Health in All Policies (HiAP) initiative. Methods: A framework of the theorised components and relationships of the HiAP initiative was produced to guide evaluation. The framework was the product of a collaborative, iterative process underpinned by a policy-research partnership and drew on social and political science theory and relevant policy literature. Results: The process engaged key stakeholders to capture both HiAP specific and broader bureaucratic knowledge and was informed by a number of social and political science theories. The framework provides a basis for exploring the interactions between framework components and how they shape policy-making and public policy. It also enables an assessment of HiAP’s success in integrating health and equity considerations in policies, thereby laying a foundation for predicting the impacts of resulting policies. Conclusion: The use of a program theory-based evaluation framework developed through a consultative process and informed by social and political science theory has accommodated the complexity of public policy-making. The framework allows for examination of HiAP processes and impacts, and for the tracking of contribution towards distal outcomes through the explicit articulation of the underpinning program theory.
Highlights
Since 2008 the South Australian Government has implemented a Health in All Policies (HiAP) approach to facilitate the development of healthy public policy
This paper examines the development of a program theory-based framework used to guide evaluation of the South Australian HiAP initiative over a five year period
Understanding the HiAP Evaluation Framework The product of this process was a graphic framework of the South Australian HiAP initiative
Summary
Evaluation of policy-making processes and outcomes presents methodological, theoretical and political challenges.[1,2] Policy and policy work is understood and described in diverse and sometimes conflicting ways[3] but these ‘multiple accounts’ can help make sense of the processes, each adding a particular perspective.[2,4] Such interpretive approaches to policy-making suggest that research should examine how institutions shape thinking and working, attend to the role of relationships and boundaries in the ‘doing’ of policy work, uncover the actual practices of policy-making and the role of ideas and ideation processes in reform efforts, problem identification and representation.[4,5,6] The activities of policy-making are broad and include the identification and conceptualisation of problems, gaining the attention of government, the formulation of policy alternatives, selection of policy solutions and implementation, evaluation and revision – not as discrete episodes but as an ongoing ‘juggling’ process.[2,3] Evaluators, as well as the policy-makers need to negotiate the political context which includes political cycles, key personnel changes and the waxing and waning of political support.[7]. A pragmatic but systematic approach to causal analysis - which tests the congruence of the program theory with the results, employs counterfactual comparisons where possible, reviews results critically and seeks to explain exceptions and identify alternative explanations - may be more appropriate.[20] Key actors can assist researchers in building a robust understanding of an intervention, its causal pathways and its context. Such involvement requires the engagement of policy actors in making sense of the data strategically and operationally. Colebatch maintains that ‘getting close’ to policy processes and policy actors is important for research to trace the ‘evolution of policy work over time.’[6]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
