Advocacy Evaluation Research Articles

Understanding the what, how, and why in advocacy: Assessing the applicability of participatory process evaluation methodology in an advocacy context

While evaluations are critical for non-governmental organizations to strengthen their advocacy strategies, evaluators and advocates encounter many difficulties evaluating such efforts. This article discusses the contribution of the participatory process evaluation methodology to advocacy evaluation, using a Dutch global health advocacy program as a case study. As participatory process evaluation is a novel methodology in the field of advocacy, the article’s primary focus concerns the application and utility of the methodology. Findings suggest that participatory process evaluation in an advocacy context can provide insights into the implementation of advocacy tools and activities, encouraging reflection and leading to ideas and practical tools to strengthen advocacy efforts. While participatory process evaluation can help overcome some of the often-experienced barriers in advocacy evaluation, further research is needed to consolidate advocacy evaluation theory and practice.

Addressing Diversity, Equity, and Inclusivity Contributions in Academic Review.

The nursing profession and academic environments are increasingly calling for nurse educators to incorporate diversity, equity, and inclusion (DEI) goals and actions into their professional practice. Such work aligns with institutional and professional missions of social justice and antiracism. Historically, there has been an undervaluing and lack of recognition in academia of DEI contributions and qualitative or community-based research focused on health equity. This lack of recognition may disproportionately impact faculty who focus their scholarship and research on marginalized communities. The current state of the academic review of DEI contributions for appointment, promotion, and tenure is presented. Examples for teaching, research and scholarship, practice, and service are provided. Recommendations include expanding recognition and evaluation of academic scholarship and advocacy against efforts to dismantle social justice and antiracism advances within academia. Examining hegemonic norms is vital to promote more equitable power structures in academic environments.

A Theoretically Derived Approach to Impact: Implementing Policy Influence Strategies.

Advocacy is an intentional act of influencing government and an important precondition for successful policy change in society. Drawing from an existing framework on policy influence, we propose an approach to quantifying the impact of policy influence efforts, specifically within the context of European Public Health (EPH) advocacy. The analysis hinges on the article "Moving from tokenism" which provides a starting point to conceptualize strategies to quantify impact. An exploratory case study approach allowed to integrate literature on advocacy evaluation in parallel with the internal documentation of a EPH advocacy organization We provide recommendations to advocacy organizations that aim to create an infrastructure towards quantifying the impact of their efforts. The framework is mostly tailored to the needs of EPH advocacy, but it can also have resonance beyond the scope of a specific sector.

‘Progression capitals’: How homeless health peer advocacy impacts peer advocates

This article presents analysis from a qualitative evaluation of a homeless health peer advocacy (HHPA) service in London, United Kingdom. Whilst evidence is growing for the impact of peer programming on clients, understanding of the impact on peers themselves is limited in the context of homelessness. Research here is vital for supporting sustainable and effective programmes.Analysis of interview data with 14 current and former peer advocates, 2 members of staff and 3 external stakeholders suggests peer advocacy and its organizational setting can generate social, human, cultural and physical resources to help peer advocates fulfil their own life goals. We explore these with reference to ‘recovery capital’, reframed as ‘progression capitals’ to reflect its relevance for pursuits unrelated to clinical understandings of recovery. Progression capitals can be defined as resources to pursue individually determined goals relating to self-fulfilment.We find engagement with, and benefits from, a peer advocacy service is most feasible among individuals already possessing some ‘progression capital’. We discuss the value of progression capitals for peers alongside the implications of the role being unsalaried within a neoliberal political economy, and comment on the value that the progression capitals framework offers for the development and assessment of peer interventions more broadly.

Human Rights Advocacy Evaluation in the Global South: A Critical Review of the Literature

Advocacy evaluation has emerged in the past 20 years as a specialized area of evaluation practice. We offer a review of existing peer-reviewed literature and draw attention to the scarcity of scholarly work on human rights advocacy evaluation in the Global South. The lack of published material in this area is concerning, given the urgent need for human rights advocacy in the Global South and the difficulties of conducting advocacy in contexts in which fundamental human rights are often poorly protected. Based on the review of the literature and our professional experiences in human rights advocacy evaluation in the Global South, we identify themes in the literature that are especially salient in the Global South and warrant more attention. We also offer critical reflections on content areas not addressed in the existing literature and conclude with suggestions as to how activists, evaluators, and other stakeholders can contribute to the development of a field of practice that is responsive to the global challenge of advocacy evaluation.

A radical arc in systematically documenting political context in advocacy evaluation

AbstractTen major global forces are fixed within the current political context: racial capitalism, neo‐slavery, neoliberalism, white supremacy, neofascism, neocolonialism, neo‐feudalism, imperialism, corporatism, and radical imagination. These forces have been underexamined in the current advocacy evaluation practice, creating missed opportunities for evaluation as a tool for liberation. Attending to these forces and the logics underpinning them should be a new arc within the advocacy evaluation field. Expanded evaluator skills, tools, and methodologies are needed to fully assess advocacy theories of change, campaign activities, and their results in context vis‐à‐vis these 10 global forces. Several regressive field‐level processes inhibiting radical portrayals of the political context and the creation and use of liberatory metrics in advocacy evaluation are discussed.

Pushing boundaries: Advocacy evaluation and real‐time learning in an HIV prevention research advocacy coalition in sub‐Saharan Africa

AbstractEvaluation processes that facilitate learning among advocates must be nimble, creative, and meaningful while transcending putative performance and accountability management. This article describes the experience, lessons, and trajectory of one such approach, Simple, Participatory Assessment of Real Change (SPARC), that a transnational HIV prevention research advocacy coalition pilot‐tested in sub‐Saharan Africa. Inspired by the pioneering work of the outcome harvesting (OH) and participatory evaluation community, we recuperate advocates' centrality as storytellers, sense‐makers, and strategists in advocacy evaluation and describe how we recalibrated SPARC to meet their evaluation and learning needs. This article highlights the normative value of deliberative discourse in evaluation as it contributes to the interpretation of OH and the enrichment of the theory and practice of advocacy evaluation.

A learning agenda for the advocacy evaluation field's future

AbstractThis volume set out to document, illustrate, and critique the progress and innovation that has occurred during the advocacy evaluation field's first phase of development. This final chapter identifies how the context in which advocacy evaluation plays out is shifting. It describes how these shifts impact how advocates, advocacy funders, and advocacy evaluators think about what works, and what has value. Given this context, and lifting up the ideas of other chapter authors, the chapter concludes with a learning agenda for the field's next phase of development—four questions to help guide future field innovation and collective learning.

Radical rerouting: New roads for advocacy evaluation

AbstractThis article discusses the evolution of evaluation of advocacy and its alignment with different parties’ interests in rigor, participation, and usefulness. We then explore responses to our paper, No Royal Road: Finding and Following the Natural Pathways of Advocacy Evaluation, which sought to reframe advocacy evaluation concepts and analytic frames in ways that take into account the complex contexts in which advocacy takes place. We suggest that a narrow conception of power is acting as a fundamental barrier to evaluation approaches that enable and support transformative advocacy. The article concludes with an argument to fully embrace complexity and power issues, noting that this points to radically different implications for future evaluation practices. This argument is, in many ways, anti‐paradigmatic to the current evaluation field, but it draws on a thread of thinking going back to the pioneers in the field.

Evaluating power building: Concepts and considerations for advocacy evaluators

AbstractPower is a fundamental dimension of social change that evaluators regularly overlook. To evaluate power effectively, the evaluation field needs a necessary reorientation in understandings of social change goals, outcomes, strategies, and actors. A focus on power is a missing component in advocacy evaluation and represents a paradigm shift for advocacy evaluators. This article explores the theoretical contributions and methodological considerations of evaluating power, drawing from our experience evaluating nine community organizations' power‐building work focused on economic justice policy reforms. We conclude that four considerations can reorient evaluations to the role of power: (1) grounding power evaluations in equitable evaluation, (2) expanding the scope of evaluations beyond a focus on policy wins to examine individual and collective liberation, (3) incorporating frameworks that acknowledge and assess the iterative and cyclical nature of power building, and (4) clarifying the unit of analysis to consider how a wide array of actors build and wield power.

An introduction to policy advocacy evaluation: The concepts, history, and literature of the field

AbstractThe development of a field of practice for policy advocacy evaluation has been relatively recent within the evaluation community. Given various unique elements, policy advocacy necessitates an adaptation of contextual understanding, methods, and use elements for evaluation practitioners. The authors of this article describe the individual elements that define the space and discuss the origins and development of advocacy evaluation within the evaluation community. The authors then review a broad (though not comprehensive) swath of the existing literature on the topic.

The art and science of developmental and advocacy evaluation

AbstractOver the past 15 years, the field of advocacy and policy change (APC) evaluation has evolved considerably in terms of refining the purpose and methods of this type of evaluation. At the same time, the dynamic nature of APC work requires evaluators to draw upon a range of evaluation approaches, including systems thinking, developmental evaluation, and participatory evaluation, among others. Developmental evaluation is particularly well‐suited for an APC context given its focus on documenting complexity and establishing frequent feedback loops. Previous research has highlighted the value of APC and developmental evaluation approaches and the methods that can be used for each one (i.e., the science of these types of evaluations). However, less guidance exists on how evaluators can navigate the dance between APC and developmental evaluation methods and the mindset needed to simultaneously draw upon the strengths of each one (i.e., the art of doing APC and developmental evaluations simultaneously). This chapter fills that gap by exploring the interplay between APC and developmental evaluation through the lens of one case study. In 2017, Engage R+D embarked on an evaluation of an initiative—funded by The California Endowment—to defend and protect the health and well‐being of Californians, particularly immigrants and other vulnerable populations. The chapter includes an overview of this case example; a description of how we applied developmental evaluation principles; considerations for methods, data collection, and analysis; and a description of how we approached utilization of evaluative information. It concludes with considerations for the field.

Book Review: Advocacy and Policy Change Evaluation: Theory and Practice

Book Review: Advocacy and Policy Change Evaluation: Theory and Practice

An evaluation of the Victorian Salt Reduction Partnership\u2019s advocacy strategy for policy change

BackgroundPublic health advocacy strategies facilitate policy change by bringing key health issues to the forefront of public and political discourse, influencing decision-makers and public opinion, and increasing policy demand. The Victorian Salt Reduction Partnership (VSRP) was established in 2014 in response to inadequate government action to improve population diets in Australia. This study aimed to evaluate the success of the VSRP’s advocacy strategy in achieving policy change.MethodsDocumentation of VSRP activities and outputs were collected, and semi-structured interviews conducted as part of a comprehensive process evaluation. For this study, the Kotter Plus 10-step public health advocacy evaluation framework was used to guide data extraction, analysis, and synthesis.ResultsA sense of urgency for salt reduction was generated by producing evidence and outlining the potential impact of a state-based salt reduction programme. This enabled the creation of a coalition with diverse skills and expertise, which facilitated the development of an innovative and collaborative advocacy action plan. A clear change vision was established, but communication of the vision to decision-makers was lacking, which reduced the impact of the programme as decision-makers were not provided with a clear incentive for policy change. As a result, while programme outputs were achieved, these did not translate to achieving broader strategic goals during a limited-term intervention in a political climate unconcerned with salt.ConclusionsThe Kotter Plus 10-step framework was a useful tool for evaluating the success of the VSRP advocacy strategy. The framework enabled the identification of key strengths, including the creation of the guiding coalition, and areas where efforts could be improved in future similar strategies, such as effective communication within partnerships and to decision-makers, to better influence policy and improve public health impact.

Advocate and Survivor Perspectives on the Role of Technology in Help Seeking and Services with Emerging Adults in Higher Education.

Emerging adults, aged 18–25, have come of age in a technology oriented world. The internet has been critical in mediating their personal relationships and their understanding of daily life. Emerging adults are also at unique risk of experiencing intimate partner and sexual violence (IPV & SV) Given the increasing infusion of information communication technology (ICT) into anti-violence advocacy, and the broad use of ICT among college-attending emerging adults, this study aimed to explore how both survivors and advocates are leveraging technology for support. Using a QUAL + qual methodology (Morse and Niehaus, 2009), data were collected as part of an evaluation of campus-based advocacy as implemented in five programs. Interviews took place with 23 campus and community-based advocates, and 25 survivors of interpersonal violence who had accessed campus-based advocacy services. Additionally, 63 survivors who engaged in campus-based advocacy services responded to an online survey. Key domains identified were: 1) technology as a means of informing potential clients about services; 2) the role of technology in help-seeking, including its role in tailoring and extending the reach of services; and 3) the importance of recognizing technology facilitated abuse in the advocacy and education process with emerging adults. As advocacy programs are rapidly shifting to technology facilitated services in the wake of COVID-19, this study provides data on advocate and survivor experiences with technology, which can inform these changes across the spectrum of IPV & SV services.

Teaching Advocacy Through Community-Based Service Learning: A Scoping Review.

Advocacy has been identified as a core element within the practice of medicine and thus a key component to medical education. However, there are challenges regarding teaching and evaluation of advocacy within medical education. Community-based service learning (CBSL) has emerged as a valuable educational tool to foster knowledge and skills related to advocacy. CBSL is particularly relevant to psychiatry, given the extent of engagement with underserved communities and opportunities to advance learning in these environments. A scoping review was conducted to identify current educational strategies and outcomes related to advocacy training among medical learners in the context of CBSL. Between July and October 2019, the authors searched PsycINFO, MEDLINE, Embase, ERIC, Web of Science, Scopus, and ProQuest for English language literature with no date limits and retrieved 2,813 articles and abstracts; 68 were included in this review. Two reviewers independently screened articles and extracted data. Data were then charted, analyzed, and discussed with the research team. Seven key themes related to approaches to advocacy education were identified: (1) type of community partner; (2) populations served; (3) program participants; (4) program structure; (5) evaluation of learner outcomes; (6) sustainability; and (7) challenges and limitations. This scoping review provides insights into the variety of CBSL-based advocacy program formats and evaluation methods, which is of particular importance to psychiatry. There is heterogeneity in the methodology by which CBSL is implemented and how outcomes are measured. A list of recommendations for future areas of inquiry is provided.

Advocacy Outcomes Are Not Self-Evident: The Quest for Outcome Identification

Advocacy outcomes are not self-evident. Identifying advocacy outcomes is extremely difficult because they are often intangible, arising from (personal) interactions, and they are not always traceable. This challenges conventional evaluation methods, which is recognized in the advocacy evaluation literature. However, current evaluation methods claim to do justice to these complexities while in reality, these methods assume outcomes are identified logically following from actions. Based on empirical findings from a multisited ethnographic study of an advocacy evaluation, this article questions these underlying assumptions and empirically demonstrates how advocacy outcomes are socially and politically constructed, leaving room for interpretation.

Short- and Long-Term Effects of a Tobacco-Free Executive Order on Employee Tobacco Use.

Smoke-free workplace policies encourage cessation, reduce tobacco consumption, and shift the pro-tobacco norm. However, no research exists evaluating the impact of mandated tobacco-free policies on government property. The purpose of our study was to examine short- and long-term effects of a tobacco-free policy (executive order 2014-747) implemented in November 2014, prohibiting tobacco use on state executive property. Cross-sectional online surveys were administered at two time points to a total of 27,000 employees of the executive branch of the Commonwealth of Kentucky. The short-term evaluation (March 2015) comprised 4,170 employees and the long term (August 2015) included 3,070. Tobacco use, plans to quit using tobacco, personal characteristics, whether the county of their workplace was covered by a smoke-free policy, and social norms for tobacco use were assessed 4- and 9-month post-policy implementation. Current tobacco use and plans to quit were compared between short- and long-term evaluations using multiple logistic regression with relevant covariates included. Controlling for demographics and employment location, employees reported lower rates of tobacco use and higher rates of planning to quit in the long term than in the short term. Tobacco-free policies reduce tobacco use prevalence and promote plans to quit, particularly over time. We found differences in tobacco use prevalence and plans to quit using tobacco products from 4 to 9 months after the policy took effect, as reported by employees following implementation of the tobacco-free policy. These findings support the potential for avoiding long-term health care costs as a result of reduced tobacco use from these policies. Nurses can play an important advocacy and policy evaluation role to promote and assess the impact of tobacco-free policies.

Narrative Assessment: A new approach to evaluation of advocacy for development.

This article proposes an approach to monitoring and evaluating advocacy that integrates theory of change and storytelling. This approach, called Narrative Assessment, addresses the feasibility of objectivity and evidence in the complex context of advocacy and proposes an evaluation methodology rooted in alternative conceptualizations of rigour and of evaluator roles. The approach centres on practical judgment and the construction and examination of stories through interaction between advocates and monitoring and evaluation specialists. The article discusses how Narrative Assessment can be useful in the evaluation of advocacy in terms of (1) monitoring and evaluation specialists’ orientation toward programmes; (2) the interpretation of outcomes; (3) the assessment of outcome relevance; (4) reflection and learning and (5) the communication of programme results. The approach builds on lessons drawn from evaluation of eight advocacy programmes in international development.

Negotiating effectiveness in transnational advocacy evaluation.

International development organizations increasingly use advocacy as a strategy to pursue effectiveness. However, establishing the effectiveness of advocacy is problematic and dependent on the interpretations of the stakeholders involved, as well as the interactions between them. This article challenges the idea of objective and rational evaluation, showing that advocacy evaluation is an inherently political process in which space for interactions around methods, processes and results defines how effectiveness is interpreted, measured and presented. In addition, this article demonstrates how this space for interaction contributes to the quality and accuracy of evaluating advocacy effectiveness by providing room to explore and address the multiplicities of meaning around identifying, measuring and presenting outcomes.