Stakeholder Involvement In Evaluation Research Articles

Principles for Stakeholder Involvement in Evaluation in Latin America

In the last three decades, the promotion of stakeholder involvement in evaluation has been gaining relevance in the Latin American and internationally, across varied agencies, institutions, and civic organizations. The 2030 Agenda and the Global Evaluation Agenda have also recognized the centrality of participation in evaluation. This article explores stakeholder involvement in evaluation based on collaborative work with stakeholders from 15 evaluative experiences. It shows what characterizes participatory evaluation in the region today and the principles of this practice.

Stakeholder Involvement in Evaluation: Does it Affect Observers' Perceptions of an Evaluation? And Which Stakeholder Group(s) Do People Think Should to Participate?

Stakeholders are often involved in evaluation, such as in the selection of specific research questions and the interpretation of results. Except for the topic of whether stakeholder involvement increases use, a paucity of research exists to guide practice regarding stakeholders. We address two questions: (1) If a third-party observer knows stakeholders were involved in an evaluation, does that affect the perceived credibility, fairness, and relevance of the evaluation? (2) Among individuals with a possible stake in an evaluation, which stakeholder group(s) do they want to see participate; in particular, do they prefer that multiple stakeholder groups, rather than a single group, participate? Six studies are reported. All studies address the former question, while Studies 3 to 5 also focus on the latter question. To study effects of stakeholder involvement on third-party views, participants read summaries of ostensible evaluations, with stakeholder involvement noted or not. To examine a priori preferences among potential stakeholders, participants completed a survey about alternative stakeholder group involvement in an evaluation in which they would likely have an interest. Across studies, effects of reported stakeholder participation on third-parties' views were not robust; however, small effects on perceived fairness sometimes, but not always, occurred after stakeholder involvement and its rationales had been made salient. All surveys showed a large preference for the involvement of multiple, rather than single stakeholder groups. We discuss implications for research and practice regarding stakeholder involvement, and for research on evaluation more generally.

Involving the public in epidemiological public health research: a qualitative study of public and stakeholder involvement in evaluation of a population-wide natural policy experiment

BackgroundPublic involvement in research is considered good practice by European funders; however, evidence of its research impact is sparse, particularly in relation to large-scale epidemiological research.ObjectivesTo explore what difference public...

The utility of stakeholder involvement in the evaluation of community-based health promotion programmes

Community-based health promotion programmes (CBHPPs) deal with cross cutting issues (social, economic, environmental) and utilise issue-based, population-based and setting-based approaches in programme planning, implementation and evaluation with the aim of empowering individuals and communities to gain control over the determinants of health. This multifaceted nature of CBHPPs has resulted in a number of questions particularly regarding the level of stakeholder involvement in evaluating such programmes. Evaluation is seen as one of the ways to provide evidence to inform health promotion decision-making and practice. Community programme implementers and lay people have lamented that evaluations are often imposed on them without recognition for the uniqueness of their community, its programmes, resources and skills. The aim of this paper is to critically examine the utility of stakeholder involvement in the evaluation of CBHPPs. We first explore the nature of CBHPPs and the methods of gathering evidence via evaluation. This is followed by a critical examination of the value of stakeholder involvement in evaluating CBHPPs. The Ottawa Charter for Health Promotion is used to conceptualise the paper with specific examples of stakeholder involvement in evaluation of CBHPPs used to support our arguments. The paper concludes that stakeholder involvement in evaluation of CBHPPs is of importance considering the empowerment and participation values of health promotion. As evaluation is done with people and not on people, stakeholder involvement should be central to the evaluation of CBHPPs rather than adopting an approach whereby the researcher is completely detached from the programme stakeholders particularly the community.

Evaluation of marine spatial planning

Abstract Evaluation is generally recognised as an essential step for learning and improvement in marine spatial planning (MSP). Practical guidance and experience relative to evaluation in MSP is, however, very limited. To address this gap, this paper addresses a set of issues relevant for evaluation design and performance: the object of evaluation is discussed in relation to the functions of planning, the choice of objectives and the timing of evaluation; factors affecting causality attribution are analysed, namely multi-causality of observed effects, counterfactual design, time lags and uncertainty; and possibilities and requirements regarding stakeholder involvement in evaluation are considered. Six models of planning evaluation are reviewed, based on which a step-wise framework for MSP evaluation is developed. Future research should pilot test and assess this framework and broaden the evidence base relative to impacts of marine management initiatives.

Stakeholder Involvement in Evaluation: Three Decades of the American Journal of Evaluation

Background: Stakeholder involvement in various phases of evaluation has received increasing attention over the past three decades. Indeed, the American Journal of Evaluation (AJE) has reflected this overall philosophy and strategy through a number of publications about several theoretical frameworks and practical applications. Those AJE articles outline the primary assumptions of the stakeholder approaches to evaluation, their practical applications, constraints, and benefits for providing a new direction in evaluation. Purpose: What lessons are there for our field concerning the way that stakeholder involvement has been conceptualized and applied in AJE? This article focuses on the ways in which AJE authors have approached the notion of stakeholder involvement over the past three decades, drawing on key articles from scholarship on this topic. Setting: Not applicable. Intervention: Not applicable. Research Design: Not applicable. Data Collection and Analysis: Desk review. Findings: This review shows how AJE has enhanced our understanding of the evolution of these approaches to evaluation, from a global perspective to differentiated approaches with a shared theme. Keywords: collaborative evaluation; participatory evaluation; empowerment evaluation; stakeholder approaches to evaluation; stakeholder involvement

Macro, Meso, and Micro‐Efficiencies in Water Resources Management: A New Framework Using Water Balance1

Haie, Naim and Andrew A. Keller, 2012. Macro, Meso, and Micro‐Efficiencies in Water Resources Management: A New Framework Using Water Balance. Journal of the American Water Resources Association (JAWRA) 48(2): 235‐243. DOI: 10.1111/j.1752‐1688.2011.00611.xAbstract: One of the most important performance indicators for water resources systems (WRSs) management is efficiency. Here, water balance, based on mass conservation, is utilized to systemically develop three levels of composite efficiency indicators for a WRS, which are configurable based on two types of water totals: total inflow and total consumption (outflow that effectively is not available for reuse). The indices characterize hydrology of an area by including in their formulations the flow dynamics at three integrated levels. Furthermore, the usefulness of water is incorporated into the indicators by defining two weights: one for quality, and the other for beneficial attributes of water use. Usefulness Criterion is the product of quality and beneficial weights, emphasizing the equal significance of the two dimensions. Both of these weights depend on the system itself and the priorities of the supervising organization, which also are shaped by the objectives and values of the given society. These concepts lead to the definition of Macro, Meso, and Micro‐Efficiencies, which form a set of integrated indicators that explicitly promotes stakeholder involvement in evaluation and design of WRSs. Macro, Meso, and Micro‐Efficiencies should be maximized for both water totals, which is an integrated prerequisite for sustainability and is less promoted by competing stakeholders. To demonstrate this new framework, it is applied to published data for urban and agricultural cases and some results are explained.

Organizational Capacity To Do and Use Evaluation: Results of a Pan-Canadian Survey of Evaluators

Abstract: Despite increasing interest in the integration of evaluative inquiry into organizational functions and culture, the availability of empirical research addressing organizational capacity building to do and use evaluation is limited. This exploratory descriptive survey of internal evaluators in Canada asked about evaluation capacity building in the context of organizational characteristics (learning, support structures), evaluative activity and use, and variables that mediate use. We received a total of 340 usable responses to an online survey. This article provides a descriptive account of the findings with a cursory look at differences across respondent role, organization type, and self-reported perceived level of evaluation knowledge. Results showed a pattern of moderately high ratings of organizational learning and support functions, the extent to which evaluation is being conducted and used, and stakeholder involvement in evaluation. Some differences across respondent roles, organization type, and evaluation knowledge were observed. Results are discussed in terms of an agenda for future inquiry.

Evaluation of a Positive Youth Development Program in Hong Kong: Issues, principles and design

Several issues concerning evaluation of positive youth development programs in Hong Kong are outlined. These include the choice of evaluation paradigm, evaluation frameworks and evaluation standards, role of quantitative and qualitative research methods, process vs. outcome evaluation, and choice of objective outcome measures. With reference to the above issues, the evaluation design of a positive youth development program in Hong Kong is presented. Several principles are maintained in the design of the program. These include objective outcome assessment, use of multiple objective outcome indicators, subjective outcome evaluation, involvement of stakeholders in evaluation, assessment of program implementers' views, qualitative evaluation, collection of quantitative and qualitative data, process evaluation, triangulation of data collected from different sources by different methods, use of validated measures, following existing guidelines governing the evaluation of positive youth development programs, and continuous evaluation. Five evaluation strategies, including objective outcome evaluation, subjective outcome evaluation, qualitative evaluation, evaluation based on repertory grid tests, and process evaluation are proposed to evaluate the project.

Stakeholder Involvement in Evaluation: Suggestions for Practice

Essays, opinions, and professional judgments are welcome in this section of EP. Forum articles speak to and about the philosophical and ethical dilemmas of our profession. Authors are invited to express their views to encourage constructive dialogue centered on issues. To keep the dialogue substantive, other articles moti vated by previous Forum presentations should have independent titles and themes. Items labeled "Comments on ..." and "Rejoinder to ..." will not be published inForum— such responses, and those not of article length or nature, are welcome and encouraged in the Letters section of EP. Standard citations and reference lists should acknowledge and identify earlier contributions and viewpoints. Manuscripts should not exceed 10 double-spaced typewritten pages in length.

Stakeholder involvement in evaluation: Suggestions for practice

This paper reports on the evaluation of some of the elements of a multi-component program aimed at improving the accessibility of health and social services for the homeless. The program involved provincial, regional and municipal authorities and more than 20 non-profit and public agencies. In accordance with the overall approach of the program, the principle of stakeholder participation in the evaluation was emphasized by the funding agencies. However, even if the evaluation started under promising conditions, including administrative support, a strong commitment from the involved agencies and experience in formative evaluation within the research team, participation of stakeholders from charitable non-profit agencies met with many difficulties. Using a case study approach, this paper attempts to analyze and discuss how the organizational context of the evaluation, certain stakeholder characteristics and methodological features may have impacted on the evaluation process.