Utilization Of Evaluation Results Research Articles

Organizational Leadership Styles and Utilization of Evaluation Results

The greatest concern in programme management, given the potential of programme or project evaluations to inform decision-making, remains how to ensure the results of an evaluation are used for the intended purpose and how information from an evaluation process and/or the products are applied to practices and decision-making processes. This article presents a research that sought to explore the degree to which organizational leadership style impacts the use of evaluation findings for programme improvement. Based on the pragmatic paradigm and informed by the Utilization Focused Evaluation Model and Knowledge Use Theory, the research adopted a descriptive and correlational design using mixed methods. The study’s sample comprised of 232 project staff from Non-Governmental Organizations (NGOs) based in Kisumu Central Sub-County, Kenya. Content analysis method that involved coding and identifying themes from respondents’ responses was used to analyse the qualitative data obtained from key informant interviews. Both percentages and frequencies were used to summarize the quantitative data. The instrument reliability was assessed through pilot testing and Cronbach alpha of 0.908 attained. A hypotheses H01: Leadership style does not significantly influence utilization of evaluation results was tested at α = .05 level of significance, but it was rejected because P = 0.000 < 0.05. The study established a statistically significant relationship between organizational leadership style and the utilization of evaluation results. In so doing, it reinforces the existing literature by helping to understand the way organizational leadership style influences the utilization of evaluation results. It fills a gap in the literature, thereby contributes to the appreciation of the factors that enhance and predict the utilization of evaluation results.

ORGANIZATIONAL CAPACITY AND UTILIZATION OF EVALUATION RESULTS

A major concern in programme management, given the potential of programme or project evaluations to inform decision-making, has been how to ensure the results or findings of an evaluation are used for the intended purpose. How information from an evaluation process and/or the products are used in practice and decision making is equally a concern for evaluators. This article presents a research that sought to explore the extent to which organizational capacity impacts the utilization of evaluation findings for programme improvement. Guided by the pragmatic paradigm and informed by the Utilization Focused Evaluation Model and Knowledge Use Theory, the research adopted a descriptive and correlational design using mixed methods. The sample size for this study was 232 project staff from Non-Governmental Organizations (NGOs) in Kisumu Central Sub-County, Kenya. To analyze qualitative data, the open ended responses from key informant interviews were recorded and coded appropriately for further analysis for themes through content analysis and comparative analysis. Frequencies and percentages were calculated to describe the basic characteristics of the quantitative data. To ensure validity and reliability of the research instruments, pilot testing was conducted. Cronbach’s alpha at α = 0.908 was attained as the reliability coefficient of the pre-test instruments. Tests of statistical assumptions were carried out before data analysis to avoid invalidation. A hypotheses H01: Organizational capacity does not significantly influence utilization of evaluation results was tested at α = .05 level of significance and was rejected since P = 0.000 < 0.05. The study findings demonstrate that there is a significant relationship between organizational capacity and the utilization of evaluation results. This research therefore reinforces literature and helps to understand the ways in which organizational capacity influences the utilization of evaluation results. It thus informs the evaluation field of study, fills gaps in the evaluation use literature and contributes to the appreciation of factors that predict and enhance the utilization of evaluation results.

Utilization of Evaluation Results in Policy-Making Aad Administration – A Challenge to Political Science Research

The paper deals with utilization of evaluation results in policy-making and public administration. In the pursuit of this guiding question, the article is written in five steps. First, major variants of evaluation are presented with a particular eye to their respective utilization potential. Second, different concepts are sketched that have been forwarded to capture the utilization of (social) science-generated knowledge by political, administrative, and social actors. Third, looking at Germany, Switzerland and the European Union as cases in point pertinent research findings are discussed. Parts four and five contain summary, conclusions and some remarks on ensuing research needs. The political rationality and the underlying political will of the decision-makers prevail in crucial political decisions while in less important decisions evaluation-generated evidence does show some effect and a dose of scientific rationality comes to the fore. However, the stock of evaluation-generated information and knowledge about the effects of policies and measures has seen an exponential growth, which makes the call for evidence-based policy-making and for utilizing the available evaluation findings all the more mandatory and urgent.

EVALUASI RENCANA KERJA SATUAN KERJA PERANGKAT DAERAHBADAN PERENCANAAN PEMBANGUNAN DAERAH KOTA SEMARANG

Departing from the problems of weak evaluation planning at local government level work unit, we conducted this study with the aim to get an overview of the evaluation work plan on working units, and the formulation of solving the problem and provide solutions. This research was conducted with a qualitative approach, using in-depth interviews and documentation of data, and then analyzed with domain analysis techniques.The results showed evaluation work plan in Bappeda still not effective. Bappeda only to evaluate the results of the work plan at this stage, while at the policy and implementation stages have not been carried out. Some of the main problems found in the evaluation work plan Bappeda are no guidelines for internal evaluation work plan, limited information technology support, the limited number and quality planners, and not the good aspects of the legality of the work plan.Recommendations for addressing concerns in Bappeda evaluation work plan are: strengthening the commitment of the planning; the establishment of guidelines for the evaluation of the work plan, the data developing regional planning system; to improve coordination, communication and development planning; increased monitoring of the quarter; utilization of evaluation results; and improving the quality and welfare of workers planner.

Assessment of the Needs on Evaluation of Different Schools in the Philippines

The study assessed the evaluation needs of 81 schools nationwide. A survey questionnaire was constructed and validated by measurement experts. Descriptive statistics were obtained to describe the evaluation activities, practices and areas on evaluation needing technical assistance. Findings show that all schools conduct evaluations mostly on teacher performance (N=29.36%). There was an expressed need for technical assistance practically from planning to utilization of evaluation results, especially in areas of instrumentation and data analysis which were rated highly. The t-test results further indicate no significant difference in the needs between public and private schools. Implications and recommendations related to improve effective training programs that were drawn are discussed in the study.

Moderating Effect of Quality of Evaluation on the Relationship between Theory-Driven Evaluation of Structured On the Job Training Program and Utilization of Evaluation Results in Small and Medium-sized Enterprises

Moderating Effect of Quality of Evaluation on the Relationship between Theory-Driven Evaluation of Structured On the Job Training Program and Utilization of Evaluation Results in Small and Medium-sized Enterprises

Teacher evaluation practices in Malaysian primary schools: issues and challenges

This paper describes a qualitative case study that investigated the underlying issues in implementing school-based teacher evaluation practices in four Malaysian primary schools. The participants of the study comprised eight school administrators and sixteen teachers. Data obtained through interviews, observations, and document reviews were analyzed using the constant comparative method. The analysis captured four thematic constructs as key aspects of the school-based teacher evaluation system, namely: policy direction, evaluation methods, instruments, and utilization of evaluation results. The paper recommends revamping of the existing school-based teacher evaluation system along the emergent constructs identified in the study.

Strategies and policies for improving energy efficiency programs: Closing the loop between evaluation and implementation

Program implementers often use evaluation results to improve the performance of their programs, but, as described in this paper, this is not always the case. Based on a review of the literature, participation in workshops, and interviews with over 50 program implementers, evaluators, and regulators in the United States and Canada, the utilization of evaluation results is investigated by asking the following questions: (1) How are program evaluation results used by program implementers and other stakeholders? (2) How are program evaluation results communicated to program implementers and other stakeholders? (3) Are the needs of program implementers being met by program evaluation? (4) What is the role of the utility regulator in facilitating the use of program evaluation results? (5) What other mechanisms can facilitate the use of program evaluation results? While there is some consensus on the answers to these questions, the type of interest in and use of evaluation varies by functional role (e.g., evaluator versus implementer), maturity of the energy efficiency market, institutional context (e.g., evaluation and implementation conducted inside the same organization, or evaluation and implementation conducted by separate entities), and by regulatory demands and evaluation interests.

Investigating the Impact of Gifted Education Evaluation at State and Local Levels: Problems with Traction

This article highlights results from 2 gifted program evaluations and a follow-up case study investigating how evaluation data were used. Although program evaluation and evaluation utilization are critical issues in driving gifted program development, literature that shares the results of programs with external evaluations is scant. Common findings across the 2 studies were the absence of data on student learning, the fragmentation of service models across the K–12 expanse, the lack of systemic staff-development strategies, and the lack of meaningful parent involvement. Utilization of evaluation results were mixed. Although respondents cited evaluation as a consciousness-raising experience and an important process in effecting change, they were essentially stymied in their efforts to use the evaluation data to catalyze program improvement, particularly in the short run. Barriers to effective use of data are grounded in the insufficient resource base available to sustain high-quality programs and in the complex leadership demands that require movement on multiple fronts simultaneously.

The Utility of the Knowledge Use System in a Higher Education Evaluation Setting

This study uses the Knowledge Use System (KUS) that was developed by Barabba and Zaltman in 1991 to delineate decisionmaker variables and characteristics that may thwart utilization of evaluation results. The KUS was originally developed as a tool enabling corporate market reseatch to overcome "obstacles to implementation." The KUS is a process used to point out decisionmaker biases, preconceived notions and other decisionmaker variables. It facilitates discussion and consensus building about important evaluation issues that, if not recognized, might diminish utilization. The results of this study suggest that the KUS is successfully adaptable to higher education contexts. Its use during evaluation appears to improve the quality of the evaluation and the implementation of results.

On the demise of academic evaluation

The thesis of this paper is that there is a paucity of theory underlying the practice of evaluation, and that this lack of general evaluation theory means that evaluation is unsuitable as an academic specialty. To the extent that academics continue to be involved in evaluation, it is secondary to a commitment to some content area (e.g., education, mental health, methodology, etc.). This plus the increasing sophistication of federal evaluations, making them more like research and less like evaluation, suggest the likelihood of decreasing involvement of academics with evaluation. A reversal of this trend would require the identification of one or more theoretical issues which span the range of evaluation content, and the generation of a body of research about evaluation which would contribute to generalized evaluation theory. Utilization of evaluation results is suggested as one such topic.

On the Organizational Theory of Evaluation

This article uses Wamsley and Zald's discussion of the political economy of public organizations to develop categories of constraints on evaluation in bureaucratic settings. It argues that the relevant literature focuses largely on political factors, in part because this literature concentrates on the utilization of evaluation results. Interviews with federal administrators concerning the establishment of an evaluation unit are used to explore a mix of economic and political factors. The article concludes that program managers choosing an evaluation agent have some latitude to determine the timing and context of different struggles over evaluation.

DESIGNING A TRAINING EVALUATION SYSTEM

This paper calls for improved evaluation of training and development programs, and provides guidelines for effective and useful evaluation. Attention is given to what to evaluate, methods of evaluation and data collection, analysis, and utilization of evaluation results. A comprehensive model is given for designing an evaluation system.

Decision Makers' Judgments

Evaluators often assume that outcome studies assessing agency effectiveness should pro vide the most relevant data for decision makers who must form judgments about treat ment in order to make policy, program, and clinical decisions. Yet evaluators have found that decision makers often fail to use results of evaluation studies. To shed light on the utilization problem, the research reported here was undertaken to learn about the criteria, information sources, and beliefs decision makers in the environment of nine drug treat ment agencies used to form judgments about treatment programs before evaluative data concerning outcomes became available. The results showed that decision makers in different occupations tended to use similar information sources (primarily informal verbal exchanges) yet used different evaluative criteria to judge treatment programs. The impli clations of these results for the design of evaluation studies and utilization of evaluation results are discussed.