Evaluation Of Best Practices Research Articles

Response to paper “Systems thinking” by D. Cabrera et al.: Is it systems thinking or just good practice in evaluation?

Response to paper “Systems thinking” by D. Cabrera et al.: Is it systems thinking or just good practice in evaluation?

Pure dead brilliant?

PurposeThe purpose of this paper is to present an evaluation of The Glasgow Story (TGS) digitisation project, funded by the UK's National Lottery's New Opportunities Fund digitisation (NOF‐Digi) programme, and a critique of the evaluation process itself. The paper emphasises the need for user impact evaluation and for results to be brought into the public domain in order to substantiate the claimed benefits of digitisation projects and programmes and inform ongoing digitisation activity. By critiquing the evaluation methods used the paper also hopes to contribute to the development of good practice in evaluation methodology.Design/methodology/approachQuestionnaires, focus groups, data logs, online surveys and feedback forms were used to gather user responses and make impact assessments.FindingsThe paper suggests that whilst the evaluation can point to some positive impacts that justify the project's innovative approach, practical constraints on the evaluation and methodological flaws ultimately limit the value of the results. The paper concludes that effective evaluation of digitisation needs to extend beyond individual projects, or at the very least, employ generic evaluation tools that facilitate comparison between different projects and approaches.Originality/valueFew digitisation projects attempt to assess their impact and fewer still make their results available. As one of the larger NOF‐Digi projects, the results from the TGS evaluation provide a unique window on one of the major digitisation initiatives in recent years.

Conservation and sustainable use of crop wild relatives

Conservation of crop wild relatives (CWRs) is a complex interdisciplinary process that is being addressed by various national and international initiatives, including two Global Environment Facility projects (‘ In situ Conservation of Crop Wild Relatives through Enhanced Information Management and Field Application’ and ‘Design, Testing and Evaluation of Best Practices for in situ Conservation of Economically Important Wild Species’), the European Community-funded project ‘European Crop Wild Relative Diversity Assessment and Conservation Forum (PGR Forum)’ and the European ‘ In situ and On Farm Network’. The key issues that have arisen are: (1) the definition of what constitutes a CWR, (2) the need for national and regional information systems and a global system, (3) development and application of priority-determining mechanisms, (4) the incorporation of the conservation of CWRs into existing national, regional and international PGR programmes, (5) assessment of the effectiveness of conservation actions, (6) awareness of the importance of CWRs in agricultural development at local, national and international levels both for the scientific and lay communities and (7) policy development and legal framework. The above issues are illustrated by work on the conservation of a group of legumes known as grasspea chicklings, vetchlings, and horticultural ornamental peas ( Lathyrus spp.) in their European and Mediterranean centre of diversity.

The Role of Evaluation in Democracy: Can it be Strengthened by Evaluation Standards? A European Perspective

Evaluation should make a contribution to the further development of politics and society. Evaluation is itself a productive factor in democratic societies and is obliged to function according to its goals and rules. The general theme around this paper is to discuss possibilities to enhance the contribution that evaluation can make to foster democratic governance. I will address a very small aspect of this big issue. My perspective will be on evaluation as a profession and a community which is on the way to establishing its position in modern democracies, supported by making explicit its own standards for good evaluation practice.

Combining IM and Vendor-based Chat: A Report from the Frontlines of an Integrated Service

Driven by studies showing that Millennials (ages 18–24) have adopted IM (instant messaging) as a standard communications medium, many libraries are reaching out to their own patrons in this way. At the same time, some of these libraries have invested thousands of dollars in vendor-based chat reference solutions that provide duplicate functionality. This article assesses a pilot project that offered both IM and vendor-based chat reference simultaneously to users. Impact on workflow, staff stress, training, and other management issues are highlighted, as well as evidence of user preferences and an evaluation of best practices for offering multi-service chat reference.

Evaluation – The Cinderella Science of ICT in Health

Information systems are expensive to create, and consume resources in their operation. They are justified in order to achieve clearly defined objectives in health service quality and efficiency. To ensure that these objectives are being met, and in order to build up an evidence-base to support well focused policies, systems, and practice, evaluation would seem to be fully justified and indeed essential. However, there are major factors jeopardising good evaluation practice, ranging from reluctance to commit resources or expose decisions to scrutiny, through to lack of appreciation of the full range of dimensions which should be studied. This paper seeks to create fuller appreciation of the issues and their importance. Experiential and literature reviews form the basis of an exposition of principles, methods, and key current activities. A summary has been produced of the key challenges to health ICT evaluation, and its adverse comparison with other sectors. An explanation of the viewpoints and levels of health information evaluation is followed by a short history of principal milestones, current scientific methodology activity, and key collaborative activities. The need for a stronger commitment to health ICT evaluation is demonstrated if the application of health informatics systems is to receive due recognition as scientifically and empirically justified. Commitment of resources and scientific endeavour are needed, and thus the recent Declaration of Innsbruck, arising out of an event sponsored by the European Science Foundation, is timely and fully justified.

Comparative evaluation of practices: lessons from R&D organizations

PurposeTo highlight a few critical issues related to indicators that are used to compare organizational practices.Design/methodology/approachThe approach is as follows: to compare the efficacy of qualitative and quantitative indicators; objectivity in identification of processes and practices; and make a distinction between essence and logistics of benchmarking.FindingsFor any benchmarking exercise the most important job is objectivity in identification of processes and practices. The distinction between the two is not always apparent. Qualitative evaluation of practices is as important as quantitative evaluation. The quantitative performance indicators for evaluation of best practices come only at the end of a series of critical qualitative analysis of the organizational processes and practices.Research limitations/implicationsThe subject of the paper has been the enhancement of the effectiveness of R&D organizations. Such organizations form an important part of the innovation system of a nation, generally known as national innovation system (NIS). NISs of more successful countries are being continuously evaluated and emulated by other countries for attaining respective technological aspirations. This leaves a wide scope for application of benchmarking methodology. The dynamics of networks like NIS being different from that of a business organization, the methodology for best practices has to be refined. The present paper has addressed only the R&D organization part of NIS. Much of the validity of the conclusions will depend upon studies on NIS in particular and organizations not driven by profit motive in general.Practical implicationsThe exact definition of the organization in terms of its mandate, long‐term vision and source of competence is the critical task. This is an important lesson for extending benchmarking exercise beyond business enterprises. The study envisages that the methodology would have wide scope of application for organizational restructuring of R&D organizations. However, methodology has to be refined depending on the specificity of the problem of R&D organizations.Originality/valueIt is in presenting the comparative efficacy of qualitative and quantitative indicators for R&D organizations and also in highlighting the importance of benchmarking, the essence of organizational practices, to arrive at the logistics.

A Critical Appraisal of Literature Reviews About the Transition to Adulthood for Youth with Disabilities

A critical appraisal of five review articles on the transition to adulthood for youth with disabilities was conducted to identify evidence about (1) the factors that help or hinder the transition process, and (2) “what's working” in transition services. The appraisal identified a number of important “success” factors and elements of service delivery that are worthy of consideration by service providers and researchers. These include the need for skill development of youth with disabilities, environmental supports, and an individualized approach to service delivery. All of the reviews identified the need for more evidence to support the implementation and evaluation of best practice models/approaches that address the complex issue of the transition from paediatric to adult services for youth with disabilities.

Managing customer satisfaction through efficient listening tools: an evaluation of best practice in four world-class companies

Measuring customer satisfaction leads to identifying ways to improve product/service quality, which in turn leads to increasing the company's competitive advantage. This best-practice study examines four world-class companies, two of which are multinational corporations and the others are two of the largest companies in the UK. One of the objectives of this study is to examine how best-practice companies use various listening tools to obtain information about customers' needs, preferences, and perceptions of their performance. Another is to identify some best practices, which are salient activities for managing customer perception and satisfaction. The primary contribution of this article is its discussion of a variety of different customer listening tools used by firms and the creation of both a fitness framework and an implementation framework.

Local Public Health Performance Measurement

This case study summarizes the development and implementation of the evaluation framework, preliminary data results, and lessons learned for overcoming barriers and challenges. The framework development was based on best practices in evaluation and the Public Health Essential Services. Results were as follows: (1) implemented a department-wide framework for program evaluation; (2) developed and implemented 34 program evaluation plans; (3) collected data on 52 indicators across 492 measurable objectives; and (4) identified challenges and recommendations to improve evaluation processes. Although the importance of program evaluation is recognized in public health, program evaluation models do not take into account the variety of goals inherent in the diversity of public health programming. The case study presented provides guidance and lessons learned for the evaluation of public health practice in local governments.

Good Evaluation Practice: a proposal of guidelines

The intention of this proposal is to describe principles for Good Evaluation Practice (GEP) of toxicological and risk evaluations with the aim to improve the quality, reliability, and transparency of any evaluation in toxicology. The GEP has been outlined in the same way as the principles for Good Laboratory Practice (GLP) to facilitate its introduction and compliance. The main characteristic of GEP is transparency. The GEP procedure should include the following elements: (1) definition of terms; (2) description of evaluation organization; (3) description of the quality assurance of the programme; (4) description of the standard evaluation procedures; (5) performance of the evaluation; (6) reporting of the evaluation; (7) storage and retention of records and material; and (8) guidelines for the evaluation procedure. The authors’ intent is to raise awareness, stimulate a debate about the proposal, expect and welcome feedback in order to refine it, and make it into an important tool for all toxicological evaluations in the future.

Good Evaluation Practice: a proposal of guidelines

Good Evaluation Practice: a proposal of guidelines

The Rigidity and Comfort of Habits: A Cultural and Philosophical Analysis of the Ups and Downs of Mainstreaming Evaluation

AbstractMainstreaming evaluation requires establishing aesthetic and ethical frameworks, as well as developing knowledge and skills that make “doing evaluation” seem like the right thing. Mainstreaming, however, can pose challenges to good evaluation practice; a view from cultural anthropology suggests that evaluation can have distinct meanings in different settings. Stories from program evaluation research in Indonesia illustrate the ways comforts and rigidities associated with mainstreamed evaluation processes may hinder high‐quality evaluation.

Evaluating the athlete’s perception of the sport psychologist’s effectiveness: What should we be assessing?

Abstract Objective: To extend understanding of factors pertinent to the effective practice of applied sport psychologists in order to influence good evaluation practice. Method: Semi-structured interviews with 30 elite United Kingdom (UK) athletes (mean age=22.7±8.2) from a range of sports about their experiences of sport psychology. The interview transcripts were inductively content analyzed and appropriate trustworthiness criteria were incorporated. Results: A number of themes emerged related to the characteristics (e.g. good communicator, knowledgeable about the sport) and activities (e.g. provides feedback, uses appropriate formats) of effective practitioners. These findings support and extend previous research on consultant effectiveness and highlight areas in which to develop current evaluation practice. Conclusion: This study has extended knowledge of consultant effectiveness, with specific relevance to elite UK athletes, and highlights the need for further development of evaluation instruments to assist practitioners in functionally evaluating the effectiveness of pertinent aspects of their practice.

An evidence-based approach to drainage of the pleural cavity: evaluation of best practice.

Recent surveys have uncovered major variations in key aspects of intercostal drain management, suggesting that decisions are being made on individual preferences without resorting to sound evidence. We provide an up-to-date review of the best practice with evidence-based recommendations and expert consensus views. The following aspects of chest drain management have been addressed: indications for drainage, insertion technique, complications, management of an indwelling chest drain, indications and technique for removal. The emphasis in this review is that safe intercostal drain practice relies upon adherence to a few important principles. Furthermore, when in doubt, particularly with a complex thoracic problem, one should seek prompt specialist advice.

Consulting Parents to Improve a Child Guidance Service

This paper describes a study undertaken with a group of parents whose children had been referred to the Knowsley Child Guidance Service. A qualitative methodology was used to analyse in-depth interviews with parents. The results led to practical recommendations for the improvement of service delivery. These recommendations had implications for schools and other services involved with parents/carers and children. The survey also provided a base from which to develop a questionnaire for the continuing evaluation of good practice within the child guidance service.

The Metaevaluation Imperative

The evaluation field has advanced sufficiently in its methodology and public service that evaluators can and should subject their evaluations to systematic metaevaluation. Metaevaluation is the process of delineating, obtaining, and applying descriptive information and judgmental information about an evaluation’s utility, feasibility, propriety, and accuracy and its systematic nature, competence, integrity/honesty, respectfulness, and social responsibility to guide the evaluation and publicly report its strengths and weaknesses. Formative metaevaluations—employed in undertaking and conducting evaluations—assist evaluators to plan, conduct, improve, interpret, and report their evaluation studies. Summative metaevaluations—conducted following an evaluation—help audiences see an evaluation’s strengths and weaknesses, and judge its merit and worth. Metaevaluations are in public, professional, and institutional interests to assure that evaluations provide sound findings and conclusions; that evaluation practices continue to improve; and that institutions administer efficient, effective evaluation systems. Professional evaluators are increasingly taking their metaevaluation responsibilities seriously but need additional tools and procedures to apply their standards and principles of good evaluation practice.

Can We Put Precision into Practice? Commentary and Thoughts Engendered by Abrami's “Improving Judgments About Teaching Effectiveness Using Teacher Rating Forms”

AbstractEvaluation researchers and practitioners agree that although measurement issues remain important to good evaluation practice, a greater area of concern is unreliable or invalid day‐to‐day practice. Can the precision of statistical techniques be brought to bear on this problem? This chapter reviews and summarizes responses to Abrami's proposal.

The metaevaluation imperative

The evaluation field has advanced sufficiently in its methodology and public service that evaluators can and should subject their evaluations to systematic metaevaluation. Metaevaluation is the process of delineating, obtaining, and applying descriptive information and judgmental information about an evaluation’s utility, feasibility, propriety, and accuracy and its systematic nature, competence, integrity/honesty, respectfulness, and social responsibility to guide the evaluation and publicly report its strengths and weaknesses. Formative metaevaluations—employed in undertaking and conducting evaluations—assist evaluators to plan, conduct, improve, interpret, and report their evaluation studies. Summative metaevaluations—conducted following an evaluation—help audiences see an evaluation’s strengths and weaknesses, and judge its merit and worth. Metaevaluations are in public, professional, and institutional interests to assure that evaluations provide sound findings and conclusions; that evaluation practices continue to improve; and that institutions administer efficient, effective evaluation systems. Professional evaluators are increasingly taking their metaevaluation responsibilities seriously but need additional tools and procedures to apply their standards and principles of good evaluation practice.

Innovation in the every day life of libraries

A state of the Art trend analysis of the Library and Information science reveals that the Library world is in the midst of the technological restructuring. Every branch of library work changes continuously because of the technological facilities, although the missions and goals remain unaltered. The libraries belong to these professional divisions which involve the innovation and technology transfer in everyday life. In most of the cases innovation either as a product or as a procedure goes into the libraries, it's not a phenomenon produced by them. On the strategic level libraries on average have developed the ability to collect, process and assimilate technological innovation. On the organizational level libraries follow the changes which technology transfer dictates such as close co-operation with computer centers and external commercial information suppliers, software and hardware suppliers, science and technology achievements, etc. It is too common to libraries to develop routines and Best Practices for every task, derived by the technology they use, for the cost decrease, the time save and the increase of the effectiveness and efficiency. The innovative process is the kind which libraries could adopt, as innovation is composed of the systematic retrieve of changes and systematic analysis of the opportunities which these changes could contribute to economic and social improvement. Innovation above all, is a social procedure, as it is interpreted in terms such as creativity, desires, fantasy and the ability to take risks and to change. So innovation and technology transfer differentiates the role of the librarian and extends it beyond the limits of the information intermediary. The librarian in order to involve, anticipate and especially produce innovation has to learn how to learn, as libraries have to learn how to change. The presentation will be a methodological one and will focus on Best Practices, the definition and a short report, how they combine with the library functions and how they produce innovative processes and new Best Practices. An evaluation of Best Practices according the Value Analysis method is included as a prerequisite of the final choice of which practice is characterised as Best and which not.