Personnel Evaluation Standards Research Articles

"Best Tradition": CREATE, JCSEE and the Program Evaluation Standards

Background: Evaluation “is a task in the best tradition of the most abstract theoretical science as well as the most practical applied science” (Scriven, 1968, p .9). The Program Evaluation Standards of the Joint Committee on Standards for Educational Evaluation (JCSEE) operationalize the theoretical aspects of evaluation and, when used, facilitate sound evaluation methods in applied settings. Between the publications of the first and second editions of The Program Evaluation Standards, the Center for Research on Educational Accountability and Teacher Evaluation (CREATE) was funded in 1990 at Western Michigan University with federal monies of $5.2 million, and between 1990 and 1995 by the United States Department of Education, Office of Educational Research and Improvement (OERI). CREATE was established for the betterment of evaluation within the educational context (Stufflebeam, 1991; Stufflebeam & Shinkfield, 1994). CREATE’s mandate and subsequent mission furthered the work of the Program Evaluation Standards and the JCSEE by using the standards in applied settings. Keeping to Scriven’s notion of evaluation as the best tradition, the collaborative work between CREATE and JCSEE is a well-established tradition that furthers the development of theoretical aspects of evaluation and the application of the evaluation standards. Purpose: Examine CREATE’s impact on the Program Evaluation Standards’ theoretical development and applied use. Setting: Not applicable. Intervention: Not applicable Research Design: Not applicable. Data Collection and Analysis: Systematic review of the theoretical development and applied use of the Program Evaluation Standards in the books, journal articles, monographs, special papers, meeting minutes, conference programs, and presentations associated with CREATE. Findings: CREATE has contributed to the operationalization of the theoretical aspects of evaluation with the Program Evaluation Standards and facilitated their use in applied settings. CREATE has also furthered the work of the Personnel Evaluation Standards and the Classroom Assessment Standards (formerly the Student Evaluation Standards). Leading scholars from CREATE and the JCSEE have contributed to the standards since the 1990s. Members of CREATE have published a notable range of books, journal articles, monographs, special papers and conference presentations related to the Program Evaluation Standards. Organizational capacity and shared goals of both the JCSEE and CREATE guided the practical application and theoretical development of the Program Evaluation Standards. Keywords: Program Evaluation Standards; Consortium for Research on Educational Assessment and Teaching Effectiveness; Joint Committee for Standards on Educational Evaluation

Research on the Risks and Countermeasures of Talent Introduction in Colleges—Based on the Perspective of Risk Management

Talent introduction in colleges has some problems, such as the talent introduction planning, the difficulties in talent selection, the low level of matching degree, the imbalance of input and output, the brain drain and so on. Colleges need to establish a risk assessment index system for the introduction of their own situation, and scientific assessment of the occurrence of various types of risk probability and the degree of loss, etc. At the same time, establishing scientific personnel by the introduction planning, establishing comprehensive personnel evaluation standards, implementing the contract management, strengthening performance appraisal, perfecting the mechanism of talent management system, establishing a comprehensive preventive measures the risk of introducing talents into colleges.

Contextual adaptation of the Personnel Evaluation Standards for assessing faculty evaluation systems in developing countries: the case of Iran

BackgroundFaculty evaluations can identify needs to be addressed in effective development programs. Generic evaluation models exist, but these require adaptation to a particular context of interest. We report on one approach to such adaptation in the context of medical education in Iran, which is integrated into the delivery and management of healthcare services nationwide.MethodsUsing a triangulation design, interviews with senior faculty leaders were conducted to identify relevant areas for faculty evaluation. We then adapted the published checklist of the Personnel Evaluation Standards to fit the Iranian medical universities' context by considering faculty members' diverse roles. Then the adapted instrument was administered to faculty at twelve medical schools in Iran.ResultsThe interviews revealed poor linkages between existing forms of development and evaluation, imbalance between the faculty work components and evaluated areas, inappropriate feedback and use of information in decision making. The principles of Personnel Evaluation Standards addressed almost all of these concerns and were used to assess the existing faculty evaluation system and also adapted to evaluate the core faculty roles. The survey response rate was 74%. Responses showed that the four principles in all faculty members' roles were met occasionally to frequently. Evaluation of teaching and research had the highest mean scores, while clinical and healthcare services, institutional administration, and self-development had the lowest mean scores. There were statistically significant differences between small medium and large medical schools (p < 0.000).ConclusionThe adapted Personnel Evaluation Standards appears to be valid and applicable for monitoring and continuous improvement of a faculty evaluation system in the context of medical universities in Iran. The approach developed here provides a more balanced assessment of multiple faculty roles, including educational, clinical and healthcare services. In order to address identified deficiencies, the evaluation system should recognize, document, and uniformly reward those activities that are vital to the academic mission. Inclusion of personal developmental concerns in the evaluation discussion is essential for evaluation systems.

Adapting the Utilization-Focused Approach for Teacher Evaluation

Mandated teacher evaluations are of little use to educators for a variety of reasons, including lack of reliability and validity of evaluation instruments and methods, failure of evaluation systems to adhere to the Personnel Evaluation Standards (Joint Committee, 2008), and the inherent bias in using classroom observations as a method of evaluating teachers. As a result, teacher evaluations fail to improve teaching or impact student learning. This paper suggests that the utility, and ultimately the impact, of teacher evaluations can be improved by adapting the utilization-focused approach to teacher evaluations. Specific suggestions for implementing the Utilization-focused Evaluation Checklist (Patton, 2008) are also provided.

Highlights of the Consortium for Research on Educational Accountability and Teacher Evaluation (CREATE): Expanding Research and Promising Practices in Personnel, Program, and Student Evaluation

This Special Issue of the Journal of Personnel Evaluation in Education (JPEE) consists of six articles associated with presentations at the 15th annual National Evaluation Institute (NEI) of the Consortium for Research on Educational Accountability and Teacher Evaluation (CREATE) held in Dallas, Texas (July 6–8, 2006). The theme of the CREATE conference was Expanding Research and Promising Practices in Personnel, Program, and Student Evaluation, with the objective of recognizing the importance of linking theory to practice. The vision of CREATE is to improve student learning, development, and achievement in PK-12 schools, institutes of higher education, and other educational settings. To that end, the mission of CREATE is (a) to advance the field of educational evaluation through a national forum for the presentation, discussion, and dissemination of sound practices of personnel, student, and program evaluation research and policy; (b) to facilitate a network among those who actively engage in personnel, program and/or student evaluations and those who benefit from such evaluations; and (c) to promote the development, dissemination, knowledge, and use of the personnel, program, and student evaluation standards as issued by the Joint Committee on Standards in Educational Evaluation. This Special Issue evolved in close collaboration between JPEE and CREATE. CREATE is grateful for this cooperation, bringing the opportunity to share some important manuscripts concerning evaluation to the attention of the readership of JPEE. Reviews and revision cycles resulted in the selection of the six articles among more than 50 submissions to the annual CREATE conference. The authors of the selected papers had an J Pers Eval Educ (2006) 19:1–3 DOI 10.1007/s11092-007-9034-7

The Case for Expanding Standards for Teacher Evaluation to Include an Instructional Supervision Perspective

Despite the concerns of scholars in the field of instructional supervision, teacher evaluations continue to emphasize bureaucratic accountability and standardization. This article presents an argument for extending the Joint Committee on Standards' Personnel Evaluation Standards to include standards related to the practice of supervision. The proposed standards call for differentiated procedures, collaborative identification of teachers' professional development goals, multiple sources of data, emphasis on formative evaluation processes, consideration of both teachers' personal development goals and school/program improvement goals, and the formalization of formative evaluation processes to achieve clear and shared understanding of their purpose and goals. These standards would align teacher evaluation with aspects of instructional supervision namely,clarification and shared understanding of the process and purpose of evaluations, interpretation of teaching performance in the context of teachers' classrooms and professional values, and deliberation with teachers about how evaluation evidence depicts and informs their work.

A Collaborative Approach to the Development and Validation of a Principal Evaluation System: A Case Study

This article provides a case history of collaboration between the South Carolina Educational Policy Center (SCEPC) at the University of South Carolina (USC), the South Carolina Department of Education (SDE), and stakeholder committees in developing a principal evaluation system for South Carolina. This case, study is based on the authors’ experiences in the process and their review of program documents. The project staff used the Personnel Evaluation Standards and the Standards for Educational and Psychological Testing to guide the review of the standards. They also worked with qualified job experts to develop performance standards, evaluation criteria, and instrumentation. The documented activities serve as evidence that validity issues were appropriately addressed. The project staff and stakeholder committees conducted an internal review and a field review of the proposed standards and criteria, as well as a pilot study of the proposed instrumentation. Stakeholder feedback served as an additional source of validity evidence. Although this case study focused on development of an evaluation system in one state, the information presented may be helpful to other states or districts that are planning to develop similar systems.

Assessment of Teacher Reactions to a Standards-Based Teacher Evaluation System: A Pilot Study*

This article reports on teachers' reactions to a pilot implementation of a new standards-based teacher evaluation system in a medium-sized Midwestern school district. The system, based on the Framework for Teaching (Danielson, 1996), was intended to serve as the foundation for a new knowledge and skill pay plan for teachers. Teacher reactions were measured using both interview and survey methods. Teachers' overall favorableness toward the system, as measured by the survey, was correlated with acceptance of the teaching standards, perception of the evaluation process as fair and the evaluator as capable and objective, and perception that the evaluation system had a positive impact on their teaching. In interviews, teachers perceived both positive and negative aspects of the system, and for many these more or less balanced each other. While most teachers accepted the evaluation standards and the need for an evaluation system, many also perceived that the system added too much to teachers' workloads and did not provide enough feedback. Based on these results several changes were made in the evaluation system prior to full implementation. The results are discussed with reference to previous research and the Personnel Evaluation Standards.

Superintendent Evaluation in a Standards-Based Environment: A Status Report from the States.

In the current political climate, in which accountability and standards-based reform represent the educational centerpiece, a renewed interest has emerged in superintendent evaluation as a focal point for school improvement. This paper examines the current state-of-the-art of superintendent evaluation. Findings from the study suggest that despite the recommendations for personnel evaluation standards proposed by The Joint Committee on Standards for Educational Evaluation (1988), 8 of the 50 state AASA/NSBA affiliates do not have requirements or guidelines for the evaluation of the superintendent. Evaluation materials provided by the affiliates in the other 42 states do not meet all of the essential criteria for quality personnel evaluation.

Achieving Legal Viability in Personnel Evaluations

Legal viability of an evaluation means that the evaluator can successfully address pertinent legal issues and avoid debilitating legal difficulties in conducting an ethical personnel evaluation and applying its results. While the 1988 Personnel Evaluation Standards contain no specific standard on legal viability, following the document’s twenty-one standards substantially aids evaluators to avoid legal difficulties. Nevertheless, meeting legal viability requirements is becoming increasingly complex and difficult in the face of the expanding role of personnel evaluations in high-stake decisions, including initial and advanced certification, sanctions and rewards, and terminations. Evaluators increasingly need sound, definitive guidance to meet procedural and substantive due process requirements and, in general, to keep their evaluations legally defensible. Eight years of experience in applying the 1988 Standards provides a foundation for developing a specific legal viability standard. This article argues for the inclusion of such a standard in the next edition of The Personnel Evaluation Standards. A prototype standard is presented to help the Joint Committee and its constituents deliberate on this matter.

Introducing a ''Legal Notes and Issues'' Section: Law's Growing Influence in the Field of Personnel Evaluation

In the ®eld of personnel evaluation, scholars and practitioners may disagree on many issues. Most would concede, however, that the ®eld has become more and more burdened by legal constraints. In the previous issue, the Journal of Personnel Evaluation in Education recognized this trend by devoting, for the ®rst time, a complete publication to legal matters. Authors explored such topics as employment discrimination, the legal viability of personnel evaluation standards, and teacher unions responsibilities for improving the quality of the teaching force. In this issue, JPEE introduces a Legal Notes and Issues section, where law-oriented articles will appear in the future. The following article by Kathleen A. Sullivan and Perry A. Zirkel is the ®rst contribution to this new section. Appropriately enough, their essay is an overview of recorded court cases on teacher evaluation issues. Updating Zirkel's (1996) survey of law in this area, the authors note several decisions in which courts invalidated a school district's personnel action based solely on procedural errors by school evaluators. They also discuss cases involving remedial plans for de®cient teachers, the application of open records laws to teachers' performance records, and damages claims brought by teachers who receive negative evaluations or references. In the months to come, more articles will appear on legal topics. We anticipate no shortage of manuscripts, since litigation and legislation in the area of personnel evaluation seem to be a growth industry. For example, within the last eighteen months, courts have issued these provocative opinions:

Looking backwards with the personnel evaluation standards: An analysis of the development and implementation of a statewide teacher assessment program

Looking backwards with the personnel evaluation standards: An analysis of the development and implementation of a statewide teacher assessment program

Analysis of alternative evaluation models

Based on our review of current evaluation practice, we have identified twelve main models for evaluating superintendent performance. The models may be grouped into three main categories according to how evaluation judgments are made: global judgment, judgment driven by specified criteria, or judgment driven by data. These three main categories of evaluation model are reflected below. This article presents an analysis of the twelve alternative evaluation models. It provides, for each model, a description or characterization of the model and a summary evaluation in relation to the 21 Joint Committee (1988) Personnel Evaluation Standards. The descriptions and evaluations presented here are drawn from an exhaustive indepth analysis of each alternative model published in a separate technical supplement (Candoli, Cullen, & Stufflebeam, 1994). This in-depth analysis of each model includes a description of the approach under a series of headings incorporating the main elements of an evaluation model. These descriptions were drawn from the research literature and the evaluation materials acquired from various states and school districts; as well as the various project participants' combined knowledge of an experiences with superintendent evaluation systems. In addition, the in-depth analysis involved a thorough evaluation of each model using 21 separate checklists, each comprising a series of questions relating to one of the 21 Standards. The information obtained using these checklists was then synthesized to produce, for all twelve models, an extensive list of strengths and weaknesses in relation to each of the 21 Standards. The summary of this work, presented here, is divided into twelve parts, one for each evaluation model. Each part includes first a description or characterization of the model, structured under eight main headings that are defined as follows:

Designing a personnel management system for school psychologists

The development of the personnel evaluation system for special services personnel in North Carolina exists as a case model for the implementation of the Personnel Evaluation Standards (Joint Committee on Standards for Educational Evaluation, 1988). Reliance on expertise from both those to be evaluated as well as those with expertise in the development of personnel evaluation systems, the consideration of the needs of individuals and of educational organizations, and the emphasis on the collection of data in naturalistic settings all tend toward the development of comprehensive systems of personnel management, of which performance evaluation is but one part, albeit an important part. In this way, of course, many of the cautions enumerated by Stronge and Helm (1991) have been accounted for and a strong system of personnel evaluation is the result. In any case, the development of personnel evaluation systems in North Carolina, of which the school psychologist is only one example, have made it clear that a performance evaluation system cannot be successfully built in isolation. The other components of a fully articulated personnel management system, complete with job description, methods and forms for data collection and analysis, training for both evaluators and evaluatees, and procedures for performance improvement and enhancement, are essential if the evaluation of performance is to be seen as anything other than a bureaucratic ritual in which organizations compel our participation.

Personnel evaluation standards: The use in principal assessment systems

This article reports on portions of a recently completed study (Martens, 1990) which explored the operation of personnel evaluation systems in local school districts. The study focused on the evaluation of principals by their superordinates (e.g., superintendents, assistant superintendents). Of special interest was the extent to which each evaluation standard is explicitly used in each of the districts studied. For this ex-post investigation the standards employed were those developed recently by a nationally representative group of educational educators and policymakers Joint Committee, 1988). The study found that several of the personnel evaluation standards are extensively used and that significant differences exist in the extent of use among the standards. The discussion section in the article deals with the relationship between the evaluation system and district personnel policy priorities, district contextual realities, and also with some theoretical considerations associated with evaluation as an administrative function in education.

The Kansas internship program assistance/assessment model: A design for beginning teachers' professional growth

In the words of the recent Personnel Evaluation Standards, " . . . educational institutions have often been ineffective in carrying out their personnel evaluation responsibilities" (1988, p. 6). According to these standards, this failure is documented in many current books, articles, and reviews that call for a reform in evaluating educational personnel (Andrews, 1985; Brodinsky, 1984; Carnegie Task Force on Teaching as a Profession, 1986; Coker, Medley & Soar, 1980; Educating Americans for the 21st CenturypNational Science Foundation, 1983; Goodlad, 1983; Millman, 1981; A Nation at Risk--National Commission on Excellence in Education, 1983; Wise & Darling-Hammond, 1985). State departments and school districts look to teacher evaluation as a means of enhancing teacher performance (Educational Research Service, 1978; Bolton, 1973). However, given the incompetence of many evaluation practices, most are not achieving this goal to any degree that is impressive (Stiggins & Duke, 1988). Kansas recognized the need for educational reform in the area of teacher evaluation by taking an innovative approach to evaluating and assisting beginning teachers.

The development, validation, and applicability of The Personnel Evaluation Standards

The development, validation, and applicability of The Personnel Evaluation Standards

Program evaluators foray into personnel evaluation

Historically, most program evaluators have not dealt with the evaluation of educational personnel. Stufflebeam and Sanders (1987), in writing about the need for improving personnel evaluations in education, point out that the program evaluation movement of the late 1960s and early 1970s largely excluded concerns for personnel evaluation. Despite historical practice, Stufflebeam and Sanders call upon evaluators to address personnel evaluation issues in addition to evaluations of students and programs. The position advocated by these authors raises the question of how evaluators might become involved in personnel evaluation. The purpose of this article is to describe one role that program evaluators can play: reviewing local personnel evaluation systems and addressing issues related to personnel evaluation. The need for local review of personnel evaluation systems has been documented by Ellett and Garland (1987), who report that policies in nearly 60% of the 100 largest U.S. school districts do not expressly call for systematic review of their teacher evaluation systems. They suggest that this percentage is likely to be higher for smaller districts. Two factors contributing to the absence of policy mandates in many districts may be the lack of a process for local review and of clear standards by which personnel evaluation systems can be judged. Work by the Joint Committee on Standards for Educational Evaluation has resulted in The Personnel Evaluation Standards (1988). These standards provide a structure that local educational agencies might use to judge the adequacy of their personnel evaluation systems. Described in this article is a process used by two program evaluators to review a local school district's teacher appraisal system based on the Standards. This article is based on our experience using the field test draft of the Standards to judge the adequacy of the design of the teacher appraisal sys-