Multirater Feedback Research Articles

How Effective is the 360-degree Feedback Appraisal Method for Evaluating Employee Performance?

360-degree feedback, also known as multisource feedback, multi-rater feedback, or the 360-degree review system, is a comprehensive employee evaluation method designed to promote the all-around development of employees. Unlike traditional top-down appraisal methods, this system gathers input from various sources to create a holistic view of performance and areas for improvement. The 360-degree feedback process is designed to be continuous, supporting ongoing development throughout the year, rather than relying solely on periodic evaluations. This approach helps to address performance issues and foster growth on a regular basis, promoting a culture of continuous improvement.

Unlocking the power of emotional intelligence in medical writers

Emotional intelligence (EI), the ability to recognise and understand emotions (one’s own and those of other people), is a valuable attribute for medical writers (MWs). MWs play a pivotal role in communicating medical and scientific information to regulators, patients, caregivers, and healthcare professionals. EI helps MWs to understand the needs of the audience, develop effective communication skills, author impactful content, and collaborate efficiently with multiple stakeholders. It is thus essential for MWs to develop strong EI skills, i.e., be self-aware and self-motivated, be able to self-regulate, take ownership and responsibility for actions, be problem solvers, show empathy, foster healthy relationships, evaluate feedback positively, and implement balanced solutions. To enhance their EI, MWs can adopt EI-focussed training and development sessions at workplace, practice mindfulness daily, assess EI using multi-rater feedback scales, seek EI coaching, and set realistic goals.

Leading under pressure: a contextualised development approach

PurposeThe study’s aim was to design and test a leadership development approach using blended learning, to equip leaders for strengthening their own resilience and that of their teams.Design/methodology/approachA contextualised leadership development intervention was produced and evaluated following the principles of design-based research. Participants were from three organisations that work internationally to address the impact of economic disadvantage. Initial research used the behavioural event interview technique. Online assessment incorporated measures of situational judgement, emotion recognition and attributional style. Validity measures were multi-rater feedback (criterion), and NEO-PI 3 (construct). Individual feedback and a simulation-based peer workshop were followed by a four-to-six month period of experience-driven development and a final peer workshop for consolidating and evaluating learning outcomes.FindingsThe online assessment was a valid measure of leaders’ personal resilience resources and their resilience-building capability. Overall, the intervention improved participants’ understanding of, and engagement with, the processes of strengthening individual and collective (team) resilience.Research limitations/implicationsThe target sample size for the study was relatively small, to ensure it would be practical to replicate the approach when designing similar interventions for a senior leadership population in other contexts. Significant results provided robust evidence for the validity of the assessment approach. Findings for the workshops and experience-driven development phase were more tentative, but the value of the design iterations was clearly demonstrated.Practical implicationsThe leadership development approach is suitable for application in other organisations, if similar principles are followed to produce and evaluate materials relevant to each broad sector context. Roll-out is cost-effective, with relatively few hours of blended or virtual delivery supporting experience-driven learning.Social implicationsThe impact leaders have on the wellbeing of those who report to them is well established, but less has been done to develop and formally evaluate practical, cost-effective interventions to improve this impact. The approach validated in this study can be applied more widely to benefit employee wellbeing as well as performance.Originality/valueThe study developed and evaluated a new approach to preparing leaders for the challenge of building team resilience, an aspect of leadership capability that has been given relatively little attention to date.

Leadership development through 360-degree multi-rater feedback—An experience sharing of need, approach, roll-out, and the impact

The organizational development (OD) interventions carried out by the assessment center identifying the competencies becomes effective only if the expectation of the stakeholders is met. A 360-degree multipurpose intervention is to assess and develop project director’s leadership capabilities and enable them to execute Mega Infrastructure Projects in India executed by Larsen and Toubro, the leading engineering, procurement and construction (EPC) players of India. Implementing such Mega Projects by maintaining time, cost, safety, and quality parameters require leadership skills of high order. To meet out this requirement implementation partner interacted with few project directors from each business verticals such as nuclear, hydel, metro, defense, special bridges and ports and harbor, to understand the role holders’ challenges and the expectations. L&T Heavy Civil Infrastructure (HCI) Leadership Competency framework was thus drafted looking into the role intensity and specific role-based challenges of the Mega Project Leaders. The questionnaires were designed to cover all dimensions and sub dimensions of 8 competencies, covering 40 behavioral Indicators. the questionnaires were hosted on the online platform to including self, supervisor, direct repartees, peers, customers, and other stakeholders. Validation to the assessment of each dimension and sub-dimensions as qualitative feedback is done with simple average scores and the range. The study found that execution leadership was the highest rated competency followed by customer engagement and talent management. The study also found that strategic orientation and decision making were rated low compared to other competencies, got validated by comments of customers as well. This is quite evident from the feedback that was taken after 9 months of the intervention. The study by using Delphi technique revealed that by providing feedback from all stakeholders supported by systematic coaching, 88% participants showed improvement in the low rated items. The study further proved that with proper feedback and coaching at an individual level with required trust and keeping development as the core agenda and without linking to any other HR processes like appraisal and promotion, majority of the participants took full ownership, interest and worked on their individual competency development showing their learning agility.

Assessment of National Population and Family Planning Agency Employees as Part of the Implementation of Bureaucratic Reform

The formulation of the problem in the research is how to assess the performance of BKKBN employees in implementing bureaucratic reform and the obstacles faced in assessing the performance of BKKBN employees in implementing bureaucratic reform. The aim to be achieved in this research is to determine the employee performance assessment carried out at BKKBN regarding the implementation of bureaucratic reform in achieving the program for structuring the apparatus' HR management system. The research approach used is a qualitative approach with a case study method. In this case, the researcher wants to see employee performance assessments carried out by the BKKBN in the context of implementing bureaucratic reform as outlined in the BKKBN bureaucratic reform roadmap Based on the results of the research that has been carried out, it can be concluded that the results of the discussion and research, namely: 1. In general, the implementation of bureaucratic reform has gone well, starting with the formation of a Bureaucratic Reform Team consisting of 9 Working Groups, a Road Map for Bureaucratic Reform, and an Independent Assessment of the Implementation of Bureaucratic Reform (PMPRB) carried out by the Inspectorate. 2. In general, employee performance assessment has gone well, starting with various employee performance assessment systems, namely the Balance Scorecard method, employee work performance assessment, and employee discipline and work behavior assessment, in which there is the latest performance assessment application with the Multi-Rater Feedback application. 3. Obstacles in assessing employee performance at BKKBN, namely: a. The application of the performance assessment system with the Balance Scorecard and Multi-Rater Feedback system is still not optimal b. There are still leaders who are less objective in providing assessments. c. There are limitations to the performance assessment system used, such as the BSC method which is only synchronous for Echelon IV while it cannot be done at the staff level. d. Appraisers lack the skills needed to carry out Performance Appraisals effectively.

Multirater Feedback of Leadership of Underrepresented Groups in a Civil Engineering Capstone Design Course

AbstractThis study examined self, peer, and instructor ratings of leadership behaviors in four areas: collaboration, creativity, organization/control, and competition for 98 students in one civil e...

Leading Organizational Change: Improved Leadership Behaviors Among Public Health Leaders After Receiving Multirater Feedback and Coaching

Leading Change is one of five Executive Core Qualifications (ECQs) used in developing leaders in the federal government. Leadership development programs that incorporate multirater feedback and executive coaching are valuable in developing competencies to lead change. We examined the extent by which coaching influenced Leading Change competencies and identified effective tools and resources used to enhance the leadership capacity of first- and midlevel leaders at Centers for Disease Control and Prevention's National Center for HIV/AIDS, Viral Hepatitis, Sexually Transmitted Diseases, and Tuberculosis Prevention. Data included qualitative data collected via semi-structured interviews that focused on leadership changes made by leaders in the Coaching and Leadership Initiative (CaLI), a leadership development program for Team Leads and Branch Chiefs. Ninety-six participants completed leadership coaching; 94 (98%) of whom completed one or more interviews. Of those 94 respondents, 74 (79%) reported improvements in their ability to lead change in 3 of 4 leading change competencies: creativity and innovation, flexibility, and resilience. All respondents indicated tools and resources that were effective in leading change: 49 (52%) participated in instructor-led activities during their CaLI experience; 33 (35%) experiential activities; 94 (100%) developmental relationships, assessment, and feedback; and 25 (27%) self-development. First- and midlevel leaders in a public health agency benefitted from using leadership coaching in developing competencies to lead organizational change. Leadership development programs might benefit from examining Leading Change competencies and including instructor-led and experiential activities as an additional component of a comprehensive leadership development program.

Exploring the future of startup leadership development

Leadership development (LD) is a crucial success factor for startups to increase their human capital, survival rate, and overall performance. However, only a minority of young ventures actively engage in LD, and research rather focuses on large corporations and SMEs, which do not share the typical startup characteristics such as a rather young workforce, flat hierarchies, resource scarcity, and high time pressure. To overcome this practical and theoretical lack of knowledge, we engage in foresight and explore which leadership development techniques will be most relevant for startups within the next five to ten years. To formulate the most probable scenario, we conduct an international, two-stage Delphi study with 27 projections among industry experts. According to the expert panel, the majority of startups will engage in leadership development over the next decade. Most startups will aim to develop the leadership capabilities of their workforce as a whole and use external support. The most prominent prospective LD measures in startups include experiential learning methods, such as action learning, developmental job assignments, multi-rater feedback, as well as digital experiential learning programs, and developmental relationships such as coaching in digital one-to-one sessions. Self-managed learning will play a more important role than formal training.

Changing Leadership Behaviors in a Public Health Agency Through Coaching and Multirater Feedback.

Public health managers' leadership skills can be improved through multirater feedback and coaching. To explore to what extent participation in a coaching intervention influences leadership behaviors of first- and second-level leaders in a federal public health agency. Team leads and branch chiefs in the Centers for Disease Control and Prevention's (CDC's) National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention (NCHHSTP) were invited to participate in the Coaching and Leadership Initiative (CaLI), which incorporates the US Office of Personnel Management (OPM) Leadership 360 assessment, 6 coaching sessions, and 2 in-depth interviews. NCHHSTP is one of 16 CDC national centers, institute, and offices. Staff serving as team leads or branch chiefs. Two in-depth interviews explored CaLI's influence on leadership behaviors regarding the government-wide Leading People executive core qualification. A total of 103 (93%) CaLI participants completed the OPM 360 feedback, 82 (80%) completed leadership coaching; 71 of 82 (87%) completed phase 1 interview, and 46 of 71 (65%) completed phase 2 interview. Eighty unique participants completed 1 or more interviews; all indicated that CaLI helped provide new perspectives, practices, and approaches that led to better communication and relationships, different approaches to conflict resolution, and awareness of individual leadership practices. Of the 71 participants who completed phase 1 evaluation, 66 (93%) said they made changes in developing others, 56 (79%) completed conflict management and team building, and 16 (23%) completed leveraging diversity. Of the 46 participants who completed both phase 1 and phase 2 interviews and among those who made changes post-CaLI, 23 of 26 (88%) sustained those leadership changes in developing others, 21 of 27 (78%) in team building; 24 of 34 (71%) in conflict management; and 5 of 10 (50%) in leveraging diversity. This study demonstrates the benefits and effectiveness of using multirater feedback and leadership coaching for first- and midlevel public health leaders.

A Scalable Approach for 360° Feedback in Cooperative Learning

We present a standard-based methodology to data-mine and eventually compute the results from 360° feedback surveys. We pay special attention to its suitability in the educational and pedagogical sphere, where very little research efforts have been accomplished. A 360° feedback study (also known as multi-rater feedback) provides a holistic view of an appraisee's performance based on the information and opinions gathered from raters about the person being evaluated. An evaluator or appraiser can be any peer, coworker, subordinate, supervisor, customer, classmate, and so on and even the appraisee himself/herself. This type of surveys is usually carried out in big companies/institutions and by specialized human resources departments. The computer tools normally used for the processing of the data generated by these studies are based on simple spreadsheets and ad hoc scripts backed by untapped databases. In this context, we advocate the use of a refined, standardized, and straightforward computing environment based on stored procedures. The Persistent Stored Module and the stored procedures there defined are part of the Structured Query Language ISO norm, and they can be leveraged to perform complex calculations without the need of developing external software or requiring extra network transactions. This approach can also be transferred to general 360° appraising studies and, with special emphasis, to those performed in academic levels. As a testbed, the presented techniques have been put into practice in a multi-rater evaluation experience executed with over 240 high-school students. The technical and pedagogical lessons learned in this initial test can be eventually exported to the three main academic levels in a network of educational centers with more than 44k scholars enrolled. The inclusion of such assessment actions in educational environments enables the fostering of the novel pattern of cooperative learning, as it is also highlighted.

Applying a balanced score card approach and Multi-Rater feedback strategy to shift from appraising to managing head nurses’ performance at general surgical units-Main Mansoura University Hospital-Egypt

Background and objective: Traditionally, performance appraisal is used to measure behaviors, procedures or actions taken as in head nurse performance plans. Applying balanced score card measures at the nursing administration department level with participating the findings with all nursing staff and patients will help administrators to get all information needed to match head nurses performance plans with nursing administration department as well as hospital goals which help in drawing full shape of performance. The objective was to apply balanced scorecard approach with 360-degree feedback strategy to shift from appraising to managing head nurses’ performance at general surgical units -Main Mansoura University Hospital-Egypt.Methods: Subjects: All supervisors (n = 12), head nurses (n = 10), staff nurses (n = 96) working in general surgical units and all available patients admitted to these units at the time of study (n = 113) were included. Tools: Eight tools that were used for data collection involved feedback questionnaires for patients, head nurses, staff nurses and supervisors, observational checklist, activity analysis checklist, auditing performance appraisal form, and auditing personnel decisions form.Results: There was a statistically significant difference between head nurses, staff nurses and supervisors’ perspectives from one side and among patients’ perspectives from the other side regarding the performance of head nurses’ in general surgical units. In addition, nearly of third of head nurses’ time spent in unclassified activities. However, none of personnel decisions depended on performance appraisal outcomes.Conclusions: The results of the present study indicated that the methods used to measure head nurses’ performance in general surgical units at Main Mansoura University Hospital were not integrated or depend on clear work standards to develop and improve the performance of Head Nurses (HNs).

Is 360 Degree Feedback Appraisal an Effective Way of Performance Evaluation?

The 360 degree feedback approach has been implemented by many organisations for development or performance evaluation objectives. This paper questions the effectiveness of 360 degree feedback implementations in rating employee performance. This literature review was conducted on 360 degree feedback practice with performance evaluation purposes. The aim of this review was to define and discuss the 360 degree feedback; contrasting the process with the other methods and identifying whether this is a good way of performing appraising or not. The superiority of multi-rater feedback to the traditional methods and the dominance of advantages over disadvantages lead us to conclude that the 360 degree feedback is effective in rating performance.

Framing Feedback for School Improvement Around Distributed Leadership

Purpose: The purpose of this article is to examine the utility of framing formative feedback to improve school leadership with a focus on task-based evaluation of distributed leadership rather than on role-based evaluation of an individual leader. Research Methods/Approach: Using data from research on the development of the Comprehensive Assessment of Leadership for Learning, we examine ways to design formative evaluation and feedback organized around distributed leadership practices. This study draws on qualitative data from iterative design research conducted with middle and high school principals, assistant principals, teachers, and staff in 2011. Findings: Many challenges in providing actionable, multirater feedback were addressed by using an assessment instrument that focused on measuring distributed instructional leadership practices. Users reported that task-based multirater feedback provided transparency in communicating a clear theory of action for school improvement and fostered formal and informal conversations around school improvement. Implications for Research and Practice: The study suggests that focusing on distributed leadership practices may help overcome some of the limitations to the use of evaluation feedback that is targeted to an individual leader. Key features of formative feedback design desired by school principals included transparency in the theory of action underlying assessment items to prime teachers and leaders for collaborative discussions of current and potential practices, and research-based guidance on next steps schools can take to build distributed instructional leadership capacity in their schools.

How to Develop, Implement, and Assess Multi-Source Feedback (MSF) for Trainees

Abstract Introduction Multi-source feedback (MSF) imparts feedback from a multidisciplinary group of raters within the trainee's sphere of influence. Faculty involved in residency and fellowship tr...

360-Degree Feedback

Originally started during World War II by the Germans, and later implemented by the Esso Research and Engineering Company in the 1950’s, utilization of the 360-degree feedback model is increasing in popularity. In this model, qualitative or quantitative ratings are collected from designated individuals, which may include supervisors, subordinates, peers, and internal and/or external customers (Dessler, 2010). Proponents of this model contend that it allows for a clear representation of behaviors, leadership skills, strengths, and weaknesses so that the individual may create a plan for personal growth and development, thereby improving the company as a whole. Opponents, however, argue that this model typically does not provide accurate data and must consider too many variables to be successful. Regardless of these arguments, the 360 feedback model is a very important component within Human Resources with nearly all Global 1000 companies implementing some type of multi-rater feedback instrument to evaluate key contributors and managers.

Personality Antecedents of Self-Other Rating Discrepancy

Multi-rater feedback systems are increasingly being used in organizations for assessment and development. Research suggests self-ratings are often uncorrelated with other-ratings, which has implications for ratee receptivity to feedback. Prior research suggests ratee personality may offer insight into selfother rating discrepancy. We explored the extent to which five work-related personality dimensions offer explanatory variance in self-other rating discrepancy using archival data of 527 employees from a large financial organization. Analyses of 487 self-peer dyads and 501 self-supervisor dyads revealed low agreement among self-, supervisor-, and peer-ratings of performance. Self-raters high on extraversion, dominance, cultural conformity, cynicism, and detail orientation were somewhat likely to over-rate their performance as compared to peer ratings. Cynical and detail-oriented raters were likely to over-rate their performance as compared with supervisor ratings. Detail orientation and cultural conformity interacted to predict self-other rating discrepancies for both supervisors and peers. Implications for research and practice are discussed.

Developmental coaching for high school teachers: Executive coaching goes to school.

Teachers are in a very real sense the embodiment of leadership, providing direction, guidance, and feedback to their students in addition to acting as role models. Teachers may well thus benefit from developmental coaching that draws on theories of leadership. This study was both an experimental (randomly assigned conditions) and a quasiexperimental (pre‐post) study. A randomized controlled design was used to explore the impact of coaching on goal attainment, mental health, workplace well-being, and resilience, and a quasi-experimental (pre‐post) design was used to explore the impact of coaching on leadership styles. Forty-four high school teachers were randomly assigned to either coaching or a waitlist control group. The coaching used a cognitive‐behavioral, solution-focused approach and was informed by theories of self-leadership and transformational leadership. Participants in the coaching group received multirater feedback on their leadership style and undertook 10 coaching sessions conducted by professional coaches over a 20-week period. Compared with randomly allocated controls, participation in coaching was associated with increased goal attainment, reduced stress, and enhanced workplace well-being and resilience. Pre‐post analyses for the coaching group indicated that coaching enhanced self-reported achievement and humanistic‐ encouraging components of constructive leadership styles and reduced self-reported aggressive/defensive and passive/defensive leadership styles. Findings suggest that coaching, as a professional development methodology, has great potential to contribute to the development and well-being of society beyond the corporate and organizational settings with which leadership coaching and executive coaching are normally associated.

Multi-rater feedback with gap analysis: An innovative means to assess communication skill and self-insight

Objective Multi-rater assessment with gap analysis is a powerful method for assessing communication skills and self-insight, and enhancing self-reflection. We demonstrate the use of this methodology. Methods The Program for the Approach to Complex Encounters (PACE) is an interdisciplinary simulation-based communication skills program. Encounters are assessed using an expanded Kalamazoo Consensus Statement Essential Elements Checklist adapted for multi-rater feedback and gap analysis. Data from a representative conversation were analyzed. Results Likert and forced-choice data with gap analysis are used to assess performance. Participants were strong in Demonstrating Empathy and Providing Closure, and needed to improve Relationship Building, Gathering Information, and understanding the Patient's/Family's Perspective. Participants under-appraised their abilities in Relationship Building, Providing Closure, and Demonstrating Empathy, as well as their overall performance. The conversion of these results into verbal feedback is discussed. Conclusions We describe an evaluation methodology using multi-rater assessment with gap analysis to assess communication skills and self-insight. This methodology enables faculty to identify undervalued skills and perceptual blind spots, provide comprehensive, data driven, feedback, and encourage reflection. Practice implications Implementation of graphical feedback forms coupled with one-on-one discussion using the above methodology has the potential to enhance trainee self-awareness and reflection, improving the impact of educational programs.

공기업 다면평가제도의 각 요인별 조직성과에 미치는 영향 분석

본 연구는 공기업을 중심으로 다면평가제도의 선행요인이 평가효용성에 미치는 영향과 평가효용성이 조직성과에 미치는 영향을 파악하고 평가제도의 사전교육정도에 따라 평가효용성에 미치는 영향에 차이가 있는지를 알아보고 궁극적으로 피평가자가 인식하는 평가효용성이 조직성과에 어떠한 영향을 미치는지 4가지 관점에서 연구하였다. 연구 결과 선행요인인 평가자 신뢰성, 최고경영자의 지지는 평가효용성에 유의한 영향을 미치고 있으나 평가준거는 유의적이지 못하였다. 사전교육의 정도에 따라 선행요인과 평가효용성 간에 조절효과가 있는지를 검증한 결과 평가자 신뢰성은 조절효과가 있는 것으로 판명되었다. 그러나 평가준거, 최고경영자의 지지는 조절효과가 없는 것으로 나타났다. 결과적으로 본 연구의 평가효용성이 선행요인의 영향을 받아서 조직의 성과 즉, 조직몰입도, 직무만족도, 자기계발기회, 조직활성화에 직접 또는 간접적으로 영향을 미치며, 사전교육이 부분적으로 조절효과가 있는 것으로 확인할 수 있었다. This study, centering on public enterprises, has been conducted from 4 different perspectives such as grasping the effects that the factor precedent of the multi-rater feedback affects on the usefulness of evaluation and the effects that the usefulness of evaluation affects on the organizational achievement, checking on whether there is any discrepancy among the effects affecting on the usefulness of evaluation in accordance with the degree of advance training on the evaluation system, and being what kind of effects the usefulness of valuation that the appraised eventually understand affects on the organizational achievement. As result of the study, the factor precedent that are confidence of the appraisers and the support from CEO reasonably affected on the usefulness of evaluation, but the standard referred to evaluation was not reasonable. It became resultantly clear that the confidence of the appraisers held the effect of control from the verification of whether there is the effect of control between the factor precedent and the usefulness of evaluation in accordance with the degree of advance training. However, it turned out that the standard referred to evaluation and the support from CEO held no effect of control. As result, It was possible to confirm that the usefulness of evaluation in this study affects directly or indirectly to the organizational achievement such as organizational immersion, job satisfaction, self development opportunity, and organizational revitalization as influenced by the prior factors, and the advance training was partially effective on the adjusting effect.

A randomized, controlled crossover study to discern the value of 360-degree versus traditional, faculty-only evaluation for performance improvement of anesthesiology residents

360-degree evaluation, or multi-rater feedback, is a means of providing evaluation from a variety of stakeholders. The Accreditation Council for Graduate Medical Education (ACGME) lists 360-degree feedback as a recommended method for evaluating residents. Our study determines if 360-degree evaluation, as compared to traditional evaluation, affords anesthesiology residents greater potential for performance improvement. After IRB exemption and resident consent to participate, sixteen anesthesiology residents of various training levels at The George Washington University Medical Center were randomly assigned to receive either 360-degree evaluation or traditional evaluation. Mid-way through the study, the groups were crossed over. Three faculty members blinded to which type of evaluation each resident received evaluated all residents pre-study, midway, and at the conclusion of the study according the six core competencies set forth by the ACGME. The 360-degree study included evaluations by faculty (traditional), self, medical students, nursing staff and patients. Performance improvement in all core competencies demonstrated a trend toward greater scoring for residents who received early exposure to 360-degree evaluation compared to later exposure. Paired t-Tests demonstrated significance for resident performance improvement with early 360-degree evaluation versus early traditional evaluation for the Interpersonal and Communication Skills core competency over the entire study. Systems-based Practice and Practice-Based Learning and Improvement suggested improvements, but failed to reach statistical significance (P = 0.09, 0.07 respectively). Does a 360-degree evaluation, as compared to traditional, faculty-only evaluation afford anesthesiology residents greater potential for performance improvement? Sixteen residents were exposed to a prospective, single-blinded, crossover design study to determine the answer. For 360-degree evaluations, residents were rated according to the ACGME core competencies after adjusting to Program Director review of evaluations from self, faculty, nursing staff, medical students and patients.