OPEN ACCESSJuly 11, 2013How to Develop, Implement, and Assess Multi-Source Feedback (MSF) for Trainees Elizabeth Grace Doherty, MD Elizabeth Grace Doherty, MD Children's Hospital Boston Google Scholar More articles by this author https://doi.org/10.15766/mep_2374-8265.9476 SectionsAbout ToolsDownload Citations ShareFacebookTwitterEmail Abstract Introduction: Multi-source feedback (MSF) imparts feedback from a multidisciplinary group of raters within the trainee's sphere of influence. Faculty involved in residency and fellowship training programs are charged with the task of implementing a MSF approach for trainee evaluation. However, many of these faculty members have little knowledge as to how to begin the process. This resource will allow faculty members to gain an understanding of MSF, how to approach implementation, how to address and uncover barriers, and, most importantly, how to present the approach to faculty within the program. Methods: This workshop lasts about 60 minutes. The PowerPoint presentation is delivered with comments and questions encouraged throughout. Approximately 30 minutes should be devoted to the group project. This workshop may be used by faculty seeking to form a MSF design and implementation subcommittee within their particular discipline. Results: This workshop was used successfully at the Boston Children's Hospital Academy for Teaching and Learning Annual Retreat in November 2012. Fellowship directors, or those involved in the assessment and evaluation of trainees, found the workshop helpful to assist in development of their own MSF assessment programs. Some faculty brought the MSF toolkit back to their department to form subcommittees to work through the steps for design and implementation of their own department-specific MSF assessment program. Discussion: This workshop is designed to provide the background and process of developing a MSF assessment program. Faculty that have already implemented a MSF program may not find the entire resource particularly helpful. However, the MSF toolkit may be reviewed to uncover areas for modification. Although most of the assessment forms have been converted into the generic form to be more universal, some may still be too specific to neonatal-perinatal medicine and may be less helpful for particular programs. Educational Objectives By the end of this resource, participants will be able to: Identify key aspects of multi-source feedback (MSF) development and implementation.Utilize the MSF toolkit.Identify barriers to MSF.Use this workshop as a resource in one's own practice to introduce and begin the development of MSF within a particular department or discipline. ReferencesACGME/ABMS Joint Initiative Toolbox of Assessment Methods Version 1.1. September 2000. Toolbox of Assessment Methods. 2000 Accreditation Council for Graduate Medical EducationGoogle ScholarAtwater LE, Brett JF, Charles AC. Multisource feedback: lessons learned and implications for practice. Hum Res Manage. 2007;46:285–307. http://dx.doi.org/10.1002/hrm.20161Google ScholarBrodsky D, Doherty EG. Providing effective feedback. NeoReviews. 2010;11:e117. http://dx.doi.org/10.1542/neo.11-3-e117Medline, Google ScholarCalhoun AW, Rider EA, Meyer EC, et al. Assessment of communication skills and self-appraisal in the simulated environment: feasibility of multirater feedback with gap analysis. Simul Healthcare. 2009;4:22–29. http://dx.doi.org/10.1097/SIH.0b013e318184377aMedline, Google ScholarDalton M. Multi-rater feedback and conditions for change. Consult Psychol J Pract Res.1996;48:12–16. http://dx.doi.org/10.1037/1061-4087.48.1.12Google ScholarDoherty EG, Brodsky D. Educational Perspectives: The 360-degree Assessment: A New Paradigm in Trainee Evaluation. NeoReviews2011;12;e191. http://dx.doi.org/10.1542/neo.12-4-e191Google ScholarDubinsky I. 360-degree physician performance assessment. Health Hum Res. 2010;13:71—76Google ScholarHall W, Violato C, Lewkonia R, et al. Assessment of physician performance in Alberta: the physician achievement review. Can Med Assoc J.1999;161:52—57Google ScholarHiggins RS, Bridges J, Burke JM, et al. Implementing the ACGME general competencies in a cardiothoracic surgery residency program using 360-degree feedback. Ann Thorac Surg.2004;77:12–17. http://dx.doi.org/10.1016/j.athoracsur.2003.09.075Medline, Google ScholarJones SB. Assessment: Why and How? BMJ PowerPoint presentation. group.bmj.com/products. January 2010.Google ScholarLelliott P, Williams R, Mears A, et al. Questionnaires for 360-degree assessment of consultant psychometric properties. Br J Psychiatry. 2008;193:156–160. http://dx.doi.org/10.1192/bjp.bp.107.041681Medline, Google ScholarLockyer JM, Clyman S. Multi source feedback. In: Holmboe ES, Hawkins RE, eds. Practical Guide to the Evaluation of Clinical Competence. Philadelphia, PA: Mosby Elsevier; 2008:75–85Google ScholarOvereem K, Wollersheim H, Driessen E, et al. Doctors' perceptions of why 360- degree feedback does (not) work: a qualitative study. Med Educ.2009;43: 874–882. http://dx.doi.org/10.1111/j.1365-2923.2009.03439.xMedline, Google ScholarSmither JW, London M, Reilly RR. Does performance improve following multisource feedback? A theoretical model, meta-analysis, and review of empirical findings. Personnel Psychol. 2005;58: 33–66. http://dx.doi.org/10.1111/j.1744-6570.2005.514_1.xGoogle ScholarSwick S, Hall S, Beresin E. Assessing the ACGME competencies in psychiatric training programs. Acad Psychiatry. 2006;30:330–351. http://dx.doi.org/10.1176/appi.ap.30.4.330Medline, Google ScholarSwing SR. Assessing the ACGME general competencies: general considerations and assessment methods. Acad Emerg Med.2002;9:1278–1288.Medline, Google ScholarVeloski J, Boex JR, Grasberger MJ, et al. Systematic review of the literature on assessment, feedback and physicians' clinical performance: BEME Guide No 7. Med Teach. 2006;28:117–128. http://dx.doi.org/10.1080/01421590600622665Medline, Google ScholarViolato C, Lockyer J, Fidler H. The assessment of pediatricians by a regulatory authority. Pediatrics. 2006;117: 796–802. http://dx.doi.org/10.1542/peds.2005-1403Medline, Google Scholar Sign up for the latest publications from MedEdPORTAL Add your email below FILES INCLUDEDReferencesRelatedDetails FILES INCLUDED Included in this publication: Attending Feedback Survey (pre launch of MSF).pdf Fellow Feedback Survey (pre launch of MSF).pdf Instructor's Guide.docx MSF Workshop Outline.docx Sample 360 Assessment Survey for Fellows or Residents.pdf Sample Fellow Self-Assessment Form.doc Sample Parent Survey for Fellow.pdf Workshop for MSF Assessment Program.ppt To view all publication components, extract (i.e., unzip) them from the downloaded .zip file. Download editor’s noteThis publication may contain technology or a display format that is no longer in use. CitationDoherty EG. How to Develop, Implement, and Assess Multi-Source Feedback (MSF) for Trainees. MedEdPORTAL. 2013;9:9476. https://doi.org/10.15766/mep_2374-8265.9476 Copyright & Permissions© 2013 Doherty. This is an open-access article distributed under the terms of the Creative Commons Attribution license.KeywordsTraining ProgramMultisource Feedback360-Degree AssessmentAssessmentCore CompetenciesQualitative SkillsEducationEducational MeasurementACGMEAcknowledgmentsPlease note NeoReviews (Luann Zanzola) has approved use of the 2 slides in the Powerpoint presentation, MSF Toolkit and the Factor Linked to Success/ Factors Leading to Potential Failure. Each has the reference: E. Doherty, D. Brodsky. Educational Perspectives : The 360-degree Assessment: A New Paradigm in Trainee Evaluation NeoReviews 2011;12;e191. http://dx.doi.org/10.1542/neo.12-4-e191 Disclosures None to report. Funding/Support None to report. Loading ...