Development of an Operative Performance Rating System for Plastic Surgery Residents

Abstract

Listen

Translate article

PURPOSE A standardized measure of operative performance is an essential component of the Patient Care Competency and is critical to the training of plastic surgery residents. The Operative Performance Rating System (OPRS) developed and validated by the Department of General Surgery at Southern Illinois University consists of procedure-specific evaluations for resident intraoperative performance. The OPRS provides an objective measure of procedure-specific resident performance that is not currently being assessed in plastic surgery training programs. The purpose of this study is to describe OPRS for plastic surgery residents and propose methodology for assessing the reliability, validity, and feasibility of this instrument. METHODS Ten procedure-specific rating instruments were developed for sentinel cases, each consisting of critical procedure-specific steps based on literature review and faculty focus group consensus. Sentinel cases were chosen based on review of the American Council of Graduate Medical Education Milestones and resident logs of the most commonly performed plastic surgery procedures, both at our institution and nationally. The degree of guidance required from the attending surgeon is recorded for each step. General operative performance competency is evaluated from validated items developed by the University of Toronto.1 All items use a 5-point Likert scale with behavioral anchors. The OPRS assessments will be incorporated into the internet-based resident management platform New Innovations. Sentinel procedures for evaluation will be identified on a weekly basis by the residency coordinator, based on resident operative assignments (postgraduate year 2–6) organized by the chief resident. OPRS assessment forms will be available electronically immediately following the procedures, with an e-mail reminder notification 24 hours later to help encourage compliance. In addition, resident self-assessment using the same OPRS will be conducted and correlated with faculty OPRS evaluations. Each OPRS assessment will be evaluated for internal consistency reliability and inter-item correlation. Inter-rater reliability will be measured by faculty assessment of videotaped sentinel procedures using the appropriate OPRS instrument. Performance variation based on resident PGY level will be analyzed using 1-way analysis of variance. Feasibility will also be determined based on attending and resident response rates and time to completion for the OPRS evaluations, as well as a short written survey to assess resident and attending satisfaction and obtain feedback following OPRS implementation. CONCLUSIONS A web-based OPRS provides timely and objective feedback to improve residents’ technical and decision-making skills, as demonstrated by the experiences of other surgical specialties.2 This instrument will provide both formative and summative resident feedback, encouraging faculty and residents to focus on demonstrated competencies and areas for improvement.3 Furthermore, resident operative performance can be monitored across time and residents, allowing program directors to have a long-term objective method of evaluating resident technical performance.3 A reliable and valid OPRS may provide a feasible method of intraoperative assessment that could be implemented across all plastic surgery training programs.