Use Of Workplace-based Assessments Research Articles

A phenomenological study of resident and faculty experiences with learner engagement in the normalization of workplace-based assessment

Background: Workplace-based assessments (WPBA) have become integral to learner-centred medical education. As previous research has linked learner engagement to WPBA implementation, this study explores residents’ and faculty members’ experiences with learner engagement in the normalisation of WPBA practice. Methods: Transcendental phenomenology was used as the qualitative approach, focusing on the participants’ lived experiences. A semi-structured interview guide was used to interview five faculty members and five residents who had conducted WPBA. The interviews were transcribed and analysed using phenomenological data analysis. Results: Three themes were identified between learner engagement and WPBA conduct: (a) work environment, (b) roles and relationships, and (c) mutually beneficial teaching and learning. WPBA learner engagement occurred when participants interacted with each other and with the clinical setting to facilitate teaching and learning. Both participant groups reported a desire to participate in WPBA, but time constraints at times hindered participation. The residents indicated that WPBA improved their knowledge and admitted to experiencing negative emotions during the assessment. Overall, participants recognised the reciprocal benefits of WPBA participation for their professional development. Conclusion: The findings of the study suggest that learner engagement influences the use of WPBA. Consequently, it may be beneficial to consider the role of learner engagement to normalise WPBA application for teaching and learning in the clinical context.

Next steps for workplace-based assessments of entrustable professional activities.

Florida International University Herbert Wertheim College of Medicine (FIU-HWCOM) participated in the AAMC Core Entrustable Professional Activities (EPA) implementation pilot. Entrustment decision processes based on data from workplace-based assessments (WBAs) were piloted. Outcomes illustrated challenges including variability across EPAs with regards to learner level alignment and feasibility of data collection in the form of WBAs. In addition, students reported discomfort requesting WBA completion by preceptors and dissatisfaction with associated feedback. To guide future directions, we conducted a survey of third-year students to better understand their experience with and perceptions of WBAs used to evaluate EPAs at FIU-HWCOM. Survey response was 96% (n= 107/112). Most (84%) reported that WBAs were not valuable to their development and that preceptors often did not complete WBAs in a timely fashion. Many (47%) reported not receiving verbal feedback. Most students (78%) used language in written responses demonstrating confusion between the EPAs and the WBAs used to assess them. The use of WBAs to assess EPAs did not have its intended impact at FIU-HWCOM. For future classes, WBA forms will consist of paper cards with questions directly assessing performance of skills aligned with EPAs 1, 5 and 6 only. To continue to promote feedback, students will be required to collect WBAs on all clerkships, but the number of required WBAs will be less than prior and no entrustment decisions will be made.

The Use, Effectiveness, and Impact of Workplace-Based Assessments on Teaching, Supervision and Feedback Across Surgical Specialties

To investigate the use and effectiveness of Workplace-based assessments (WBAs) and their impact on training, feedback, and perioperative teaching in surgical training programs. A mixed methods cross-sectional, national electronic survey was conducted with surgical trainees and consultant trainers. The trainees and supervising faculty were from all 8 major surgical training universities across 11 surgical disciplines in South Africa. A total of 108 surgical trainees and 41 supervising consultant trainers from 11 surgical disciplines across 8 surgical training universities responded to the survey. The most significant educational gap identified by both the surgical trainees and trainers across all surgical disciplines was inadequate perioperative feedback. A third of the respondents were currently using workplace-based assessments. The WBA users (both trainees and trainers) had a higher rating for the general quality of surgical feedback than WBA nonusers (p = 0.02). WBA users also had a higher rating for the general quality of feedback given to trainees on their skills and competence (p = 0.04) and a higher rating for trainee supervision (p = 0.01) and the specialist training program overall (p = 0.01). The WBA users also had a higher rating for the assessment of competencies such as the trainee as an effective communicator (p < 0.01) and collaborator (p = 0.04). This study found that the use of WBAs enhances the quality and effectiveness of feedback in surgical training programs. We also found that the use of WBAs enhance perioperative teaching and learning and improves the assessment of relational competencies. This was also associated with high ratings for the quality of trainee supervision. Faculty and trainee development, strengthening the trainee-trainer relationship, and integrating iterative stakeholder feedback could help realize the full potential of WBAs to augment surgical training across disciplines.

Aligning Medical Student Workplace-Based Assessments with Entrustable Professional Activities and the RIME Model in a Psychiatry Clerkship.

The authors piloted use of workplace-based assessments of students during the psychiatry clerkship utilizing both entrustable professional activities (EPAs) and the reporter, interpreter, manager, and educator (RIME) model. After supervising cliniciansconducted assessments of medical students (N=109) during the psychiatry clerkship using a supervisory scale aligned with both EPA and RIME models, each student received individualized formative feedback. Students were then surveyed on the usefulness of this feedback, and participating faculty/residents were surveyed on the ease of completion of the supervisory scale. Students' mean skill profile suggested they no longer needed direct supervision on EPA1 and EPA6. Mean scores on other studied EPAs suggested students were well on their way toward performing these EPAs without direct supervision. Students had mean RIME scores that exceeded the suggested levels identified for a Reporter to start clerkships, for an Interpreter to start clerkships, and for a Manager to transition to the fourth year. Close to half of the students found the feedback helpful in their development as a clinician but most felt their performance should not be shared with residency program directors, either before or after the Match. Almost all responding preceptors felt the supervisory ratings were easy to complete. This pilot RIME/EPA framework served as a successful step toward a more competency-based medical education in the psychiatry clerkship with relatively little additional faculty time commitment by using workplace-based assessments already in place and a supervisory scale based on EPAs and RIME.

Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review.

Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions. The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders. In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain. Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.

SP2.1.11Real-world use of workplace-based assessments in surgical training: a UK nationwide exploration of trainee perspectives and consensus recommendations from the Association of Surgeons in Training

Abstract Background The utility of Workplace-Based Assessments (WBAs) in surgical training depends on the way they are completed. This study aims to ascertain the real-world usage of WBAs, as perceived by UK surgical trainees. Methods An anonymous national online questionnaire was conducted via the Association of Surgeons in Training (ASiT). The evaluation employed mixed methods to interpret quantitative and qualitative data. Suggestions for improvement were identified from free-text comments and considered through a formal two-round modified Delphi consensus process by ASiT council members. Results Analysis included 906 complete responses from all surgical specialities across all training levels. WBA use was perceived to deviate markedly from the Joint Committee on Surgical Training standards with significantly identified misuse across all surgical specialities. Inaccurate completion was acknowledged by 89.6% of respondents, and some trainers appear complicit with 40.9% of respondents aware of 'unobserved sign-off', and 33.6% aware of 'password disclosure' by trainers. Most trainees felt the Annual Review of Competency Progression (ARCP) respected WBA quantity above quality (55.4%), and a third felt pressure to overstate the number completed (32.0%). Main reasons for misuse were time restraints, lack of engagement and the will to achieve career progression targets. Conclusions UK surgical trainees perceive a significant deviation from WBAs standards due to multiple factors. The frequency and nature of misuse are worrying and undermines the existing evidence for the WBAs role in training. We present consensus recommendations from ASiT for the improvement of WBA use in UK surgical training.

Are You Thinking What I'm Thinking? Exploring Response Process Validity Evidence for a Workplace-based Assessment for Operative Feedback

Workplace-based assessments (WBAs) are used in multiple surgical specialties to facilitate feedback to residents as a form of formative assessment. The validity evidence to support this purpose is limited and has yet to include investigations of how users interpret the assessment and make rating decisions (response processes). This study aimed to explore the validity evidence based on response processes for a WBA in surgery. Semi-structured interviews explored the reasonings and strategies used when answering questions in a surgical WBA, the System for Improving and Measuring Procedural Learning (SIMPL). Interview questions probed the interpretation of the three assessment questions and their respective answer categories (level of autonomy, operative performance, case complexity). Researchers analyzed transcripts using directed qualitative content analysis to generate themes. Single tertiary academic medical center. Eight residents and 13 faculty within the Department of Otolaryngology-Head and Neck Surgery participating in a 6-month pilot of SIMPL. We identified four overarching themes that that characterized faculty and resident response processes while completing SIMPL: (1) Faculty and resident users had similar content-level interpretations of the questions and corresponding answer choices; (2) Users employed a variety of cognitive, behavioral, and emotional processes to make rating decisions; (3) Contextual factors influenced ratings; and (4) Tensions during interpretation contributed to rating uncertainty. Response processes are a key source of evidence to support the validity for the formative use of WBAs. Evaluating response process evidence should go beyond basic content-level analysis as contextual factors and tensions that arise during interpretation also play a large role in rating decisions. Additional work and a continued critical lens are needed to ensure that WBAs can truly meet the needs for formative assessment.

Real-world use of workplace based assessments in surgical training: A UK nationwide cross-sectional exploration of trainee perspectives and consensus recommendations from the Association of Surgeons in Training

BackgroundDespite widespread uptake, the utility of Workplace Based Assessments (WBAs) is disputed and evidence underpinning their use is largely based upon their completion in ideal conditions, rather than the real-world setting. AimTo ascertain the real-world usage of WBAs, as perceived by UK surgical trainees. Materials and methodsAn anonymous online questionnaire conducted nationally via the Association of Surgeons in Training (ASiT). Evaluation of 906 completed trainee responses, across all surgical specialties and training levels, employed mixed methods to interpret quantitative and qualitative data. ResultsThe sample permitted a 3.0% confidence level with acceptable internal consistency (Cronbach's alpha 0.755). Formative use was supported by 72.5% and summative use was rejected by almost as many (66.3%). WBA use was perceived to deviate markedly from that recommended by the Joint Committee on Surgical Training (JCST). Significant misuse was identified and elements perceived as inaccurate appear commonplace across the breadth of surgical specialties. Inaccurate completion was acknowledged by 89.6% of respondents and some trainers appear complicit, 147 individuals (16.2%) having reported this to trainers, 40.9% aware of ‘unobserved sign-off’, and 33.6% aware of ‘password disclosure’ by trainers. Furthermore, a majority of trainees felt the Annual Review of Competency Progression (ARCP) respected WBA quantity above quality (55.4%), and a third felt pressure to overstate the number completed (32.0%). Reasons for misuse appeared largely centred upon time restraints, lack of engagement and a will to achieve the required targets for career progression. 1.5 ConclusionsThis study demonstrates that UK surgical trainees perceive that most trainees deviate from guidance in their use of WBAs. This is worrying in both the apparent frequency and nature of misuse and somewhat undermines existing evidence for their role in surgical training. Trainees perceive that required numbers of WBAs are too high, that training programmes fail to encourage their use as formative assessments, and that there is a lack of engagement by many trainees and trainers. We present consensus recommendations from ASiT for the improvement of WBA use in UK surgical training.

Workplace-based assessments in postgraduate medical education: A hermeneutic review.

Since their introduction, workplace-based assessments (WBAs) have proliferated throughout postgraduate medical education. Previous reviews have identified mixed findings regarding WBAs' effectiveness, but have not considered the importance of user-tool-context interactions. The present review was conducted to address this gap by generating a thematic overview of factors important to the acceptability, effectiveness and utility of WBAs in postgraduate medical education. This review utilised a hermeneutic cycle for analysis of the literature. Four databases were searched to identify articles pertaining to WBAs in postgraduate medical education from the United Kingdom, Canada, Australia, New Zealand, the Netherlands and Scandinavian countries. Over the course of three rounds, 30 published articles were thematically analysed in an iterative fashion to deeply engage with the literature in order to answer three scoping questions concerning acceptability, effectiveness and assessment training. As each round was coded, themes were refined and questions added until saturation was reached. Stakeholders value WBAs for permitting assessment of trainees' performance in an authentic context. Negative perceptions of WBAs stem from misuse due to low assessment literacy, disagreement with definitions and frameworks, and inadequate summative use of WBAs. Effectiveness is influenced by user (eg, engagement and assessment literacy) and tool attributes (eg, definitions and scales), but most fundamentally by user-tool-context interactions, particularly trainee-assessor relationships. Assessors' assessment literacy must be combined with cultural and administrative factors in organisations and the broader medical discipline. The pivotal determinants of WBAs' effectiveness and utility are the user-tool-context interactions. From the identified themes, we present 12 lessons learned regarding users, tools and contexts to maximise WBA utility, including the separation of formative and summative WBA assessors, use of maximally useful scales, and instituting measures to reduce competitive demands.

Assessment methods in respiratory medicine training in Europe: current status and future needs.

There is a perceived need for harmonisation of training standards in medicine across Europe. Assessment methodology is a fundamental part of this harmonisation. Assessment may be: summative, measuring competency at the end of training, usually in a pass/fail mode; or formative, assessing strengths and weaknesses during training with a view to optimising performance. A survey of experienced respiratory medicine trainers in Europe was undertaken to discover the formative and summative assessment methods being used, and to explore the use of workplace-based assessments (WBAs). Structured interviews were sought with experienced trainers in adult and paediatric respiratory medicine. 35 trainers from 22 (71%) out of 31 countries were interviewed.The number of types of summative assessments required at the end of training varied from 0 to 4 including clinical (25%), written (44%) and oral (47%) examinations. Four respondents required a research thesis as a proof of clinical competence.WBA was not commonly used. Only 14% of respondents reported using a formal case-based discussion, 20% used a form of multisource feedback and 25% described some form of formal assessment of procedural skills. However, 77% of all respondents expressed a wish to have access to case-based discussion and 72% wanted to have a facility for multisource feedback. The majority also wanted training in the use of these tools. Almost half of the respondents had received no formal training in educational supervision and 80% of all respondents expressed a wish to receive such training.The findings suggest that there is no adequate process of ensuring uniform standards for specialist accreditation in Europe, and demonstrate a need and desire among trainers for more WBA tools and training in their use to be made available.

Myths and legends in workplace-based assessment; it’s all about perspective?

ABSTRACTBased on internationally published literature and the experience of the project team, we understood that already established beliefs and perceptions could influence the implementation of new workplace-based assessment (WBA) tools and processes. At the start of a project to design new WBA tools, we, therefore, attempted to understand the perspectives and perceptions of our GP training community and inform the communication strategy for the project. Online and paper surveys were disseminated to GP trainers, trainees and programme directors. Data were extracted from SurveyMonkey, added to paper-based data in an Excel spreadsheet and analysed using descriptive statistics. While it was generally acknowledged that trainers provide regular feedback to trainees, perceptions of the quality of feedback varied between the groups. The majority of participants agreed that WBA would be of value in learning but expressed concerns over increased workload. There were also some misconceptions about the purpose of WBA, with trainees stating they understood WBAs to be valid and reliable as a single-event sign-off tool. Differences in trainer, trainee and programme director perceptions of the use and usefulness of WBA may be attributed to many factors, not least of all (lack of) experience, perspective and position.

Implementation of workplace-based assessment in general practice

ABSTRACTBackground: Workplace-based assessment (WPBA) is widely accepted, but few studies have investigated implementation issues during general practice (GP) placements. This study explored possible barriers and identified key elements for successful implementation of a WPBA-programme in Danish GP specialist training.Methods: Supervisors had attended a one-day course in WPBA and trainees had received a short introduction. Questionnaires on experiences with implementation of WPBA were distributed to 106 GP supervisors and 110 trainees after the rotation was finished.Results: The response rate was 61/96 (64%) for trainees and 67/94 (71%) for supervisors. Supervisors were generally more positive towards WPBA and saw fewer barriers than trainees. Lack of planning was most often reported as an impediment to WPBA. Supervisors did not identify trainees’ uneasiness of being observed as a problem as often as trainees. A total of 34% of trainees reported uneasiness as an obstacle to WPBA.Conclusions: It seems that the education of supervisors positively influenced supervisors’ perception and use of WPBA. Adequate planning of WPBA may be just as big a problem as assigning the time. Further investigations on the impact of education on trainees’ perception of WPBA are needed.

Plus ça change, plus c’est pareil: Making a continued case for the use of MCQs in medical education

Despite the increased emphasis on the use of workplace-based assessment in competency-based education models, there is still an important role for the use of multiple choice questions (MCQs) in the assessment of health professionals. The challenge, however, is to ensure that MCQs are developed in a way to allow educators to derive meaningful information about examinees’ abilities. As educators’ needs for high-quality test items have evolved so has our approach to developing MCQs. This evolution has been reflected in a number of ways including: the use of different stimulus formats; the creation of novel response formats; the development of new approaches to problem conceptualization; and the incorporation of technology. The purpose of this narrative review is to provide the reader with an overview of how our understanding of the use of MCQs in the assessment of health professionals has evolved to better measure clinical reasoning and to improve both efficiency and item quality.

Exploring the Role of Self-Motives in Postgraduate Trainees' Feedback-Seeking Behavior in the Clinical Workplace: A Multicenter Study of Workplace-Based Assessments From the United Kingdom.

To explore trainees' feedback-seeking behavior in the postgraduate surgical workplace using a self-motives framework. Self-motives include self-assessment "to obtain accurate information about the self," self-improvement "to improve one's traits, abilities, and skills," self-enhancement "to enhance the favorability of self views," and self-verification "to maintain consistency between one's central self-view and new self-relevant information." This project constituted a further framework analysis of previously obtained qualitative focus group data that originally explored trainees' perceptions and use of workplace-based assessment (WBA). Data were collected from multiple centers in the United Kingdom from 2012 to 2013. Content was analyzed to identify references in the data that reflected the above self-motives and in relation to contextual themes identified from within the data. Trainees' motivations for seeking feedback broadly fit within a self-motives framework. Trainees' feedback seeking using WBA related to self-enhancement and self-verification, whereas outside WBA trainees reported self-improvement and self-assessment motives. Where trainees perceived WBA represented an opportunity to learn, they described a self-improvement motive toward seeking feedback, whereas when WBA represented an assessment of learning, trainees described tensions between self-enhancement and self-improvement motives. Surgical trainees' motivations for seeking feedback can be explained using a conceptual self-motives framework. Trainees need to be motivated to seek accurate informational feedback so they can improve their performance within the clinical workplace. To achieve this, trainees need training; current assessment systems must change to allow trainees to seek such feedback without fear and concern about this information being used as an assessment of learning.

McMaster Modular Assessment Program (McMAP) Through the Years: Residents' Experience With an Evolving Feedback Culture Over a 3-year Period.

Assessing resident competency in emergency department settings requires observing a substantial number of work-based skills and tasks. The McMaster Modular Assessment Program (McMAP) is a novel, workplace-based assessment (WBA) system that uses task-specific and global low-stakes assessments of resident performance. We describe the evaluation of a WBA program 3years after implementation. We used a qualitative approach, conducting focus groups with resident physicians in all 5 postgraduate years (n=26) who used McMAP as part of McMaster University's emergency medicine residency program. Responses were triangulated using a follow-up written survey. Data were analyzed using theory-based thematic analysis. An audit trail was reviewed to ensure that all themes were captured. Findings were organized at the level of the learner (residents), faculty, and system. Residents identified elements of McMAP that were perceived as supporting or inhibiting learning. Residents shared their opinions on the feasibility of completing daily WBAs, perceptions and utilization of rating scales, and the value of structured feedback (written and verbal) from faculty. Residents also commented extensively on the evolving and improving feedback culture that has been created within our system. The study describes an evolving culture of feedback that promotes the process of informed self-assessment. A programmatic approach to WBAs can foster opportunities for feedback although barriers must still be overcome to fully realize the potential of a continuous WBA system. A professional culture change is required to implement and encourage the routine use of WBAs. Barriers, such as familiarity with assessment system logistics, faculty member discomfort with providing feedback, and empowering residents to ask faculty for direct observations and assessments must be addressed to realize the potential of a programmatic WBA system. Findings may inform future research in identifying key components of successful implementation of a programmatic workplace-based assessment system.

Mind the Gap: The Prospects of Missing Data

The increasing use of workplace-based assessments (WBAs) in competency-based medical education has led to large data sets that assess resident performance longitudinally. With large data sets, problems that arise from missing data are increasingly likely. The purpose of this study is to examine (1) whether data are missing at random across various WBAs, and (2) the relationship between resident performance and the proportion of missing data. During 2012-2013, a total of 844 WBAs of CanMEDs Roles were completed for 9 second-year emergency medicine residents. To identify whether missing data were randomly distributed across various WBAs, the total number of missing data points was calculated for each Role. To examine whether the amount of missing data was related to resident performance, 5 faculty members rank-ordered the residents based on performance. A median rank score was calculated for each resident and was correlated with the proportion of missing data. More data were missing for Health Advocate and Professional WBAs relative to other competencies (P < .001). Furthermore, resident rankings were not related to the proportion of missing data points (r = 0.29, P > .05). The results of the present study illustrate that some CanMEDS Roles are less likely to be assessed than others. At the same time, the amount of missing data did not correlate with resident performance, suggesting lower-performing residents are no more likely to have missing data than their higher-performing peers. This article discusses several approaches to dealing with missing data.

A Descriptive Analysis of the Use of Workplace-Based Assessments in UK Surgical Training

Workplace-based assessments (WBAs) were introduced formally in the UK in 2007. The aim of the study was to describe the use of WBAs by UK surgical trainees and examine variations by training region, specialty, and level of training. The database of the Intercollegiate Surgical Curriculum Programme was examined for WBAs between August 2007 and July 2013, with in-depth analysis of 2 periods: August 2011 to July 2012 and August 2012 to July 2013. The numbers of validated WBAs per trainee per year increased more than 7-fold, from median 6 per trainee in 2007 to 2008, to 39 in 2011 to 2012, and 44 in 2012 to 2013. In the period 2011 to 2012, 58.4% of core trainees completed the recommended 40 WBAs, with only 38.1% of specialty trainees achieving 40 validated WBAs. In the period 2012 to 2013, these proportions increased to 67.7% and 57.0% for core and specialty trainees, respectively. Core trainees completed more WBAs per year than specialty trainees in the same training region. London core trainees completed the highest numbers of WBAs in both the periods 2011 to 2012 (median 67) and 2012 to 2013 (median 74). There was a peak in WBAs completed by London specialty trainees in the period 2012 to 2013 (median 63). The most validated WBAs were completed by ST1/CT1 (specialty surgical training year, core surgical training year), with a gradual decrease in median WBAs to ST4, followed by a plateau; in the period 2012 to 2013, there was an increase in WBAs at ST8. Core surgical trainees complete ~50% "operative" (procedure-based assessment/direct observation of procedural skills) and ~50% "nonoperative" assessments (case-based discussion/clinical evaluation exercise). During specialty training, procedure-based assessments represented ~46% of WBAs, direct observation of procedural skills 11.2%, case-based discussion ~23%, and clinical evaluation exercise ~15%. UK surgical trainees are, on an average, undertaking 1 WBA per week. Variation exists in use of WBAs between training regions. Core trainees tend to use the spectrum of WBAs more frequently than their senior colleagues do. Further work is required to examine the role of WBAs in assessment, and engagement and training of trainers in processes and validation of WBAs.

A BEME (Best Evidence in Medical Education) systematic review of the use of workplace-based assessment in identifying and remediating poor performance among postgraduate medical trainees.

BackgroundWorkplace-based assessments were designed to facilitate observation and structure feedback on the performance of trainees in real-time clinical settings and scenarios. Research in workplace-based assessments has primarily centred on understanding psychometric qualities and performance improvement impacts of trainees generally.An area that is far less understood is the use of workplace-based assessments for trainees who may not be performing at expected or desired standards, referred to within the literature as trainees ‘in difficulty’ or ‘underperforming’. In healthcare systems that increasingly depend on service provided by junior doctors, early detection (and remediation) of poor performance is essential. However, barriers to successful implementation of workplace-based assessments (WBAs) in this context include a misunderstanding of the use and purpose of these formative assessment tools.This review aims to explore the impact - or effectiveness - of workplace-based assessment on the identification of poor performance and to determine those conditions that support and enable detection, i.e. whether by routine or targeted use where poor performance is suspected. The review also aims to explore what effect (if any) the use of WBA may have on remediation or on changing clinical practice. The personal impact of the detection of poor performance on trainees and/or trainers may also be explored.Methods/designUsing BEME (Best Evidence in Medical Education) Collaboration review guidelines, nine databases will be searched for English-language records. Studies examining interventions for workplace-based assessment either routinely or in relation to poor performance will be included. Independent agreement (kappa .80) will be achieved using a randomly selected set of records prior to commencement of screening and data extraction using a BEME coding sheet modified as applicable (Buckley et al., Med Teach 31:282-98, 2009) as this has been used in previous WBA systematic reviews (Miller and Archer, BMJ doi:10.1136/bmj.c5064, 2010) allowing for more rigorous comparisons with the published literature. Educational outcomes will be evaluated using Kirkpatrick’s framework of educational outcomes using Barr’s adaptations (Barr et al., Evaluations of interprofessional education; a United Kingdom review of health and social care, 2000) for medical education research.DiscussionOur study will contribute to an ongoing international debate regarding the applicability of workplace-based assessments as a meaningful formative assessment approach within the context of postgraduate medical education.Systematic review registrationThe review has been registered by the BEME Collaboration www.bemecollaboration.org.

What Evidence is There for the Use of Workplace-Based Assessment in Surgical Training?

Recent years have seen broad changes in postgraduate training with a move toward structured formative and summative evaluation of trainees' competencies using workplace-based assessment (WBA) tools. The fitness for purpose of these instruments in surgery has been much debated. The aim of this study is to explore the evidence underlying the introduction and ongoing use of WBAs in surgical training. A critical literature review was conducted to identify studies evaluating the use of WBAs in postgraduate surgical training. The search was conducted using the electronic databases PubMed for full-text articles in English. Additional critical evaluations of the curriculum relating to WBAs were included. The articles were synthesized in a narrative review. The implementation of WBA requirements in surgical training has occurred despite a relative dearth of direct evidence of their efficacy and benefit. Studies and critical reviews are being regularly undertaken to ensure that supporting evidence is accrued and the drive for improvement and refinement is maintained. It is emerging that WBAs are (contrary to their current nomenclature) formative tools for feedback and hence learning. They can facilitate the progression toward expert practice at the center of the zone of proximal development and the higher levels of Miller's pyramid, but fall short--owing to their focus on competence--of guiding surgical trainees to the higher levels of Maslow's hierarchy. Limited evidence has potentially undermined the introduction of WBAs in surgical training to date. There are misunderstandings regarding their use as either summative or formative educational tools. These shortcomings are an opportunity for further work in examining WBAs in their current or modified form.

Competency based surgical training

With the implementation of the European working time directive and the “new deal” document in the NHS; not only led to a shorter duration of training but also increased the workload intensity on trainee doctors. This leads to a reduced opportunity for personal reflection and feedback from colleagues i.e., consultants, registrars, and fellow senior house officers that leading to a shift from the traditional experiential learning to “competency-based training,” where various workplace-based assessments (WBAs) tools are used. This article is about the use of WBAs in improving trainee doctor’s competence and workplace performance; explore why they are so unpopular amongst both trainees and trainers and what can be done to improve their work based learning.