Workplace Based Assessment Implementation Research Articles

Beyond Checking Boxes: Exploring Tensions With Use of a Workplace-Based Assessment Tool for Formative Assessment in Clerkships.

To understand the role of a workplace-based assessment (WBA) tool in facilitating feedback for medical students, this study explored changes and tensions in a clerkship feedback activity system through the lens of cultural historical activity theory (CHAT) over 2 years of tool implementation. This qualitative study uses CHAT to explore WBA use in core clerkships by identifying feedback activity system elements (e.g., community, tools, rules, objects) and tensions among these elements. University of California, San Francisco core clerkship students were invited to participate in semistructured interviews eliciting experience with a WBA tool intended to enhance direct observation and feedback in year 1 (2019) and year 2 (2020) of implementation. In year 1, the WBA tool required supervisor completion in the school's evaluation system on a computer. In year 2, both students and supervisors had WBA completion abilities and could access the form via a smartphone separate from the school's evaluation system. Thirty-five students participated in interviews. The authors identified tensions that shifted with time and tool iterations. Year 1 students described tensions related to cumbersome tool design, fear of burdening supervisors, confusion over WBA purpose, WBA as checking boxes, and WBA usefulness depending on clerkship context and culture. Students perceived dissatisfaction with the year 1 tool version among peers and supervisors. The year 2 mobile-based tool and student completion capabilities helped to reduce many of the tensions noted in year 1. Students expressed wider WBA acceptance among peers and supervisors in year 2 and reported understanding WBA to be for low-stakes feedback, thereby supporting formative assessment for learning. Using CHAT to explore changes in a feedback activity system with WBA tool iterations revealed elements important to WBA implementation, including designing technology for tool efficiency and affording students autonomy to document feedback with WBAs.

Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review.

Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions. The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders. In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain. Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.

Vocational Students Experience Workplace-Based Assessment: How Does It Work?

This case study is conducted to investigate the implementation of the Workplace Based Assessment in Pharmacy Vocational School. The research method used is R&D with observation, document analysis, and in-depth interviews as data collection methods. While for data analysis, it used the Realist Evaluation/Synthesis approach. The informants are students, teachers, competency assessors from 21 schools, the Directorate of PSMK, Chairperson of APMFI, and representatives from two pharmaceutical companies. The results showed that the Workplace Based Assessment was built based on the integration of learning processes in schools and industrial work practices. Workplace-Based Assessment is carried out by developing competency standards, learning at schools, and industrial work practices. BNSP has a significant role in developing work competency standards. Based on the findings of this study, the revitalization of vocational schools through the implementation of Workplace-Based Assessment is needed to foster vocational graduates who have great skill and competent in facing the Industrial Revolution 4.0 and challenges in the workplace.

Evaluation of Work Place Based Assessment implementation in Postgraduate medical education at a Medical University in Bhutan

Introduction: The Postgraduate Medical Education has witnessed transition from traditional cognitive based to more competency-based learning globally. Khesar Gyalpo University of Medical sciences of Bhutan introduced Competency Based Medical Education (CBME) through implementation of work place-based assessment (WPBA) in June 2018. The primary objective of this initiative was to produce specialist of highest quality and cultivate competency and outcome based, yet learner centered curricula. Methods: The evaluation was conducted in June, 2019. The mixed methods of data collection techniques were utilized such as survey, interview, and review of the documents and focus group discussion. It was to provide understanding of local challenges and needs in implementation of WPBA. Results: A total of 90% of the faculty members and 40% of administrators evaluated were aware of the implementation of WPBA. Majority of the faculty felt that WPBA is beneficial to both faculty and the residents and all residents felt that it’s beneficial in terms of learning. OBGYN residents have been exposed to maximum numbers of WPBA at 20. The maximum numbers of WPBA activities were performed by residents of general practice department which stood at 56. Lack of time as hindrance of practice of WPBA was implicated by 28% of the faculty and 61% residents. Conclusions: Despite WPBA being implemented for a short duration there is a high level of awareness and acceptability among both the residents and faculties as an effective teaching and learning tool.

Fostering teaching-learning through workplace based assessment in postgraduate chemical pathology residency program using virtual learning environment

BackgroundThe principle of workplace based assessment (WBA) is to assess trainees at work with feedback integrated into the program simultaneously. A student driven WBA model was introduced and perception evaluation of this teaching method was done subsequently by taking feedback from the faculty as well as the postgraduate trainees (PGs) of a residency program.MethodsDescriptive multimethod study was conducted. A WBA program was designed for PGs in Chemical Pathology on Moodle and forms utilized were case-based discussion (CBD), direct observation of practical skills (DOPS) and evaluation of clinical events (ECE). Consented assessors and PGs were trained on WBA through a workshop. Pretest and posttest to assess PGs knowledge before and after WBA were conducted. Every time a WBA form was filled, perception of PGs and assessors towards WBA, time taken to conduct single WBA and feedback were recorded. Faculty and PGs qualitative feedback on perception of WBA was taken via interviews. WBA tools data and qualitative feedback were used to evaluate the acceptability and feasibility of the new tools.ResultsSix eligible PGs and seventeen assessors participated in this study. A total of 79 CBDs (assessors n = 7 and PGs n = 6), 12 ECEs (assessors n = 6 and PGs n = 5), and 20 DOPS (assessors n = 6 and PGs n = 6) were documented. PGs average pretest score was 55.6%, which was improved to 96.4% in posttest; p value< 0.05. Scores of annual assessment before and after implementation of WBA also showed significant improvement, p value 0.039, Overall mean time taken to evaluate PG’s was 12.6 ± 9.9 min and feedback time 9.2 ± 7.4 min. Mean WBA process satisfaction of assessors and PGs on Likert scale of 1 to 10 was 8 ± 1 and 8.3 ± 0.8 respectively.ConclusionBoth assessors and fellows were satisfied with introduction and implementation of WBA. It gave the fellows opportunity to interact with assessors more often and learn from their rich experience. Gain in knowledge of PGs was identified from the statistically significant improvement in PGs’ assessment scores after WBA implementation.

Quality assessment in undergraduate medical training: how to bridge the gap between what we do and what we should do.

Introductionthe outcome of the undergraduate medical training programme in South Africa is to produce competent medical doctors who can integrate knowledge, skills and attitudes relevant to the South African context. Training facilities have a responsibility to ensure that they perform this assessment of competence effectively and defend the results of high-stakes assessments. This study aimed to obtain qualitative data to suggest practical recommendations on best assessment practices to address the gaps between theoretical principles that inform assessment and current assessment practices.Methodsa focus group interview was used to gather this data. The teaching and learning coordinators for five of the six modules that are offered in the clinical phase of the undergraduate medical programme participated in the focus group interview. The focus group interview proceeded as planned and took 95 minutes to complete. The responses were transcribed and recorded on a matrix.Resultsthe lack of formal feedback to students was identified as an area of concern; feedback plays an important role to promote student learning and improve patient care. The role of teaching and learning coordinators as drivers of quality assessment were recognized and supported. All participants agreed on the outcome of the programme and the central role of the outcome in all assessments.Conclusionthe training of assessors and the implementation of workplace-based assessment and assessment portfolios were recommended and can also address feasibility challenges. Participants recommended decreasing summative assessments and only performing these for borderline students.

Implementation of workplace-based assessment in general practice

ABSTRACTBackground: Workplace-based assessment (WPBA) is widely accepted, but few studies have investigated implementation issues during general practice (GP) placements. This study explored possible barriers and identified key elements for successful implementation of a WPBA-programme in Danish GP specialist training.Methods: Supervisors had attended a one-day course in WPBA and trainees had received a short introduction. Questionnaires on experiences with implementation of WPBA were distributed to 106 GP supervisors and 110 trainees after the rotation was finished.Results: The response rate was 61/96 (64%) for trainees and 67/94 (71%) for supervisors. Supervisors were generally more positive towards WPBA and saw fewer barriers than trainees. Lack of planning was most often reported as an impediment to WPBA. Supervisors did not identify trainees’ uneasiness of being observed as a problem as often as trainees. A total of 34% of trainees reported uneasiness as an obstacle to WPBA.Conclusions: It seems that the education of supervisors positively influenced supervisors’ perception and use of WPBA. Adequate planning of WPBA may be just as big a problem as assigning the time. Further investigations on the impact of education on trainees’ perception of WPBA are needed.

Workplace-based assessment and students’ approaches to learning: a qualitative inquiry

Background: We have performed this research to assess the effect of work-place based assessment (WBA) practice on medical students’ learning approaches.Methods: The research was conducted at the King Saud bin Abdulaziz University for Health Sciences, College of Medicine from 1 March to 31 July 2012. We conducted a qualitative, phenomenological research utilizing semi-structured individual interviews with medical students exposed to WBA. The audio-taped interviews were transcribed verbatim, analyzed, and themes were identified. We preformed investigators' triangulation, member checking with clinical supervisors and we triangulated the data with a similar research performed prior to the implementation of WBA.Results: WBA results in variable learning approaches. Based on several affecting factors; clinical supervisors, faculty-given feedback, and assessment function, students may swing between surface, deep and effort and achievement learning approaches. Students’ and supervisors’ orientations on the process of WBA, utilization of peer feedback and formative rather than summative assessment facilitate successful implementation of WBA and lead to students’ deeper approaches to learning. Interestingly, students and their supervisors have contradicting perceptions to WBA.Conclusion: A change in culture to unify students’ and supervisors’ perceptions of WBA, more accommodation of formative assessment, and feedback may result in students’ deeper approach to learning.

Workplace-based assessment for general practitioners: using stakeholder perception to aid blueprinting of an assessment battery

The implementation of an assessment system may be facilitated by stakeholder agreement that appropriate qualities are being tested. This study investigated the extent to which stakeholders perceived 8 assessment formats (multiple-choice questions, objective structured clinical examination, video, significant event analysis, criterion audit, multi-source feedback, case analysis and patient satisfaction questionnaire) as able to assess varying qualities of doctors training in UK general practice. Educationalists, general practice trainers and registrars completed a blueprinting style of exercise to rate the extent to which each evaluation format was perceived to assess each of 8 competencies derived primarily from the General Medical Council document 'Good Medical Practice'. There were high levels of agreement among stakeholders regarding the perceived qualities tested by the proposed formats (G = 0.82-0.93). Differences were found in participants' perceptions of how well qualities were able to be assessed and in the ability of the respective formats to test each quality. Multi-source feedback (MSF) was expected to assess a wide range of qualities, whereas Probity, Health and Ability to work with colleagues were limited in terms of how well they could be tested by the proposed formats. Awareness of the perceptions of stakeholders should facilitate the development and implementation of workplace-based assessment (WPBA) systems. These data shed light on the acceptability of various formats in a way that will inform further investigation of WPBA formats' validity and feasibility, while also providing evidence on which to base educational efforts regarding the value of each format.