Direct Observation Of Procedural Skills Research Articles

Curriculum for small-bowel capsule endoscopy and device-assisted enteroscopy training in Europe: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement.

The European Society of Gastrointestinal Endoscopy (ESGE) has recognized the need to formalize training in small-bowel endoscopy across European centers. The following criteria and framework for training in small-bowel capsule endoscopy (SBCE) and device-assisted enteroscopy (DAE), which aim to provide uniform and high quality training to ensure that small-bowel endoscopists are competent to operate independently, are based on the current literature and experience of experts in the field. Three main areas are covered: skills required prior to commencing training in small-bowel endoscopy; structured training for trainees to become independent endoscopists; and ways of ensuring competence is achieved. 1 : Centers providing training in SBCE should perform a minimum of 75 - 100 SBCEs/year. 2 : Experience in bidirectional endoscopies is desirable for structured training in SBCE. 3 : SBCE courses should consist of at least 50 % hands-on training and cover information on technology, indications and contraindications for SBCE, pathologies that can be encountered on SBCE, and standard terminology that should be used during reporting of SBCE. An SBCE course should be completed prior to achieving competence in SBCE reporting. 4 : Competence in SBCE can be assessed by considering a minimum of 30 SBCEs. Direct Observation of Procedural Skills, short SBCE videos, and multiple-choice questions can be useful to assess improvement in the skills of trainees. 5 : Centers offering training in DAE should aim to carry out at least 75 DAEs/year, should have direct links with an SBCE service, and should allow regular discussion of cases at a radiology small-bowel MDT. Training centers with lower numbers are encouraged to offer training by "buddying-up" with other centers, or using mentoring systems. 6 : DAE trainees must be independent in bidirectional endoscopies and have experience in level 1 polypectomy prior to commencement of training. They should also be competent in reviewing SBCEs. 7 : Training in DAE should be structured with a minimum of 75 procedures, including 35 retrograde DAEs, with therapeutic procedures undertaken in at least 50 % of the DAEs performed. Training should cover the indications, contraindications, complications including prevention, and technicalities of the DAE procedure; formal evaluation should follow. DAE trainees must acquire skills to independently manage and advise on small-bowel pathology following DAE procedures. 8 : It is highly recommended that international societies develop online modules and courses on DAE, which are currently lacking across Europe.

THU0586 ESTABLISHING THE KEY COMPONENTS OF A EULAR PORTFOLIO FOR TRAINING IN RHEUMATOLOGY: A EULAR SCHOOL OF RHEUMATOLOGY INITIATIVE

Background:In clinical training, a portfolio is expected to stimulate learning and encourage critical reflection. Some, but not all, European countries use a portfolio in rheumatology training, and their scope varies widely. A EULAR portfolio for Rheumatology trainees could contribute to improve overall training, raise educational standards, foster the setting of common goals and harmonize rheumatology training across countries.Objectives:Develop key components that should be included in a EULAR portfolio of Rheumatology.Methods:A working group (WG) composed of 9 rheumatologists and 1 educationalist was established. A systematic literature review (SLR) was conducted in November 2018, according to the PIM structure: Population: trainees; Instrument of interest: portfolio; Measurement of properties of interest: content portfolio. A survey was disseminated among the WG group and WG members of the EMerging EUlar NETwork (EMEUNET), inquiring about the content and structure of existing national portfolios. Portfolio materials of selected countries were reviewed. Last, the WG elected the key components of the portfolio.Results:13/2,034 articles were included in the SLR (12 high/1 moderate risk of bias). Information on direct observation of procedural skills (DOPS) (9/13), personal reflections (8/13), learning goals (5/13) and multisource feedback (5/13) were most often included in the portfolio. Twenty-five respondents filled out the survey (response rate ≈ 50%). Reflective writing (n=7), learning goals (n=4) and feedback (n=4) were considered the most useful components of a portfolio. About half indicated that a portfolio was a bureaucratic burden; 4 respondents mentioned lack of feedback by supervisors as a barrier. Portfolio materials of 7 European countries were reviewed. Several portfolios (Germany, Italy, Greece and Spain) were logbooks, i.e. a record of clinical activities. Other portfolios (UK, Denmark, The Netherlands) also included information on workplace-based assessments, learning goals, and personal reflections. The proposed key components of the portfolio are included in Table 1.Table 1.Key components of the EULAR portfolio of Rheumatology.Key componentContentCurriculum vitaePersonal record of achievements, experiences, knowledge and skillsPersonal Development PlanLearning goals and action planClinical workInformation on managing patients (e.g. rheumatoid arthritis)Skills (e.g. joint aspiration)Assessments (summative and formative)Personal reflectionsProfessional behaviourMultisource feedbackPersonal reflectionsEducationContinuing professional development, list of formal and non-formal learning activitiesAssessments (e.g. teaching assessment, evidence based medicine assignment)Personal reflectionsResearchList of abstracts, published articlesInformation on research funding, scholarships, bursaries, academic postsConclusion:This initiative resulted in the establishment of a list of key components to be included in a EULAR portfolio of Rheumatology. Assessment forms for each key portfolio component are currently being developed. Portfolio implementation, particularly in countries that do not use it yet, may contribute significantly to promote a higher standard of patient care across Europe.Disclosure of Interests:Marloes van Onna: None declared, Sofia Ramiro Grant/research support from: MSD, Consultant of: Abbvie, Lilly, Novartis, Sanofi Genzyme, Speakers bureau: Lilly, MSD, Novartis, Catherine Haines: None declared, M. Holland-Fischer: None declared, José Antonio P. da Silva Grant/research support from: Pfizer, Abbvie, Consultant of: Pfizer, AbbVie, Roche, Lilly, Novartis, Jean Dudler: None declared, Christopher Edwards Grant/research support from: Abbvie, Biogen, Roche, Consultant of: Abbvie, Samsung, Speakers bureau: Abbvie, BMS, Biogen, Celgene, Fresenius, Gilead, Janssen, Lilly, Mundipharma, Pfizer, MSD, Novartis, Roche, Samsung, Sanofi, UCB,Alessia Alunno: None declared, Elena Nikiphorou: None declared, L. Falzon: None declared, Francisca Sivera: None declared

Factors contributing to students’ satisfaction with direct observation of procedural skills in the school of nursing sciences at the University of Zambia

Direct observation of procedural skills is an evidence-based assessment tool used for assessing competence in the practical procedures that nursing students undertake during clinical placement. Having knowledge about students’ satisfaction with their educational issues is very important as it helps both faculty and students to achieve educational goals. However factors that may influence student satisfaction with this method of assessment are not known in the school of nursing sciences at the University of Zambia. The purpose of this study was to investigate factors influencing student satisfaction with direct observation of procedural skills in order to get students views on this assessment tool. A cross sectional quantitative survey was used on ninety two (92) conveniently sampled final year undergraduate nursing students. Data were collected using a validated self-reported questionnaire and analysed using IBM SPSS Version 20. Fisher’s exact tests were used to determine association between student satisfaction and the independent variables. A p-value of ≤ .05 was considered statistically significant. Major findings revealed that poor clinical environment 98.9% and faculty non availability 98% influenced student’s satisfaction negatively. Other significant associations where clinical experiences, and feedback, with p ≤ .05. In conclusion, the major factors that influenced student satisfaction included poor clinical environment which was not conducive for assessments as it lacked many essential medical surgical supplies and non-availability of faculty in the clinical area who were needed to provide feedback, guidance and supervision to the students. Improving the clinical environment, ensuring faculty availability in order to provide timely and constructive feedback may help students achieve their learning objectives. Direct observation of procedural skills still remains an appropriate tool to assess student clinical competence. However, further research and development of cheap, less stressful and economic methods of clinical evaluation to blend with this tool is required in the school of nursing sciences.

Effectiveness of Laparoscopic Skills Workshop on Enhancing Knowledge and Skills of Surgical Residents and Its Comparison with DOPS (Direct Observation of Procedural Skills) Scores: Prospective Cohort Study.

Simulation-based surgical skills workshops are commonly used in the surgical training programs to enhance the knowledge and psychomotor skills of the residents, but there is scarcity of objective data on their effectiveness. The aim of our study was to explore if a 1-day laparoscopic skills workshop enhanced the knowledge and skills of surgical residents in minimal access surgery and if it had any correlation with the Direct Observation of Procedural Skills (DOPS) scores. This was a quasi-experimental, single-arm, repeat measure, prospective cohort study design. Thirty-three residents from general surgery and obstetrics/gynecology programs of Aga Khan University participated in the study. A daylong laparoscopic skills workshop consisting of lectures, videos, and hands-on practice was offered. The pre-workshop knowledge and skills scores of residents were assessed and compared with immediate post-workshop scores and scores of repeat assessment at 2months. The scores of DOPS on laparoscopic procedures before and after the workshop were also compared. The results of our study indicated that mean post-workshop (p < 0.001) and interval scores (p < 0.001) for both cognitive and psychomotor skills were significantly higher compared with mean pre-workshop scores. Similarly, post-workshop DOPS scores were significantly higher (p < 0.011) compared with pre-workshop DOPS scores. One-day laparoscopic skills workshop resulted in significant improvement in knowledge and psychomotor skills of the surgical residents. The skills gained from the workshop also resulted in improvement of DOPS scores reflecting the transfer of skills to real-life performance. ClinicalTrials.gov registration number: NCT03982927.

To evaluate the effect of small-group demonstration–based training module in improving clinical skills in undergraduate medical students

BackgroundIn today's modern era, the two most important aspects of medical education are evaluation- and outcome-based learning. Directly Observed Procedural Skills (DOPSs ) is a well-known method of evaluation which constitutes a direct observation of the practical skills being performed by the student and simultaneous written feedback by the teacher. MethodA total of 40 undergraduate students were taught by a module based on DOPS. A pre-test and post-test was conducted on DOPS examination pattern and was compared by the Cochran's Q test. ResultsIn this study, a total of 40 medical undergraduate students and 10 teachers participated. Each student was given one pre-test and five post-test with ultrasound proven mild to moderate splenomegaly. Each student was individually assessed on a module based on DOPS and was given direct feedback by the teachers. A total of 34 (85%) students strongly agreed that they felt comfortable and confident with this methodology. ConclusionThe results of this study revealed that DOPS tests can be used as an effective evaluation method to assess medical students because of its appropriate validity and reliability, positive impact on learning, and high satisfaction level amongst students. However, special attention needs to be given to the quality of these tests.

Comparing Short Clinical Evaluation Exercise and Direct Observation of Procedural Skills With the Traditional Evaluation Approach on the Clinical Skills of Forensic Medicine Residents

Background: The development of clinical teaching necessitates the use of novel and appropriate clinical evaluation methods. In the meantime, the use of new evaluation approaches that enhance learning at the same time has been recommended. This study aimed to compare the effect of the two new evaluation methods of direct observation of procedural skills (DOPS) and short clinical evaluation exercise (Mini-CEX) with the conventional evaluation method on clinical skills of forensic medicine residents.Methods: This is a randomized trial performed among forensic medicine residents. Using Cochran's formula, the minimum sample size was calculated to be 25 individuals per group. All the residents were randomly divided into either the experimental or the control groups. After training and evaluation, the experimental group was tested using the DOPS and Mini-CEX methods three times during the course, and the control group was evaluated using the conventional method. The basis for the evaluation of the procedures (physical examination and autopsy) in both groups was the valid and reliable checklists prepared by the researchers. The results were analyzed using descriptive and inferential statistics (e.g., the Chi-square and independent t-test).Results: There was no significant difference between the two groups in terms of demographic variables such as age, sex, and grade point average (GPA) (P&gt;0.06). The means of total scores were significantly higher in the experimental group (using the Mini-CEX and DOPS methods) than the control group (conventional method) (P&lt;0.0002).Conclusion: The use of new evaluation methods of DOPS and Mini-CEX improved the clinical skills of forensic medicine residents. Therefore, it is recommended that resident training centers use these two methods of assessment, together with other methods, to evaluate clinical procedures and boost forensic residents' learning.

The impact of using Direct observation of procedural skills (DOPS ) evaluation method on the some clinical skills of midwifery students

Background: The present study was done in order to evaluate the students’ clinical skill, enhance the quality of cares given, and consider limitations of common evaluation ways, as well as loss of texts about impact of new evaluation in education. The aim of this study was the impact of using direct observation of procedural skill (DOPS) evaluation method on some clinical skills of midwifery students. Methods: This quasi-experimental study was performed on 70 midwifery students in the delivery ward of Fatemieh Hospital in Hamadan during the first semester of 2017-2018. In this stage, the students in the control group were evaluated according to the current method of the faculty and the experimental group was evaluated by the DOPS method through a checklist. Finally, the evaluation scores of students were compared in both control and experimental groups. Results: According to T-test results, the average score in vaginal examination, Leopold maneuver and hearing fetal heart rate has been equal before doing the test (P>0.05); however, after intervention the difference between the results obtained from these two groups was significant (p 0.05), the scores have significantly increased in test group (p <0.05). Conclusion: Findings showed that using DOPSevaluation way can be effective in enhancing clinical skills of midwifery students in vaginal examination; however, Leopold maneuver and hearing fetal heart rate are recommended for other procedures.

A272 THE CANADIAN DIRECT OBSERVATION OF PROCEDURAL SKILLS (CANDOPS) TOOL FOR ENDOSCOPIC RETROGRADE CHOLANGIOPANCREATOGRAPHY: A MULTI-CENTRE PROSPECTIVE STUDY

Abstract Background Previous studies have demonstrated that many graduating trainees may not have all of the skills needed to independently practice endoscopic retrograde cholangiopancreatography (ERCP) safely and effectively. As a part of competency-based learning curriculum development, it is essential to provide formative feedback to trainees and ensure they acquire the knowledge and skills for independent practice. Aims To assess the performance of advanced endoscopy trainees across Canada using the Canadian Direct Observation of Procedural Skills (CanDOPS) ERCP assessment tool. Procedural items evaluated include both technical (cannulation, sphincterotomy, stone extraction, tissue sampling, and stent placement) and non-technical (leadership, communication and teamwork, judgment and decision making) skills. Methods We conducted a prospective national multi-centre prospective study. Advanced endoscopy trainees with at least two years of gastroenterology training or five years of general surgery in North America and minimal experience performing ERCPs (less than 100 ERCP procedures) were invited to participate. The CanDOPS tool was used to measure every fifth ERCP performed by trainees over a 12-month fellowship training period. ERCPs were evaluated by experienced staff endoscopists at each study site under standard clinical protocol. Cumulative sum (CUSUM) analyses were used to generate learning curves. Results The data from five Canadian sites and 11 trainees participated in the study. A total of 261ERCP evaluations were completed. Median number of evaluations by site and trainee was 49 (IQR 31–76) and 15 (IQR 11–45). Median number of cases trainees performed prior to their ERCP training was 50 (IQR 25–400). There was a significant improvement in almost all scores over time, including selective cannulation, sphincterotomy, biliary stenting and all non-technical skills (P&amp;lt;0.01). CUSUM analyses using acceptable and unacceptable failure rates of 20% and 50% demonstrated trainees achieved competency for most measures in their final month of their training. Competency in tissue sampling was not achieved within a one-year training period. Conclusions This is the first ERCP performance evaluation tool that examines multiple technical and non-technical aspects of the procedure. Although trainee ERCP skills do improve during their training period, there exists a notable variability in time to competency for the different skills measured using the CanDOPS tool. Large prospective research is required to determine if competency is achieved using more stringent definitions of ERCP competency and to determine factors associated with reaching competency. Funding Agencies None

Effect of Feedback on Direct Observation of Procedural Skills in Estimation of RBC Count among First Year Medical Students

Background: Formative evaluation has been effective in promoting active learning and a better futureperformance in students after immediate personalized feedback.Aim and Objectives: To analyze the effect of “Direct observation of procedural skills” (DOPS) assessmentand immediate feedback for first year MBBS students in estimation of RBC Count.Methodology: The study was done on first year MBBS students (n=120) during the Physiology practicalclass. Checklist for estimation of RBC count (15 steps) was prepared by six faculty members. The formativepractical test was divided into two practical sessions. During the first session, each faculty observed 4students performing RBC estimation one by one (DOPS) and the details were recorded in the checklist.Similarly in five practical classes 120 participants were evaluated and immediate feedback was given tothem for two minutes. In the second session the same procedure was followed and results were compared.Students perception about the assessment process (DOPS) was documented by feedback questionnaire asper Likert scale.Results: The mean practical marks of the participants before feedback was 8.32 ± 2.25 and after feedbackwas 10.18 ± 1.81 (P<0.05). 92% of the study participants felt DOPS in RBC estimation was useful to themConclusion: DOPS assessments worked well in the student’s practical laboratory setting by serving as aneffective educational tool. Immediate personalized feedback to the students has resulted in reasonable skilldevelopment both subjectively and objectively.

Non‐technical skills curriculum incorporating simulation‐based training improves performance in colonoscopy among novice endoscopists: Randomized controlled trial

Non-technical skills (NTS), involving cognitive, social and interpersonal skills that complement technical skills, are important for the completion of safe and efficient procedures. We investigated the impact of a simulation-based curriculum with dedicated NTS training on novice endoscopists' performance of clinical colonoscopies. A single-blinded randomized controlled trial was conducted at a single center. Novice endoscopists were randomized to a control curriculum or a NTS curriculum. The control curriculum involved a didactic session, virtual reality (VR) simulator colonoscopy training, and integrated scenario practice using a VR simulator, a standardized patient, and endoscopy nurse. Feedback and training were provided by experienced endoscopists. The NTS curriculum group received similar training that included a small-group session on NTS, feedback targeting NTS, and access to a self-reflective NTS checklist. The primary outcome was performance during two clinical colonoscopies, assessed using the Joint Advisory Group Direct Observation of Procedural Skills (JAG DOPS) tool. Thirty-nine participants completed the study. The NTS group (n = 21) had superior clinical performance during their first (P < 0.001) and second clinical colonoscopies (P < .0.001), compared to the control group (n = 18). The NTS group performed significantly better on the VR simulator (P < 0.05) and in the integrated scenario (P < 0.05). Our findings demonstrate that dedicated NTS training led to improved performance of clinical colonoscopies among novices.

Assessment of suitability of direct observation of procedural skills among postgraduate students and faculty in periodontology and implantology

Purpose: Direct observation of procedural skills (DOPS) is widely used on the basis of workplace-based assessments, which consist of direct observation and feedback. The DOPS covers all related professional qualities including knowledge, clinical perception, communication skills, ethics, the rights of patients, and the speed and accuracy of the task. However, the use of DOPS in dental treatment, especially in the field of periodontics is limited. Objective: The aim of the study was to find the utility of DOPS as assessment tool for the periodontal procedure. Methodology: The departmental faculty and postgraduate (PG) students were sensitized to the DOPS concept. A total of 12 PG students performed the periodontal procedure and were assessed on the basis of the DOPS rating scale for the 4 encounters each. The student's clinical skills were assessed by faculty members and then both students and faculty members were asked to give feedback for the procedure and overall experience. Results: Faculty and students both were comfortable in giving the feedback. Improvements in the knowledge of different clinical skills like relevant anatomy, technique of procedure were observed from 1st encounter to 4th encounter from 70% to 73.33%. Taking informed consent was improved from 53.33% to 90% from 1st encounter to 4th. Moreover, it helped the student to develop and improve their clinical skills, patient examination, and decision-making and treatment. Conclusion: The present study finding shows that faculty and students both were comfortable in giving the feedback and it helped the student to develop and improve their clinical skills. Our experience indicates that DOPS is an appropriate and functional tool in PG setup to enhance the clinical competencies.

The effect of direct observation of procedural skills method on learning clinical skills of midwifery students of medical sciences.

INTRODUCTION:Clinical education is one of the most important parts of medical students' education, and it is a major part of the education of qualified and professional people. Therefore, this study was conducted to determine the effect of applying Direct Observation of Procedural Skills (DOPS) on midwifery students' clinical skills.MATERIALS AND METHODS:This is a quasi-experimental, two-group study conducted as a pre- and post-study on midwifery students in 2017–2018. Cluster and randomized sampling method was used. The processors involved in this study were three main skills of vaginal examination, pelvic examination, and vaginal delivery. The DOPS method was used to assess the practical skills in the interventional group during three times the process at day, 1 day, and at least 1 week later; the usual logbook method was used in the control group. Two groups were evaluated at the end of the midwifery course by Comprehensive Final Midwifery checklist. The tools were checked for validity and reliability, and data were analyzed using descriptive and analytical statistics.RESULTS:There was no statistically significant difference between the two groups in terms of important demographic variables such as age, grade, marital status, and initial assessment score (P > 0.05). The mean of final scores in the normal delivery, vaginal examination, and pelvimetry was statistically significantly higher in the interventional group (P < 0.001). On the other hand, the functional field of the students in the interventional group was statistically significantly improved in normal delivery and pelvimetry (P < 0.05), and this difference was not significant in the vaginal examination. In addition, the mean scores of students before and after the DOPS method were statistically significantly different in every skill in Comprehensive Final Midwifery checklist (P < 0/05).CONCLUSIONS:The DOPS assessment methodology is not only a useful tool of clinical evaluation, but also an effective tool for clinical learning of students. For this purpose, it is suggested that educational members of midwifery take enough time to design DOPS method in the same process.

Formative assessment in practical for Indian postgraduates in health professions education: A strategic initiative towards competency-based education.

Background:Observational assessment (OA) or Workplace Based Assessment (WPBA) is being increasingly used to assess the trainees by direct observation to shape their learning. The current deficiencies in our conventional assessment system are due to lack of conceptualization of assessment as a process for continuous improvement and learning, leading to non-utilization of many available tools of assessment.Objective:The present study aimed to introduce formative assessment (FA) for postgraduates at all the constituent colleges (Medical, Dental, Ayurveda, Nursing) under the ambit of Datta Meghe Institute of Medical Sciences (Deemed University) Sawangi (M) Wardha, (Maharashtra), India.Methodology:Observational Assessment tools used for FA were Mini CEX (Mini Clinical Evaluation Exercise) and DOPS (Direct Observation of Procedural Skills)in addition to Objectively Structured Clinical/Practical Examination (OSCE/OSPE). Six encounters of OAs were conducted at the end of each semester. The OA tool used was either the MiniCEX or/and DOPS, depending on the type of subject (Preclinical, Para-clinical, Clinical).Results:A significant improvement in scores from one semester to other as the postgraduate learner progresses, was observed. The overall response from the postgraduate students and faculty for FA was satisfactory.Conclusion:Formative assessment in practical's, was feasible, acceptable and effective program to improve the overall learning and competency of postgraduates.

Effectiveness of simulation based teaching of ventilatory management among non-anaesthesiology residents to manage COVID 19 pandemic - A Quasi experimental cross sectional pilot study.

Background and Aims:Simulation is one of the important learning tools when it comes to skill acquisition and as a supplemental tool for training in high stake situations like COVID-19. The aim of this study is to meet the global requirements of knowledge on ventilatory management, prepare and to evaluate the effectiveness of the teaching module for non-anesthesiology trainees on COVID-19 patients.Methods:Quasi experimental cross sectional pilot study was conducted with a sample of twenty-six trainees. A teaching module was prepared and validated which consisted of lectures, audio-video sessions, demonstrations with hands-on training, debriefing, analytical-phase and reflection. Pre and Post evaluations from student t-test and direct observation of procedural skills (DOPS) were used for knowledge and skill assessment respectively and feedback obtained from Likert's score.Results:Pre- and Post-tests had a mean score of 7.42 ± 2.12 and 14.92 ± 2.9 respectively (P value 0.00001). DOPS included 16 point score, in which 23 trainees (88.4%) met the expectations and above expectations as per training objectives. A five-point Likert's score feedback revealed satisfactory and highly satisfactory scores of 100% (ABG), 96.1% (mechanical ventilation), and 84.6% (ventilation in COVID-19 patients). Overall satisfaction for the workshop among respondents was 100 per cent. Confidences of handling scores were 84.5% (interpreting ABG), 65.3% (maneuvering mechanical ventilation), and 96.15% (intubation in COVID-19 patients).Conclusion:A planned teaching module in ventilation management helps to train non-anaesthesiologists more effectively as a part of COVID-19 preparedness. Simulation with debriefing based training to the medical fraternity is the best alternative in the present pandemic and it will also ensure the safety of health care professionals.

Work-Based Assessments in Higher General Surgical Training Program: A Mixed Methods Study Exploring Trainers' and Trainees' Views and Experiences.

Introduction In the United Kingdom, work-based assessments (WBAs) including procedure-based assessments (PBAs), case-based discussions (CBDs), clinical evaluation exercises (CEXs), and direct observation of procedural skills (DOPS) have been used in Higher General Surgical Training Program (HGSTP) since the introduction of Modernising Medical Careers. Although the Intercollegiate Surgical Curriculum Project states that they should be used for the formative development of trainees using feedback and reflection, there is no study to look at the perception of their usefulness and barriers in using them, particularly in HGSTP. The aim of this study is to investigate trainer's and trainee's perception of their usefulness, barriers in using them, and way forward for their improvement in HGSTP. Methods This was a mixed method study. In phase I, after ethics committee approval, an online survey was sent to 83 trainers and 104 trainees, with a response rate of 33 and 37%, respectively, using Online Surveys (formerly Bristol Online Survey) from July 2018 to December 2018. After analysis of this result, in phase II, semistructured interviews were conducted with five trainees and five trainers who had volunteered to take part from phase I. Thematic analysis was performed to develop overarching themes. Results For professional formative development, 15% of the trainers and 53% of the trainees felt that WBAs had a low value. Among 4 WBAs—CEX, CBD, PBA, and DOPS—PBA was thought to be the most useful WBA by 52% trainers and 74% trainees. More trainers than trainees felt that it was being used as a formative tool (33 vs. 16%). The total number of WBAs thought to be required was between 20 and 40 per year, with 46% of the trainers and 53% of the trainees preferring these numbers.The thematic analysis generated four themes with subthemes in each: theme 1, “factors affecting usefulness,” including the mode of validation, trainer/trainee engagement, and time spent in validating; theme 2, “doubt on utility” due to doubt on validity and being used as a tick-box exercise; theme 3, “pitfalls/difficulties” due to lack of time to validate, late validation, e-mail rather than face-to-face validation, trainer and trainee behavior, variability in feedback given, and emphasis on number than quality; and theme 4, “improvement strategies.” Conclusions The WBAs are not being used in a way they are supposed to be used. The perception of educational impact (Kirkpatrick levels 1 and 2) by trainers was more optimistic than by trainees. Improvements can be made by giving/finding more time, trainer training, more face-to-face validation, and better trainer trainee interactions.

Internal Assessment in New MBBS Curriculum: Methods and Logistics.

Assessment is a pivotal element of competency-based curriculum. The implementation of competency-based undergraduate medical curriculum in India requires proper implementation of updates in assessment for which universities, colleges, and teachers need to plan and design internal assessment (IA) modules as well as guidelines. IA provides opportunity to assess many competencies and hence should be implemented judiciously. Multiple assessment methods should be used to improve utility of IA. The process should involve all the teachers of a subject and all competencies. Capacity building trainings should be organized by institutes in basic concepts of assessment as well as training in methods such as the objective structured clinical/practical examination, direct observation of procedural skills, and mini-clinical evaluation exercises. The culture of providing regular feedback needs to be instilled at institute levels. The learners who are not able to achieve competencies and required criteria in university examination should be provided predecided remedial measures for improving their performance. The article discusses all these aspects in detail.

Validating written feedback in clinical formative assessment

Formative assessment is widely accepted as being crucial to promoting student learning and, since 2010, the UK General Medical Council has mandated its use in workplace-based clinical training for all new doctors. As a result, the Royal College of Radiologists (RCR) instituted a range of formative workplace-based assessments including the Radiology Direct Observation of Procedural Skills (Rad-DOPS), in which supervisors appraise trainees’ performance in carrying out clinical procedures. This paper reports on the quality of the written feedback in 2,500 Rad-DOPS online feedback forms in addressing the aims of the new assessment approach. Random samples of 500 were selected from the first three years of the new assessment implementation, 2010–13, and from 2016 to 17. Using an appropriate coding frame, the feedback was analysed across the samples against key trainee attributes including stage of training and level of adjudged competence. Criteria for identifying high quality feedback were derived from the literature and a simplified form of qualitative comparative analysis was used to identify the conditions associated with high quality feedback. An average of 97% of the assessments contained written feedback but the number of instances of high quality feedback was found to be exceedingly small at around 5%. The paper offers suggestions for making the feedback process more purposeful in achieving the aims of formative assessment.

Are workplace-based assessment methods (DOPS and Mini-CEX) effective in nursing students’ clinical skills? A single-blind randomized, parallel group, controlled trial

Background: Evaluation of clinical skills is critically important for nursing students. However, the quality of evaluation tools is poor.Objectives: To evaluate the effectiveness of Direct Observation of Procedural Skills (DOPS) and Mini-Clinical Evaluation Exercise (Mini-CEX) on clinical skills of nursing students.Methods: This study was conducted among 108 senior nursing students. Mini-CEX and DOPS were utilized to evaluate clinical skills in the intervention group.Results: The mean of students’ scores in all of the five procedures was significantly higher in the intervention group compared to control group.. Students’ scores for the procedures significantly raised through the first stage of DOPS and Mini-CEX to the third stage.Conclusions: Utilization of DOPS and Mini-CEX for evaluation of clinical skills in nursing students effectively enhance their learning ability. Implementing of such assessment methods lead to promoting clinical skills of students which eventually help them to provide high quality care for their patients.

ERCP assessment tool: evidence of validity and competency development during training.

The endoscopic retrograde cholangiopancreatography (ERCP) direct observation of procedural skills (DOPS) is a 27-item competency assessment tool that was developed to support UK ERCP training. We evaluated validity of ERCP DOPS and competency development during training. This prospective study analyzed ERCP DOPS performed in the UK between July 2016 and October 2018. Reliability was measured using Cronbach's alpha, and DOPS scores were benchmarked using the contrasting groups method. The percentage of competent scores was averaged for each item, domain, and overall rating, and stratified by lifetime procedure count to evaluate learning curves. Multivariable analyses were performed to identify predictors of DOPS competence. 818 DOPS (109 trainees, 80 UK centers) were analyzed. Overall Cronbach's alpha was 0.961. Attaining competency in 87 % of assessed DOPS items provided the optimal competency benchmark. This was achieved in the domain sequence of: pre-procedure, post-procedure management, endoscopic non-technical skills, cannulation & imaging, and execution of selected therapy, and across all items after 200 - 249 procedures (89 %). After 300 procedures, the benchmark was reached for selective cannulation (89 %), but not for stenting (plastic 73 %; metal 70 %), sphincterotomy (80 %), and sphincteroplasty (56 %). On multivariable analysis, lifetime procedure count (P = 0.002), easier case difficulty (P < 0.001), trainee grade (P = 0.03), and higher lifetime DOPS count (P = 0.01) were predictors of DOPS competence. This study provides novel validity, reliability, and learning curve data for ERCP DOPS. Trainees should have a minimum of 300 hands-on ERCP procedures before undertaking summative assessment for independent practice.

Undergraduate ultrasound education at German-speaking medical faculties: a survey.

Background: The purpose of this study was twofold: to assess the status of undergraduate medical ultrasound (US) education in the German-speaking area and to suggest a possible framework for a longitudinal undergraduate medical US curriculum based on the study results and a literature review.Methods: The survey included 44 medical faculties in the German-speaking area: 37 in Germany, four in Austria and three in German-speaking Switzerland. A standardized questionnaire focused on the following aspects of undergraduate medical US education: general information, organization, resources, assessment methods and evaluation. Results: Data from 28 medical faculties were analysed. 26 out of 28 medical faculties offered US courses, 21 offered compulsory as well as elective courses, four offered compulsory and one elective courses only. 27 medical faculties supported US skills implementation. Abdominal US (n=25) was most common in teaching basic US skills. A learning objective catalogue was provided at 15 medical faculties. At 22 medical faculties, medical specialists were involved in undergraduate medical US education. 24 out of 26 medical faculties thought that peer-teaching is important to convey US skills. Medical faculties used the following methods to assess US skills: objective structured clinical examination (OSCE, n=7), non-standardized practical exams (n=4), non-standardized combined oral-practical exams (n=2) or direct observation of procedural skills (DOPS, n=1). 25 out of 26 medical faculties evaluated their US courses and 19 made suggestions for improvements in undergraduate medical US education.Conclusion: Medical faculty members in the German-speaking area have recognized the relevance of undergraduate medical US education. So far, courses are offered heterogeneously with rather short hands-on scanning time and high student-instructor ratio. Based on the results of this study and a literature review we suggest a possible framework and milestones on the way to a longitudinal undergraduate medical US curriculum.