Observation Of Procedural Skills Research Articles

Work Base Assessment in Anesthesia for Postgraduate Residents

Workplace-based assessment (WBA) is a collection of evaluation methods that evaluate trainee performance in clinical settings. The aspect of scrutiny of the learner's act in a real-world setting, coupled with an appropriate response, is a hallmark of WPBA, promoting reflective practice. WBA consists of peer, coworker, and patient feedback on clinical performance (mini-clinical assessment implementation, direct opinion of technical assistance), the conversation of clinical circumstances (case-based assessment), & observation of procedural skills (multi-source feedback). Medical schooling commenced transitioning from a method to an outcomes-primarily targeting what graduates can do while being concerned for sick patients and their families. This fashion is especially sizeable in postgraduate education programs, in which the number one approach of mastering is through hands-on with patients and their families. After then, postgraduate education effects are categorized as competencies. Competence is simply an observable ability of healthcare specialists, which integrates a couple of additives which includes knowledge, capabilities, values, and attitudes. Competence gives a framework for publications and tests to make certain that healthcare specialists can carry out key requirements of their discipline. In the United States, expert postgraduate education known as residency generally takes place after 4 years of undergraduate study, accompanied via way of means 4 years of clinical study. It is turning into an increasing number of facts that conventional evaluation techniques that target knowledge, know-how, and visualization degrees aren't enough to evaluate those new key abilities and meet affected person needs. Additionally, many years of failure to examine and completely examine the outcomes of fundamental scientific capabilities, which includes clinical interviews, clinical examinations, knowledgeable decision-making, and scientific reasoning, has had a vast poor effect on the great of care.

The flipped classroom is effective for medical students to improve deep tendon reflex examination skills: A mixed-method study.

Deep tendon reflexes (DTR) are a prerequisite skill in clinical clerkships. However, many medical students are not confident in their technique and need to be effectively trained. We evaluated the effectiveness of a flipped classroom for teaching DTR skills. We recruited 83 fifth-year medical students who participated in a clinical clerkship at the Department of General Medicine, Chiba University Hospital, from November 2018 to July 2019. They were allocated to the flipped classroom technique (intervention group, n = 39) or the traditional technique instruction group (control group, n = 44). Before procedural teaching, while the intervention group learned about DTR by e-learning, the control group did so face-to-face. A 5-point Likert scale was used to evaluate self-confidence in DTR examination before and after the procedural teaching (1 = no confidence, 5 = confidence). We evaluated the mastery of techniques after procedural teaching using the Direct Observation of Procedural Skills (DOPS). Unpaired t-test was used to analyze the difference between the two groups on the 5-point Likert scale and DOPS. We assessed self-confidence in DTR examination before and after procedural teaching using a free description questionnaire in the two groups. Additionally, in the intervention group, focus group interviews (FGI) (7 groups, n = 39) were conducted to assess the effectiveness of the flipped classroom after procedural teaching. Pre-test self-confidence in the DTR examination was significantly higher in the intervention group than in the control group (2.8 vs. 2.3, P = 0.005). Post-test self-confidence in the DTR examination was not significantly different between the two groups (3.9 vs. 4.1, P = 0.31), and so was mastery (4.3 vs. 4.1, P = 0.68). The questionnaires before the procedural teaching revealed themes common to the two groups, including "lack of knowledge" and "lack of self-confidence." Themes about prior learning, including "acquisition of knowledge" and "promoting understanding," were specific in the intervention group. The FGI revealed themes including "application of knowledge," "improvement in DTR technique," and "increased self-confidence." Based on these results, teaching DTR skills to medical students in flipped classrooms improves readiness for learning and increases self-confidence in performing the procedure at a point before procedural teaching.

Implementing work place based assessment: The modified direct observation of procedural skills (DOPS) across medical specialties - An experience from a developing country.

To assess the skill level of residents regarding central venous catheterisation insertion, and to assess the reliability of scores in a simulated situation. The quasi-experimental study with pre- and post-test design was conducted from February to June 2013 at the Aga Khan University, Karachi, and comprised four workshops attended by residents. The workshops were video-recorded for feedback and self-assessment. At the end of the workshops, knowledge and procedural skills were assessed using a self-generated 38-item, task-specific instrument after ensuring its content validity. Data was analysed using SPSS 19. There were 40 residents in the sample. The self-generated instrument was reliable with Cronbach's alpha value 0.83 and inter-rater coefficient 0.79. There was significant improvement in the skills level post-intervention compared to the baselines mean values (p=0.001). The subjects were satisfied with the workshops, as indicated by a mean score of 8.83±1.367. The workshops appeared to improve the central venous catheterisation insertion skills of the residents.

A97 TOOLS FOR DIRECT OBSERVATION AND ASSESSMENT OF COLONOSCOPY: A SYSTEMATIC REVIEW OF VALIDITY EVIDENCE

Abstract Background An increasing focus on quality and safety in colonoscopy has led to broader implementation of competency-based educational systems that enable documentation of trainees’ achievement of the knowledge, skills, and attitudes needed for independent practice. The meaningful assessment of competence in colonoscopy is critical to this process. While there are many published tools that assess competence in performing colonoscopy, there is a wide range of underlying validity evidence. Tools with strong evidence of validity are required to support feedback provision, optimize learner capabilities, and document competence. Aims We aimed to evaluate the strength of validity evidence that supports available colonoscopy direct observation assessment tools using the unified framework of validity. Methods We systematically searched five databases for studies investigating colonoscopy direct observation assessment tools from inception until April 8, 2020. We extracted data outlining validity evidence from the five sources (content, response process, internal structure, relations to other variables, and consequences) and graded the degree of evidence, with a maximum score of 15. We assessed educational utility using an Accreditation Council for Graduate Medical Education framework and methodological quality using the Medical Education Research Quality Instrument (MERSQI). Results From 10,841 records, we identified 27 studies representing 13 assessment tools (10 adult, 2 pediatric, 1 both). All tools assessed technical skills, while 10 assessed cognitive and integrative skills. Validity evidence scores ranged from 1–15. The Assessment of Competency in Endoscopy (ACE) tool, the Direct Observation of Procedural Skills (DOPS) tool, and the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) had the strongest validity evidence, with scores of 13, 15, and 14, respectively. Most tools were easy to use and interpret and required minimal resources. MERSQI scores ranged from 9.5–11.5 (maximum score 14.5). Conclusions The ACE, DOPS, and GiECAT have strong validity evidence compared to other assessments. Future studies should identify barriers to widespread implementation and report on use of these tools in credentialing purposes. Funding Agencies None

An 8-year retrospective survey of assessment in postgraduate dental training in complicated tooth extraction competency

Background/purposeDirect observation of procedural skills (DOPS) has been increasingly used in health education in recent years. This study evaluated the effect of education and trainees’ perception of assessment on the clinical skills of postgraduate dental trainees in complicated tooth extraction. Materials and methodsThis study was conducted as a retrospective survey among postgraduate dental trainees learning complicated tooth extraction in Taipei and Linkou Chang Gung Memorial Hospital from 2012 to 2019. Practical skills were assessed using DOPS by trainees and faculty members. Each clinical case included a complicated extraction of a permanent tooth. ResultsA total of 69 participants (26 men and 43 women, average age = 26.49 years, range = 24–34 years) were included in this study. Within the survey cohort, faculty assessments scored significantly higher than did trainees’ self-assessments in each complicated tooth extraction procedure, with no difference between both sexes. The higher-performing trainees tended to underrate their performance much more than did the lower-performing trainees. More than 40% of the trainees evaluated themselves as having “poor capability” in some invasive surgical procedures, even though their actual performance was not lower than that of those who evaluated themselves as having adequate or good capability. ConclusionSelf-assessment skills should be developed with more practice and experience. We hope that these findings can guide the planning of faculty development programs for clinical instructors, particularly the new cohort of faculty who will succeed the rapidly retiring boomer generation in the next 10 years.

Effect of Feedback on Direct Observation of Procedural Skills in Estimation of RBC Count among First Year Medical Students

Background: Formative evaluation has been effective in promoting active learning and a better futureperformance in students after immediate personalized feedback.Aim and Objectives: To analyze the effect of “Direct observation of procedural skills” (DOPS) assessmentand immediate feedback for first year MBBS students in estimation of RBC Count.Methodology: The study was done on first year MBBS students (n=120) during the Physiology practicalclass. Checklist for estimation of RBC count (15 steps) was prepared by six faculty members. The formativepractical test was divided into two practical sessions. During the first session, each faculty observed 4students performing RBC estimation one by one (DOPS) and the details were recorded in the checklist.Similarly in five practical classes 120 participants were evaluated and immediate feedback was given tothem for two minutes. In the second session the same procedure was followed and results were compared.Students perception about the assessment process (DOPS) was documented by feedback questionnaire asper Likert scale.Results: The mean practical marks of the participants before feedback was 8.32 ± 2.25 and after feedbackwas 10.18 ± 1.81 (P<0.05). 92% of the study participants felt DOPS in RBC estimation was useful to themConclusion: DOPS assessments worked well in the student’s practical laboratory setting by serving as aneffective educational tool. Immediate personalized feedback to the students has resulted in reasonable skilldevelopment both subjectively and objectively.

Internal Assessment in New MBBS Curriculum: Methods and Logistics.

Assessment is a pivotal element of competency-based curriculum. The implementation of competency-based undergraduate medical curriculum in India requires proper implementation of updates in assessment for which universities, colleges, and teachers need to plan and design internal assessment (IA) modules as well as guidelines. IA provides opportunity to assess many competencies and hence should be implemented judiciously. Multiple assessment methods should be used to improve utility of IA. The process should involve all the teachers of a subject and all competencies. Capacity building trainings should be organized by institutes in basic concepts of assessment as well as training in methods such as the objective structured clinical/practical examination, direct observation of procedural skills, and mini-clinical evaluation exercises. The culture of providing regular feedback needs to be instilled at institute levels. The learners who are not able to achieve competencies and required criteria in university examination should be provided predecided remedial measures for improving their performance. The article discusses all these aspects in detail.

Assessment of suitability of direct observation of procedural skills among postgraduate students and faculty in periodontology and implantology

Purpose: Direct observation of procedural skills (DOPS) is widely used on the basis of workplace-based assessments, which consist of direct observation and feedback. The DOPS covers all related professional qualities including knowledge, clinical perception, communication skills, ethics, the rights of patients, and the speed and accuracy of the task. However, the use of DOPS in dental treatment, especially in the field of periodontics is limited. Objective: The aim of the study was to find the utility of DOPS as assessment tool for the periodontal procedure. Methodology: The departmental faculty and postgraduate (PG) students were sensitized to the DOPS concept. A total of 12 PG students performed the periodontal procedure and were assessed on the basis of the DOPS rating scale for the 4 encounters each. The student's clinical skills were assessed by faculty members and then both students and faculty members were asked to give feedback for the procedure and overall experience. Results: Faculty and students both were comfortable in giving the feedback. Improvements in the knowledge of different clinical skills like relevant anatomy, technique of procedure were observed from 1st encounter to 4th encounter from 70% to 73.33%. Taking informed consent was improved from 53.33% to 90% from 1st encounter to 4th. Moreover, it helped the student to develop and improve their clinical skills, patient examination, and decision-making and treatment. Conclusion: The present study finding shows that faculty and students both were comfortable in giving the feedback and it helped the student to develop and improve their clinical skills. Our experience indicates that DOPS is an appropriate and functional tool in PG setup to enhance the clinical competencies.

Validating written feedback in clinical formative assessment

Formative assessment is widely accepted as being crucial to promoting student learning and, since 2010, the UK General Medical Council has mandated its use in workplace-based clinical training for all new doctors. As a result, the Royal College of Radiologists (RCR) instituted a range of formative workplace-based assessments including the Radiology Direct Observation of Procedural Skills (Rad-DOPS), in which supervisors appraise trainees’ performance in carrying out clinical procedures. This paper reports on the quality of the written feedback in 2,500 Rad-DOPS online feedback forms in addressing the aims of the new assessment approach. Random samples of 500 were selected from the first three years of the new assessment implementation, 2010–13, and from 2016 to 17. Using an appropriate coding frame, the feedback was analysed across the samples against key trainee attributes including stage of training and level of adjudged competence. Criteria for identifying high quality feedback were derived from the literature and a simplified form of qualitative comparative analysis was used to identify the conditions associated with high quality feedback. An average of 97% of the assessments contained written feedback but the number of instances of high quality feedback was found to be exceedingly small at around 5%. The paper offers suggestions for making the feedback process more purposeful in achieving the aims of formative assessment.

Workplace based assessment by Direct Observation of procedural skills for postgraduate students in Obstetrics and Gynaecology

Introduction: Structured workplace based assessment in PG curriculum is the need of the hour. This project addresses the same. This study was conducted to assess procedural skill of postgraduate students by using a novel tool-Direct Observation of Procedural skills (DOPS) and impact on learning by reassessment. Methodology: Seven second year postgraduate students and eight faculty members participated in the study. First DOPS cycle was conducted over 6 weeks for 5 basic procedures in Obstetrics and Gynecology (OBGY). The checklists of procedures were shared. Second DOPS cycle was conducted by same faculty. At the end, feedback about the entire process was collected and analyzed. Results: All the students showed improvement in DOPS scores during second session. The component task wise score analysis showed statistically significant improvement in following areas such as maintenance of asepsis, pre procedure preparations, demonstration of anatomy and indications, communication with patient, post procedure management and overall tissue handling. This was evident when ‘paired t test’ was applied to the data. Analysis of feedback showed acceptability, feasibility and relevance of DOPS in OBGY. Conclusion: We found that DOPS is highly acceptable assessment tool by both the postgraduate students and faculty. It helped students to identify gaps in learning and achieve improvement in skills.

An exploration of the training and evaluation systems of medical undergraduates in the context of tutorial system

Objective To investigate the training process of clinical practice ability in medical undergraduates in the context of tutorial system, as well as new clinical training models and evaluation methods. Methods A total of 40 medical undergraduates who entered the stage of clinical practice were randomly divided into tutorial group and control group, with 20 undergraduates in each group. A diversified evaluation system was used to perform a comprehensive evaluation of clinical practice ability of medical undergraduates at different stages, with a combination of professional theory examination, Mini-Clinical Evaluation Exercise, Direct Observation of Procedural Skills, and clinical case report. All the data were analyzed by SPSS 23.0, t test was used for statistical analysis. Results Compared with the control group after the internship ended, the tutorial group had significantly higher scores of clinical professional theory(86.24±2.93), clinical practice ability (7.53±0.48), clinical operation skills (4.35±0.47), and clinical thinking ability (6.31±0.52) (all P<0.05). Conclusion The application of the tutorial training mode during the internship of medical undergraduates can improve their comprehensive qualities such as professional quality, medical theory, and practical skills. The diversified evaluation system can evaluate the teaching effect and reflect the cultivation of comprehensive quality among students. Key words: Tutorial system; Diversified evaluation system; Clinical practice ability

Introduction and Comparison of Direct Observation of Procedural Skills (DOPS) with Conventional Method of Skill Assessment in Dental Students

Introduction and Comparison of Direct Observation of Procedural Skills (DOPS) with Conventional Method of Skill Assessment in Dental Students

Adaptation of Direct Observation of Procedural Skills (DOPS) for assessments in podiatry

Background: The Direct Observation of Procedural Skills (DOPS) is a workplace-based assessment tool widely used in medicine to assess a learner’s ability to execute a technical skill. The aim of this paper is to report on the development phase of the adaptation of the DOPS for the assessment of podiatry learners’ procedural skills. Podiatry learners are required to practise and demonstrate a variety of procedural skills in the management of foot complaints. Such skills include the use of scalpel blades, needles and local anaesthetic applied to a variety of disorders. The DOPS provides an avenue by which a learner’s procedural skills can be assessed and timely feedback provided in the workplace or in simulated environments. Methods: The DOPS was initially adapted for podiatry by a faculty team consisting of a podiatry educator, a clinical education specialist and a clinical educator from another allied health discipline. The first iteration was circulated among podiatry faculty at three other Australian universities. The second iteration was reviewed by clinical supervisors from Southern Cross University (SCU). The third iteration was administered by two clinical supervisors at SCU working with 12 learners during real-time clinical events. Eleven learners used DOPS to assess their peers during five real-time and six simulated learning events. Results: A new tool, the Direct Observation of Procedural Skills in Podiatry (DOPS-P) has emerged from this process. Face and construct validity have been confirmed, and faculty and students consider DOPS-P contributes to learning. Conclusions: Further research is necessary to confirm the validity and reliability of the DOPS-P to support assessment decisions about students’ achievement of podiatry competencies.

Application and evaluation of direct observation of procedural skills in anesthesia-related teaching

Objective To explore the application of direct observation of procedural skills (DOPS) in anesthesia-related effects skills teaching. Methods Anesthesia students in the Department of Anesthesiology, Clinical Medical College, Yangzhou University between June 2013 and June 2015, received an anesthesia-related DOPS examination at the mid-internship and pre-delivery. Through the DOPS assessment results, the analysis of teachers' and students' satisfaction degree on the evaluation results, the teaching effect of DOPS was evaluated. At the same time, the students were surveyed to find out the satisfaction degree of DOPS. SPSS 18.0 was applied to processing the date. Measurement data were expressed as mean plus or minus standard deviation, and t test was used in the comparison between groups. Results Interim assessment indicators for each individual average of more than four points, the individual departmental rotation examination scores were higher than the overall average score interim assessment results. Compared with interim assessment, satisfaction of teachers has made great progress in the end of assessment [(8.46±0.32) vs. (8.14± 0.60)], satisfaction of students was [(8.57±0.59) vs. (8.20±0.45)], satisfaction of the students was higher than the teachers [(8.57 ± 0.59) vs. (8.46 ± 0.32)], and the differences were statistically significant. Conclusion DOPS as an effective anesthesia-related skill of teaching and assessment method, can improve the level of anesthesia anesthesia-related skills of college students. Key words: Direct observation of procedural skills; Anesthesiology; Skills; Assessment method

The impact of assignment strategy on the performance of postgraduate trainees: An analysis of the assessment scores database in a military medicine system

Background: Medical students who graduate after 2003 need to participate in the Postgraduate Primary Care Medical Training Program, provided by the Taiwan Joint Commission on Hospital Accreditation. We wish to know how the education and assignment strategy influences National Defense Medical Center (NDMC) graduates. We examined whether the performance of postgraduate year (PGY) trainees would be affected by the assignment strategies. Materials and Methods: We consecutively collected 173 NDMC graduated trainees who participated in 6-months of PGY training and another 94 trainees who participated for 1-year. During the training period, all the trainees were evaluated by several assessment tools. Trainees were dichotomized according to the levels of dispatched hospitals and preassigned specialty, respectively, to evaluate those effects on the performance of the 6-month PGY training period. Results: We describe the assessment scores of NDMC graduate M.D. trainees engaged in the Tri-Service General Hospital (TSGH) PGY training program. PGY trainees who were preassigned to TSGH due to better overall averages in medical school had better scores in case-based discussion, mini-clinical evaluation exercise, and direct observation of procedural skills than those who were preassigned to the Regional Teaching Hospitals. There was no statistically significant difference in the overall scores. The preassigned specialties themselves did not affect the performance in PGY training. Incorporation of PGY scores in the assignment strategy significantly elevated the performance of PGY trainees who were dispatched to TSGH. Conclusion: The accumulated PGY training database provides educators a unique opportunity in reviewing the effects of education policies and strategies. Assignment strategy could affect the performance of PGY trainees during their training period.

An evaluation of the use of direct observation of procedural skills in the intercollegiate surgical curriculum programme

An evaluation of the use of direct observation of procedural skills in the intercollegiate surgical curriculum programme

An evaluation of the use of direct observation of procedural skills in the intercollegiate surgical curriculum programme

The Intercollegiate Surgical Curriculum Programme (ISCP) ( www.iscp.ac.uk ) is a UK-wide collaborative project involving all the surgical royal colleges and the specialty associations involved in surgical training. Workplace-based assessment (WBA) refers to the assessment of day-to-day working practice 1 and is an integral part of the ISCP assessment framework. The main aim of WBA is to aid learning through constructive feedback. Several WBAs have been adopted by the ISCP including the mini-clinical evaluation exercise (mini-CEX), direct observation of procedural skills (DOPS), case-based discussion (CBD) and procedure-based assessment (PBA). A systematic review of WBAs concluded that there was no clear evidence to show that DOPS can lead to improvements in performance but identified studies that showed largely positive results in terms of learner satisfaction. 2

Feedback and performance scores for direct observation of procedural skills

Direct observation of procedural skills (DOPS) is one of the four main workplace-based assessments (WBAs) in postgraduate medical training. Surgical trainees of all grades and specialties use the Intercollegiate Surgical Curriculum Programme (ISCP) to make an online record of each DOPS. The DOPS are used alongside other WBAs as formative assessments to aid the evaluation of trainee progression at the annual review of competence progression (ARCP).

Application of direct observation of procedural skills （DOPS） in teaching acupuncture manipulation skills

Objective To explore the effect of direct observation of procedural skills （DOPS） in teaching acupuncture manipulation skills. Methods Sophomores of acupuncture and Tuina spe- ciality were divided into two groups based on their classes, 36 students in experimental group and 31 in control group. In experimental group, DOPS on filiform needling was done twice in the middle and at the end of the semester respectively. In control group, DOPS was done once at the end of semester. The final exam＇s DOPS score and satisfaction degree of teachers and students concerning the results of the assessment in two groups were compared by independent samples t-test. P〈0.05 indicates signifi- cant differences. The teaching effect of DOPS was also evaluated by questionnaire survey of students in experimental group. Results The total score of DOPS （4.26 ±0.24）, scores of needle insertion （4.18± 0.42）, and scores of reinforcing and reducing method （4.03 ± 0.28） in experimental group were higher than those in control group （（4.14 ± 0.25）, （4.00 ± 0.29） and （3.87 ± 0.37）, P〈0.05）. The teachers＇ satisfaction （8.38±0.51） and students＇ satisfaction （8.47±1.03） concerning the results of the assessment in experimental group were higher than those in control group （（8.05 ± 0.78） and （7.94 ± 1.15）, P〈0.05）. More than 80% of students in experimental group supported the saying of ＇DOPS promoting students＇ skills training and enhancing the ability of handwork＇ Conclusions DOPS can be used as an effective teaching and examination method and can improve students＇ skills of acupuncture manipulation. Key words: Direct observation of procedural skills; Acupuncture manipulation skills; Examination methods

Short-term training of direct observation procedural skills (DOPS) for community general practitioner trainers

Objective To evaluate the efficacy of short-term training of direct observation procedural skills (DOPS) for community general practitioner (GP) trainers. Methods Thirty six GP trainers participated in one-day workshop of DOPS. The participants were interviewed with questionnaire before and after the training program and the training video was scored. Results 47.2%(17/36) of GP trainers knew and briefly understood DOPS before the training. 41.7%(15/36) of them had a rough idea of how to use it and only 2.8%(1/36) had used it before. All of these trainers wanted to learn how to perform DOPS. The vast majority of GP trainers completely or basically understood the standards for evaluation of 11 items of DOPS after the training. The accuracy rates of 4 items were close to or more than 50.0% in 9-point scale and 7 items exceeded 50.0% in 3-rank system. There was significant difference in accuracy rates of 6 items between the two scoring system (all P <0.01). Conclusions Community GP trainers recognize the advantages of DOPS in assessment of clinical skills and have strong intention of application. The efficacy of short-term workshop is satisfactory, but the evaluators need more training and practice. Key words: General practitioner; Teacher training; Direct observation of procedural skills