Direct Observation Of Procedural Skills Research Articles

Comparison of the effects of apprenticeship training by sandwich feedback and traditional methods on final-semester operating room technology students' perioperative competence and performance: a randomized, controlled trial.

Effective feedback is fundamental in clinical education, as it allows trainers to constantly diagnose the trainees' condition, determine their weaknesses, and intervene at proper times. Recently, different feedback-based approaches have been introduced in clinical training; however, the effectiveness of such interventions still needs to be studied extensively, especially in the perioperative field. Therefore, this study sought to compare the effects of apprenticeship training using sandwich feedback and traditional methods on the perioperative competence and performance of Operating Room (OR) technology students. Thirty final-semester undergraduate OR technology students taking the apprenticeship courses were randomly allocated into experimental (n = 15) and control (n = 15) groups through the stratified randomization approach. The students in the experimental group experienced Feedback-Based Learning (FBL) using a sandwich model, and the students in the control group participated in Traditional-Based Training (TBT) in six five-hour sessions weekly for three consecutive weeks. All students completed the Persian version of the Perceived Perioperative Competence Scale-Revised (PPCS-R) on the first and last days of interventions. Also, a blinded rater completed a checklist to evaluate all students' performance via Direct Observation of Procedural Skills (DOPS) on the last intervention day. Besides, the students in the FBL filled out a questionnaire regarding their attitude toward the implemented program. The mean total score of the PPCS-R was significantly higher in the FBL than in the TBT on the last intervention day (P < 0.001). Additionally, the increase in mean change of PPCS-R total score from the first to last days was significantly more in the FBL (P < 0.001). Likewise, the FBL students had higher DOPS scores than the TBT ones (P < 0.001). Most FBL students also had a good attitude toward the implemented program (n = 8; 53.3%). Apprenticeship training using a sandwich feedback-based approach was superior to the traditional method for enhancing perioperative competence and performance of final-semester OR technology students. Additional studies are required to identify the sustainability of the findings.

Assessment in Undergraduate Competency-Based Medical Education: A Systematic Review.

Studies that have methodically compiled the body of research on the competency-based medical education (CBME) assessment procedure and pinpointed knowledge gaps about the structure of the assessment process are few. Thus, the goals of the study were to create a model assessment framework for competency-based medical education that would be applicable in the Indian setting as well as to thoroughly examine the competency-based medical education assessment framework. PubMed, MEDLINE (Ovid), EMBASE (Ovid), Scopus, Web of Science, and Google Scholar were the databases that were searched. The search parameters were restricted to English language publications about competency-based education and assessment methods, which were published between January 2006 and December 2020. A descriptive overview of the included research (in tabular form) served as the foundation for the data synthesis. Databases provided 732 records; out of which 36 fulfilled the inclusion and exclusion criteria. Thirty-six studies comprised a mix of randomized controlled trials, focus group interviews, and questionnaire studies, including cross-sectional studies, qualitative studies (03), mixed-method studies, etc. The papers were published in 10 different journals. The greatest number was published in BMC Medical Education (18). The average quality score for included studies was 62.53% (range: 35.71-83.33%). Most authors are from the UK (07), followed by the USA (05). The included studies were grouped into seven categories based on their dominant focus: moving away from a behavioristic approach to a constructive approach of assessment (01 studies), formative assessment (FA) and feedback (10 studies), the hurdles in the implementation of feedback (04 studies), utilization of computer or online based formative test with automated feedback (05 studies), video feedback (02 studies), e-learning platforms for formative assessment (04 studies), studies related to workplace-based assessment (WBA)/mini-clinical evaluation exercise (mini-CEX)/direct observation of procedural skills (DOPS) (10 studies). Various constructivist techniques, such as concept maps, portfolios, and rubrics, can be used for assessments. Self-regulated learning, peer feedback, online formative assessment, an online computer-based formative test with automated feedback, the use of a computerized web-based objective structured clinical examination (OSCE) evaluation system, and the use of narrative feedback instead of numerical scores in mini-CEX are all ways to increase student involvement in the design and implementation of the formative assessment.

The Effect of Direct Observation of Procedural Skills on Learning and Satisfaction of Students in Community Dentistry Course.

Direct Observation of Practical Skills (DOPS) tests is a valuable method for clinical assessment. This study aimed to implement the DOPS test to assess the procedural skills of community dentistry courses and its effects on mastery learning and satisfaction of professors and students at Tabriz faculty of dentistry in 2021-2022. In a quasi-experimental study, 60 dentistry students of a class were assigned into two study (n=30) and control (n=30) groups by Permuted block randomization. In the case group, the skills were related to Fluoride therapy, fissure sealant therapy, and health education evaluated by DOPS. In the control group, these skills were evaluated by traditional evaluation methods. Each test was repeated three times. Finally, the satisfaction of students in the case group was assessed by a questionnaire. The chi-square test was used to compare qualitative variables. Repeated measure ANOVA test was used to compare the mean scores in three stages and two groups. P value less than 0.05 was considered significant. Data were analyzed using SPSS 16 software. A significant difference in the mean score of Fluoride therapy, pit and fissure sealant therapy, and health education was seen between the case and control groups (P<0.001). Also a significant increase in these skills in the third stage of assessment in the case group was observed (P<0.001). The professors and students' satisfaction was considerably high on the DOPS test. The DOPS method had more impact on Fluoride therapy, pit and fissure sealant therapy, and health education's learning process in dentistry students than the conventional evaluation. The professors and students' satisfaction level was high regarding DOPS. The advantages of the DOPS method are student-centeredness, objectivity, and appropriate feedback.

Application of ALSO course in standardized training Resident in Obstetric

ObjectiveTo explore the teaching effect of Advanced Life Support in Obstetrics (ALSO) Course in the standardized training resident in obstetric.Methods60 residents of obstetrics from January 2021 to December 2022 were randomly divided into two groups, observation group and control group. The experimental group used ALSO teaching method, and the control group used traditional teaching method. The teaching effect was evaluated by theoretical examination, direct observation of procedural skills (DOPS) scale and mini clinical evaluation (Mini-CEX) scale.ResultsThe theoretical achievements of the observation group were significantly higher than that of the control group (P < 0.05). The pre-procedural preparation, safe analgesia, technique of procedure, aseptic technique, seeks help when necessary, post-procedural management, communication skills, humanistic care and overall performance score of the DOPS in the experimental group were higher than those in the control group (P < 0.05). The organization efficiency, humanistic qualities, manipulative skills, clinical judgment, medical interviewing skills and overall clinical competence score of the Mini-CEX in the experimental group were higher than those in the control group (P < 0.05).ConclusionsALSO teaching method has an ideal effect in the standardization training of residents of obstetrics, indicating the prospect of active in-depth research and expanded application.

An action research to introduce direct observation of procedural skills (DOPS) as clinical skill performance assessment tool during medical internship in the department of surgery, Debre Tabor University, Debre Tabor, Ethiopia, 2023

An action research to introduce direct observation of procedural skills (DOPS) as clinical skill performance assessment tool during medical internship in the department of surgery, Debre Tabor University, Debre Tabor, Ethiopia, 2023

Development of a Cost-Effective 3D-Printed Airway Suction Simulator for Respiratory Therapy Students.

Three-dimensional (3D)-printed models are cost-effective and can be customized by trainers. This study designed a 3D-printed airway suction simulator for use by respiratory therapy (RT) students. The objective was to demonstrate the cost-effectiveness and application of 3D-printed models in respiratory care training, aiming to enhance the educational experience for RT students. This study developed a 3D-printed airway suction simulator that was cost-effective. A randomized controlled trial was conducted involving RT students to compare effectiveness in a 3D-model group and a control group. Skill assessments and written examinations were used to evaluate the participants' knowledge and skills. A total of 38 second-year RT students were randomly assigned to either the 3D-model group (n = 19) or the control group (n = 19). One participant in the 3D-model group was lost to follow-up during the planned direct observation of procedural skills (DOPS) assessment and satisfaction questionnaire completion. The posttest written examination scores were significantly higher in the 3D-model group than in the control group (100% vs 80%, P = .02). The scores from the DOPS and satisfaction questionnaire were comparable in the 2 groups. This study demonstrated that 3D printing can be used to create a safe and cost-effective airway suction simulator for use by RT students, with potential to enhance training methods. Further research is necessary.

Scaffolding Self-Regulated Learning in Operative Dentistry Through Self-Assessment Training.

This study aimed to investigate how effective self-assessment training is in improving the self-regulated learning (SRL) ability of operative dentistry students. The secondary aim was to explore students' attitudes towards self-assessment in clinical settings. Self-assessment has been the focus of dental education research for decades and has been claimed to be cornerstone in the SRL process. This is quasi-experimental study conducted at Damascus University Faculty of Dental Medicine, Syria. A modified direct observation of procedural skills (DOPS) assessment protocol with a self-assessment component was applied to a convenience sample of students across five clinical encounters in 2022. Data were collected during clinical training on the DOPS form and at the end of the experiment using a structured questionnaire with a single open-ended question. The qualitative part of the study followed a deductive theory-informed content analysis approach, relying on Zimmerman SRL theory. A total number of 32 students participated in the study, eight of whom were male. During the experiment assessment, participants showed overwhelming positive attitudes towards the utility of self-assessment (m ≥ 3.5; max:4) and the overall assessment method (3.3 ≥ m ≥ 3.5; max:4). Cronbach's alpha for the post-experiment questionnaire was 0.888. A total of 84.4% of the participants strongly agreed that self-assessment is useful in clinical training. Data collected (qualitative and quantitative) from participants revealed the positive effects of self-assessment on all three phases of SRL, especially the reflection phase. Self-assessment practice and training showed positive results in terms of improving SRL and performance in a clinical context; SRL appeared to be affected by self-assessment in all phases (forethought, performance, and reflection). Participants' perceptions of self-assessment were generally very positive. These findings imply the potential benefits of applying self-assessment training interventions to foster the SRL ability of learners within the clinical context.

Application of direct observation of operational skills in nursing skill evaluation of pressure injury: A randomized clinical trial.

This was a non-blinded, single-centre, randomized, controlled clinical trial that compared the effectiveness of direct observation of procedural skills (DOPSs)with traditional assessment methods in pressure injury (PI) care skills. The study population included 82 nursing professionals randomly assigned to the study group (n = 41) and the control group (n = 41). Both groups of nurses underwent a 6-month training in PI care skills and were subsequently evaluated. The main outcome variables were the PI skill operation scores and theoretical scores. Secondary outcome variables included satisfaction and critical thinking abilities. Independent sample t-tests and chi-square tests were used to assess differences between the two groups of nurses. The results showed no statistically significant difference in PI skill operation scores between the two groups of nurses (p > 0.05). When comparing the PI theoretical scores, the study group scored higher than the control group, and this difference was statistically significant (p < 0.05). In terms of satisfaction assessment, the study group and the control group showed differences in improving self-directed learning, enhancing communication skills with patients, improving learning outcomes and increasing flexibility in clinical application (p < 0.05). When comparing critical thinking abilities between the two groups of nurses, there was no statistically significant difference at the beginning of the training, but after 3 months following the training, there was a statistically significant difference between the two groups (p < 0.01).The results indicated that the DOPS was effective in improving PI theoretical scores, increasing nurse satisfaction with the training and enhancing critical thinking abilities among nurses.

A42A palliative care skills study day is an effective way of meeting palliative care registrar curriculum Direct Observation of Procedural Skills (DOPs) requirements

The new Speciality Training curriculum for Palliative Medicine from August 2022 identifies key procedural skills that, for the first time, can be performed in a simulation (SIM) skills lab to demonstrate proficiency [1]. These include tracheostomy care, managing non-invasive ventilation (NIV) and the management of indwelling ascitic drains. These skills can be difficult to acquire in non-hospital settings like hospices and in the community. A bespoke, pilot palliative care skills day was organized to assess the suitability for theory, skills training and subsequent opportunity to demonstrate proficiency by sign off against curriculum competencies. A total of 11 palliative care trainees from the Northern Deanery attended a pilot SIM study day in March 2023. Three parallel workshops were planned – tracheostomy care, the insertion and management of ascitic drains and managing NIV. Each workshop was delivered by local experts in the area (non- palliative care professionals) with experience of teaching and training other professionals. assessing internal medicine trainees. Trainers were briefed on learning outcomes prior to the session by two palliative care consultants, and the clinical context of each session was set within relevant palliative care environments for e.g. the care of a patient in a hospice, in the community or in a hospital. Trainee confidence was assessed before and after SIM training with the use of 10-point Likert scales and free text comments. Overall self-reported trainee confidence and competence scores increased for all three workshops (Paracentesis 7 to 8.8 out of 10, NIV 4.5 to 8.6, tracheostomy care 3.8 to 8.9). All trainees agreed it was an effective and educational way of addressing curriculum objectives; and agreed it should be a rolling programme offered regionally. Trainees commented on the positive learning environment, the small group sizes, the benefit of being taught by experts and having the opportunity to be assessed for curriculum requirements. Trainees who had previously achieved competencies commented on the benefit of refreshing skills. One area for development identified was the lack of standardization on DOPs forms about the level of proficiency required. This will be fed back to the regional training committee for the future. Trainees identified further clinical skills that could be addressed in a skills lab and hence, a second skills day will be organized. SIM training is an effective tool for delivering training around procedural skills for palliative care registrars. It also brings opportunity to demonstrate proficiency in specific practical skills. Authors confirm that all relevant ethical standards for research conduct and dissemination have been met. The submitting author confirms that relevant ethical approval was granted, if applicable.

UK and Ireland Joint Advisory Group (JAG) consensus statements for training and certification in diagnostic endoscopic ultrasound (EUS)

Background and aimsInternational endoscopy societies vary in their approach for credentialing individuals in endoscopic ultrasound (EUS) to enable independent practice; however, there is no consensus in this or its implementation....

Designing and implementing an OMMID midwifery professional competence mixed test: A multimethod, multiphasic study

The assessment of professional competence is essential to ensure the achievement of competence standards. The goal of this study is to design and implement a professional competence test model to make the test unified, comprehensive, and fair. This research was a multimethod, multiphasic study. The first qualitative phase of the nominal group technique was conducted to design the test model with specific guidelines. The second phase was a mixed-method parallel field trial conducted on 161 senior midwifery bachelor students in universities of Iran. The test was conducted following the traditional method in the control group and OMMID model in the intervention group. This model consists of three parts: the MCQs (multiple choice questions), OSCE (objective structured clinical examination), and clinically-oriented tests, which included Mini-CEX (mini-clinical evaluation exercise) and DOPS (direct observation of procedural skills). Data were collected using questionnaires and interviews, and the results were assessed qualitatively and quantitatively. The OMMID model and the associated guideline were designed. Qualitative data analysis resulted in six main themes, including organizing, structure, tension due to change, fairness, unification, and outcome. The OMMID model did not increase students' anxiety and stress and did not decrease their satisfaction. The merits of this model include having a centralized guideline, using multiple evaluation methods, comprehensive evaluation of necessary skills, promotion of fairness, and increased student satisfaction.

Teaching Bundle Care Approach to Intensive Care Nurses to Prevent Ventilator Associated Pneumonia with a Stratified Validated Module.

Module-based teaching of ventilator-associated pneumonia (VAP) with a systematic, stratified approach is expected to have a promising role in teaching. We hypothesized it to improve the skill and knowledge of intensive care unit nurses. The aim of this study was to determine the efficacy of the teaching module with debriefing sessions on the VAP bundle care approach, and the secondary objective was to estimate the improvement in individual components of the VAP bundle. A total of 200 paramedical staff were exposed to a teaching module consisting of pretest didactic lectures, debriefing sessions, reflection by participants, and feedback. Posttest analysis was done to assess knowledge. Skill assessment was done with directly observed procedural skills (DOPS) assessment, and feedback was taken from participants. Follow-up was done at 6 months to assess decay in knowledge and skills. Preworkshop and postworkshop DOPS scores were analyzed using the Mann-Whitney U test. Subgroup analysis was performed using the paired t test. Median pretest and posttest scores were 6 (interquartile range, 4-8) and 13 (interquartile range, 11-15), respectively (P < .001). Comparing DOPS scores before and after exposure to the teaching module, the number of subjects with scores that were below expectations was 24 and 4; meeting expectations was 94 and 24; borderline expectations were 36 and 104; and above expectations was 46 and 58, respectively, between the groups (P < .001). A decay in knowledge and skills was noted in the follow-up. A validated teaching module with debriefing sessions is useful in training bundle care approaches to intensive care unit nurses.

Quality Metrics and Indicators for Surgical Training: A Scoping Review

Surgical training quality is critical to ensure that trainees receive adequate preparation to perform surgical procedures independently and that patients receive safe, effective, and high-quality care. Numerous surgical training quality indicators have been proposed, investigated and implemented. However, the existing evidence base for these indicators is limited, with most studies originating from English-speaking, high-income countries. This scoping review aimed to identify the range of quality indicators that have been proposed and evaluated in the literature, and to critically evaluate the existing evidence base for these indicators. A systematic literature search was conducted using MEDLINE and Embase databases to identify studies reporting on surgical training quality indicators. A total of 68 articles were included in the review. Operative volume is the most commonly cited indicator and has been investigated for its effects on trainee exam performance and career progression. Other indicators include operative diversity, workplace-based assessments, regular evaluation and feedback, academic achievements, formal teaching, and learning agreements, and direct observation of procedural skills. However, these indicators are largely based on qualitative analyses and expert opinions and have not been validated quantitatively using clear outcome measures for trainees and patients. Future research is necessary to establish evidence-based indicators of high-quality surgical training, including in low-resource settings. Quantitative and qualitative studies are required to validate existing indicators and to identify new indicators that are relevant to diverse surgical training environments. Lastly, any approach to surgical training quality must prioritize the benefit to both trainees and patients, ensuring training success, career progression, and patient safety.

Evaluation of a workplace assessment method designed to improve self-assessment in operative dentistry: a quasi-experiment

BackgroundDental education has placed continued emphasis on self-regulated learning (SRL) and its subprocess, self-assessment. This study set out to evaluate the effectiveness of a novel workplace assessment method in developing trainees’ self-assessment of operative procedures.MethodsA Direct Observation of Procedural Skills (DOPS) form was modified for the use and measurement of self-assessment. Participants were trained on how to conduct self-assessment using the designed assessment form and its grading rubric. Feedback and feedforward sessions were given to address self-assessment and performance issues. A P-value less than 0.10 was considered significant and the confidence level was set at 90%.ResultsThirty-two Year 5 dental students with an age mean of 22.45 (SD = 0.8) completed five self DOPS encounters during the clinical operative dentistry module in 2022. The aggregated total deviation (absolute difference) between self-assessment and teacher assessment decreased consistently in the five assessment encounters with a significant mean difference and a medium effect size (P = 0.064, partial Eta squared = 0.069). Participants’ self-assessment accuracy differed from one skill to another and their ability to identify areas of improvement as perceived by teachers improved significantly (P = 0.011, partial Eta squared = 0.099). Participants’ attitudes towards the assessment method were positive.ConclusionsThe findings suggest that the self DOPS method was effective in developing participants’ ability to self-assess. Future research should explore the effectiveness of this assessment method in a wider range of clinical procedures.

Perception of the Use of the Direct Observation of Procedural Skills in Occupational Therapy Postgraduate Year Training in Taiwan: Survey of the Perspectives of Trainees and Supervisors.

Assessment of clinical competence is a significant part of the training for young occupational therapists (OTs). Objective and systematic assessment allows both supervisors and trainees to be aware of the training objectives and monitor the progress. The direct observation of procedural skills (DOPS) is a work-based assessment to evaluate professional knowledge, skills, and attitude in clinical training. This study investigated the perspectives of OT educators and trainees on using DOPS and their discrepancy for OT postgraduate year (PGY) training. This study used a quantitative online survey. Eighty-six supervisors and 41 trainees of OT PGY training programs from 95 hospitals returned the questionnaire (a 90.5% return rate), and 64 supervisors and 30 trainees who used DOPS were analyzed. Outcomes included the practicality in using the DOPS in clinical settings, the ease of rating the DOPS, and advantages and the disadvantages of the DOPS. Most respondents reported that completing one DOPS required at least 11 minutes for direct observation (11-40 minutes: teacher 92.2%; trainee 80.6%). Most respondents (teacher 96.9%, trainee 96.8%) had feedback after direct observation of DOPS, and about half of the feedback assessments took 5 to 10 minutes (teacher 53.1%, trainee 48.4%). Most OT educators and trainees agreed that clinical resources were sufficient and that DOPS matched with OT training goals, benefited OT competence training, and had a fair, objective, and consistent scoring system. Significantly higher percentages of OT trainees felt stressed in and satisfied with the DOPS assessment than trainers. Differences between teachers and trainees regarding easiness of rating DOPS items were not significant. Most OT educators and trainees agreed that DOPS was a practical and appropriate assessment for OT PGY training.

Principles for teaching sonography - current status.

Since many young medical residents require sonographic skills early on during training, increased attention has been paid to including sonography classes in undergraduate medical education, among both professional societies and medical educators responsible for medical licensing exams. Medical schools worldwide have developed and implemented a variety of ultrasound teaching formats.This article addresses evidence-based solutions to crucial challenges in planning and implementing undergraduate sonography education. In order to achieve a sustainable increase in practical sonographic competence, we suggest small-group classes with sufficient individual hands-on scanning time for each student. We recommend concentrating on a circumscribed topic and teaching it thoroughly and practically rather than superficially outlining a broad subject area. Provided that peer teachers undergo adequate training, student peer teachers are not inferior to physicians as teachers, as far as student satisfaction, theoretical knowledge and practical skills acquisition are concerned. The assessment of acquired practical skills should consist of practical examinations, such as an objective structured clinical examination (OSCE) or a direct observation of procedural skills (DOPS). In contrast to using healthy volunteers as training models, simulation trainers allow the demonstration of pathological findings in authentic sonographic images, with the disadvantages of unrealistically easy image acquisition, as well as the lack of interaction with the patient.

Workplace-based assessment - Moving from participation to engagement using direct observation of procedural skills (DOPS)

Workplace-based assessment - Moving from participation to engagement using direct observation of procedural skills (DOPS)

Effect of Workplace-Based Assessment Utilization as a Formative Assessment for Learning Among Family Medicine Postgraduates at the Faculty of Medicine, Menoufia University: A Prospective Study.

Background Workplace-based assessment (WBA) is a group of assessment approaches that assesses the trainees' performance through their observation and monitoring in real clinical settings and then provides them with constructive and relevant feedback. Many WBA tools are available, including the mini-clinical evaluation exercise (mini-CEX), direct observation of procedural skills (DOPS), case-based discussions, and multisource feedback (peers, seniors, and patients). A WBAcan help medical students improve their clinical competencies and ensure that qualified physicians graduate. Methods This prospective study was done in the family medicine department at the Menoufia Faculty of Medicine in Egypt and passed through two phases. Phase I was introducing an orientation lecture for family medicine staff and a convenient sample of 21 family medicine postgraduates about WBA. Phase II was conducting a monthly mini-CEX and DOPS for the postgraduates. Finally, students' satisfaction with the WBA was assessed, and all collected data were analyzed via Statistical Package for Social Science (SPSS) version 23 (IBM Corp., Armonk, NY). Results A total of 105 feedback sheets were obtained. These feedback sheets were subdivided into 63 mini-CEX feedback sheets (21 sheets from each mini-CEX session for three sessions) and 42 DOPS feedback sheets (21 sheets from each DOPS session for two sessions), all of which were collected and analyzed. A significant improvement was detected in the mini-CEX and DOPS feedback scores of the postgraduates throughout the consecutive sessions (9.5 ± 2.7, 24.9 ± 2.5, 27.29 ± 1.5) (P < 0.001) for Mini-CEX and (6.1 ± 1.8 versus 9.0 ± 1.2) (P < 0.001) for DOPS. About 93% of the postgraduates recommended the application of WBA for their peers, and 86% of them requested to perform it again for other different clinical cases and procedures. Conclusion Workplace-based assessment in the form of Mini-CEX and DOPS revealed its ability to improve clinical knowledge and skills among family medicine postgraduates who became motivated to undergo it again in search of improving their clinical performance and reducing their stresses related to final summative and objective structured clinical examinations (OSCEs).

Direct Observation of Procedural Skills for the Clinical Evaluation of Speech Therapy Students in the Assessment of Speech Organs

Background and Objectives: Direct observation of procedural skills (DOPS) is one of the new methods of clinical evaluation that gives feedback about students’ clinical skills. The design and psychometrics of the DOPS test tool for the clinical assessments of speech therapy students in the evaluation of speech organs have been discussed. Methods: In the current non-interventional descriptive study, 20 speech therapy students were selected from the Faculty of Rehabilitation Sciences. The DOPS test related to the speech organs evaluation procedure and its evaluation checklist was prepared. After obtaining the consent and training, students and evaluators entered the study. The evaluators observed each student’s work with a real patient, and their judgments were recorded based on a structured checklist. Each of them was given feedback in the appropriate environment. Face validity, content, and inter-rater reliability were measured and reported. Results: Face validity, including importance, clarity, and simplicity, was investigated for each item. The items’ impact scores for favorable face validity of &gt;1.5 were included in the questionnaire. The content validity index for each item was over 0.8, and the content validity ratio was &gt;0.62. All students have chosen the option of slightly satisfied to completely satisfied, and 70% have chosen the option of high satisfaction and complete satisfaction. None of the evaluators chose the options of no satisfaction to slightly satisfied. For inter-rater reliability, the intraclass correlation coefficient (ICC) was calculated using the opinions of two evaluators, and the ICC value was 0.884 with a 95% confidence interval (0.708-0.954) (P&lt;0.001). Conclusion: The results indicated good inter-rater agreement and reasonable reliability. According to this study, the use of DOPS to evaluate clinical skills in speech therapy students in evaluating speech organs has high validity and reliability.

Development and Integration of DOPS as Formative Tests in Head and Neck Ultrasound Education: Proof of Concept Study for Exploration of Perceptions.

In Germany, progress assessments in head and neck ultrasonography training have been carried out mainly theoretically and lack standardisation. Thus, quality assurance and comparisons between certified courses from various course providers are difficult. This study aimed to develop and integrate a direct observation of procedural skills (DOPS) in head and neck ultrasound education and explore the perceptions of both participants and examiners. Five DOPS tests oriented towards assessing basic skills were developed for certified head and neck ultrasound courses on national standards. DOPS tests were completed by 76 participants from basic and advanced ultrasound courses (n = 168 documented DOPS tests) and evaluated using a 7-point Likert scale. Ten examiners performed and evaluated the DOPS after detailed training. The variables of "general aspects" (6.0 Scale Points (SP) vs. 5.9 SP; p = 0.71), "test atmosphere" (6.3 SP vs. 6.4 SP; p = 0.92), and "test task setting" (6.2 SP vs. 5.9 SP; p = 0.12) were positively evaluated by all participants and examiners. There were no significant differences between a basic and advanced course in relation to the overall results of DOPS tests (p = 0.81). Regardless of the courses, there were significant differences in the total number of points achieved between individual DOPS tests. DOPS tests are accepted by participants and examiners as an assessment tool in head and neck ultrasound education. In view of the trend toward "competence-based" teaching, this type of test format should be applied and validated in the future.