Direct Observation of Procedural Skills for the Clinical Evaluation of Speech Therapy Students in the Assessment of Speech Organs

Abstract

Background and Objectives: Direct observation of procedural skills (DOPS) is one of the new methods of clinical evaluation that gives feedback about students’ clinical skills. The design and psychometrics of the DOPS test tool for the clinical assessments of speech therapy students in the evaluation of speech organs have been discussed. Methods: In the current non-interventional descriptive study, 20 speech therapy students were selected from the Faculty of Rehabilitation Sciences. The DOPS test related to the speech organs evaluation procedure and its evaluation checklist was prepared. After obtaining the consent and training, students and evaluators entered the study. The evaluators observed each student’s work with a real patient, and their judgments were recorded based on a structured checklist. Each of them was given feedback in the appropriate environment. Face validity, content, and inter-rater reliability were measured and reported. Results: Face validity, including importance, clarity, and simplicity, was investigated for each item. The items’ impact scores for favorable face validity of >1.5 were included in the questionnaire. The content validity index for each item was over 0.8, and the content validity ratio was >0.62. All students have chosen the option of slightly satisfied to completely satisfied, and 70% have chosen the option of high satisfaction and complete satisfaction. None of the evaluators chose the options of no satisfaction to slightly satisfied. For inter-rater reliability, the intraclass correlation coefficient (ICC) was calculated using the opinions of two evaluators, and the ICC value was 0.884 with a 95% confidence interval (0.708-0.954) (P<0.001). Conclusion: The results indicated good inter-rater agreement and reasonable reliability. According to this study, the use of DOPS to evaluate clinical skills in speech therapy students in evaluating speech organs has high validity and reliability.