Initial Reliability of The Standardized Orthopedic Assessment Tool (SOAT)

Abstract

Orthopaedic assessment skills are critical to the success of athletic therapists and trainers. The Standardized Orthopedic Assessment Tool (SOAT) has been content validated. To establish interrater reliability of the SOAT. Thirty-two college students, 10 raters, and 2 standardized patients (SPs) from Calgary, Alberta, Canada. Randomized observational study. Students were allowed 30 minutes to complete a mock orthopaedic assessment of an SP with an injury specific to a region of the body (shoulder, knee, or ankle). Using the region-specific SOAT, raters and SPs evaluated students' orthopaedic assessment skills. The sum totals of the SOAT for 2 raters and 1 SP were used to calculate each student's performance scores for respective scenarios. Scale reliability analysis (Cronbach alpha) was completed on the SOAT for each of the 3 body-region examinations. The mean overall reliability of 3 SOATs (ie, ankle, knee, and shoulder) was positive: alpha = .85 with the SP scores factored into the equation and alpha = .86 without the SP scores factored into the equation. Reliability for the ankle region was highest (alpha = .91), followed by the knee (alpha = .83) and the shoulder (alpha = .82). The study sample size was small, but the results will enable further study with generalization to a broader audience of athletic therapists and athletic trainers. Because a baseline measure of reliability was established using a robust statistical analysis, future researchers can employ more stringent statistical analysis and focus on the effects of various pedagogical techniques to teach and learn the underlying construct of clinical competence in orthopaedic assessment.