Audit and feedback using the brief Decision Support Analysis Tool (DSAT-10) to evaluate nurse–standardized patient encounters

Abstract

Objective To evaluate the brief Decision Support Analysis Tool (DSAT-10) for auditing the quality of nurse–standardized patient encounters, structuring feedback for nurses, and testing instrument reliability. Methods A systematic process was used to develop standardized patient scenarios, pilot-test scenarios, calibrate DSAT-10 coders, analyze taped telephone encounters using DSAT-10, and provide feedback. Inter-rater reliability was calculated using coder agreement, kappa, and intra-class correlation coefficients. Results Six scenarios portrayed patients’ decisional uncertainty from either: pressure from others ( n = 2), unclear values ( n = 2), or inadequate information ( n = 2). Scenarios were easy to use over the telephone, produced realistic role performance, and were practical for audio-recording interactions. DSAT-10 analysis of 76 nurse–standardized patient encounters revealed nurses’ strengths (e.g., information provision) and their limitations (e.g., lack of discussion of values and/or support needs). Scores discriminated between trained and untrained nurses. The kappa coefficient over all items was 0.55 (95% CI: 0.49, 0.61) with higher agreement for encounters involving trained nurses (0.62; 95% CI: 0.43, 0.80). Conclusion Auditing nurse–standardized patient encounters using DSAT-10 and providing feedback to nurses was feasible. Although DSAT-10 items had adequate inter-rater reliability and discriminated between trained/untrained nurses, some items were problematic. Practice implications Providing feedback on nurse encounters with standardized patients experiencing uncertainty has the potential to enhance nurses’ decision support skills.