Development and Validation of the ACP-CAT for Assessing the Quality of Advance Care Planning Communication (TH322C)

Abstract

•Describe the components of a novel tool for assessing advance care planning communication quality.•Describe the reliability, validity and acceptability of the ACP-CAT. High quality advance care planning (ACP) discussions are important to ensure patient receipt of goal-concordant care, however there is no existing tool for assessing communication quality in ACP encounters. To develop and validate a novel instrument that can be used to assess ACP communication skills of clinicians and trainees. We developed a 20-item ACP Communication Assessment Tool (ACP-CAT) plus two summative items. Randomized faculty rater pairs assessed internal medicine residents' performances in video-recorded standardized patient encounters before and after an ACP training program using the ACP-CAT. We tested the tool for its 1) discriminating ability, 2) interrater reliability 3) concurrent validity, 4) feasibility, and 5) raters' satisfaction. Fifty-eight pre/post-training video recordings from 29 first-year internal medicine residents at Mount Sinai Hospital were evaluated. ACP-CAT reliably discriminated performance pre- and post-training (median score 6 vs. 11, P < 0.001). For both pre/post-training encounters, interrater reliability was high for ACP-CAT total scores (ICC = 0.83 and 0.82) and the summative items overall impression of ACP skills (ICC = 0.73 and 0.80) and overall ability to respond to emotion (ICC = 0.83 and 0.82). Concurrent validity was shown by the strong correlation between ACP-CAT total score and both summative items. Raters spent an average of 4.8 minutes to complete the ACP-CAT, found it feasible, and were satisfied with its use. ACP-CAT provides a validated measure of ACP communication quality for assessing video-recorded encounters and can be further studied for its applicability with clinicians in different clinical contexts.