Using Clinical Vignettes to Evaluate VTE Protocol Adherence

Abstract

BackgroundVenous thromboembolism (VTE) prophylaxis is underutilized in hospitalized medical patients. Underutilization might occur as a result of resident practice variation incurred by using a complex risk assessment tool.ObjectiveTo examine what impact repetitive exposure to an electronic point-based VTE risk assessment tool has on resident inter-rater reliability and protocol adherence.DesignPre and post intervention cross-sectional cohort study.SettingSingle academic center.PatientsConvenience samples of Internal Medicine residents.InterventionsResidents completed clinical vignettes before and after any exposure to an electronic risk assessment tool and reminder alert. They were asked to make three determinations using a point-based VTE risk assessment tool: risk stratification, identify contraindications, and VTE prevention strategy.MeasurementsInter-rater reliability for risk assessment, contraindications, and VTE prophylaxis strategy and protocol adherence.ResultsKappa scores for VTE risk assessment did not change, but improved for VTE plan increasing from 0.28 to 0.37. Protocol adherence improved from 71% in 2008 to 79% (P = 0.06). There was a significant decrease in under-prophylaxis (22% to 6%, P < 0.0001) but a significant increase in over-prophylaxis (7% to 16%, P = 0.001).ConclusionsUsing clinical vignettes, we determined that daily exposure to an electronic risk assessment tool did not improve the inter-rater reliability of a point-based risk assessment tool when used by medical residents. This might be due to inexperienced providers using a complex point-based tool. Overall, adherence improved, and under-prophylaxis decreased, but over-prophylaxis increased. Clinical vignettes are a generalizable method to monitor resident prophylaxis practices and way to identify educational and process improvement opportunities.KeywordsResident; Inter-rater reliability; Venous thromboembolism; Agreement; Risk assessment score