Impact of Unannounced Standardized Patient Audit and Feedback on Care, Documentation, and Costs: an Experiment and Claims Analysis

Alan Schwartz,Steven Peskin,Alan Spiro,Saul J Weiner

doi:10.1007/s11606-020-05965-1

Abstract

BackgroundMeaningful variations in physician performance are not always discernible from the medical record.ObjectiveWe used unannounced standardized patients to measure and provide feedback on care quality and fidelity of documentation, and examined downstream effects on reimbursement claims.DesignStatic group pre-post comparison study conducted between 2017 and 2019.SettingFourteen New Jersey primary care practice groups (22 practices) enrolled in Horizon BCBS’s value-based program received the intervention. For claims analyses, we identified 14 additional comparison practice groups matched on county, practice size, and claims activity.ParticipantsFifty-nine of 64 providers volunteered to participate.InterventionUnannounced standardized patients (USPs) made 217 visits portraying patients with 1–2 focal conditions (diabetes, depression, back pain, smoking, or preventive cancer screening). After two baseline visits to a provider, we delivered feedback and conducted two follow-up visits.MeasurementsUSP-completed checklists of guideline-based provider care behaviors, visit audio recordings, and provider notes were used to measure behaviors performed and documentation errors pre- and post-feedback. We also compared changes in 3-month office-based claims by actual patients between the intervention and comparison practice groups before and after feedback.ResultsExpected clinical behaviors increased from 46% to 56% (OR = 1.53, 95% CI 1.29–1.83, p < 0.0001), with significant improvements in smoking cessation, back pain, and depression screening. Providers were less likely to document unperformed tasks after (16%) than before feedback (18%; OR = 0.74, 95% CI 0.62 to 0.90, p = 0.002). Actual claim costs increased significantly less in the study than comparison group for diabetes and depression but significantly more for smoking cessation, cancer screening, and low back pain.LimitationsSelf-selection of participating practices and lack of access to prescription claims.ConclusionDirect observation of care identifies hidden deficits in practice and documentation, and with feedback can improve both, with concomitant effects on costs.

Highlights

Health care delivery is rarely systematically directly observed.[1, 2] Clinicians vary in effective practice based on how well they listen and ask key questions of patients; these variations are not captured using current quality measures.[3, 4]In past research, we used “unannounced standardized patients” (USPs) to measure clinician performance during direct observation
Using USPs, we have studied the impact of physician inattention to patient psychosocial issues relevant to care planning, termed “patient contextual factors.”[5,6,7] In a study of 400 USP visits, internists who overlooked clues that patients’ clinical problems were related to contextual factors were more likely to order unnecessary tests and therapies, with a median excess cost of $231 per visit.[8]
Four USP case scripts were designed around diabetes and idiopathic low back pain, with opportunities for providers to address medication adherence, opioid use, depression screening, smoking cessation, and reluctance to engage in recommended cancer screenings

Summary

Introduction

Health care delivery is rarely systematically directly observed.[1, 2] Clinicians vary in effective practice based on how well they listen and ask key questions of patients; these variations are not captured using current quality measures.[3, 4]In past research, we used “unannounced standardized patients” (USPs) to measure clinician performance during direct observation. USPs are actors trained to present to clinicians incognito as patients, portraying standardized scripts that facilitate controlled comparisons among practices and providers. OBJECTIVE: We used unannounced standardized patients to measure and provide feedback on care quality and fidelity of documentation, and examined downstream effects on reimbursement claims. INTERVENTION: Unannounced standardized patients (USPs) made 217 visits portraying patients with 1–2 focal conditions (diabetes, depression, back pain, smoking, or preventive cancer screening). MEASUREMENTS: USP-completed checklists of guideline-based provider care behaviors, visit audio recordings, and provider notes were used to measure behaviors performed and documentation errors pre- and post-feedback. We compared changes in 3-month office-based claims by actual patients between the intervention and comparison practice groups before and after feedback.

Objectives

Methods

Results

Conclusion