Radiology Order Decision Support: Examination-Indication Appropriateness Assessed Using 2 Electronic Systems

Abstract

PurposeThe goal of the study was to determine the effects of guideline implementation strategy using 2 commercial radiology clinical decision support (CDS) systems. MethodsThe appropriateness and insurance dispositions of MRI and CT orders were evaluated using the Medicalis SmartReq and Nuance RadPort CDS systems during 2 different 3-month periods. Logistic regression was used to compare these outcomes between the 2 systems, after adjusting for patient-mix differences. ResultsApproximately 2,000 consecutive outpatient MRI and CT orders were evaluated over 2 periods of 3 months each. Medicalis scored 60% of exams as “indeterminate” (insufficient information) or “not validated” (no guidelines). Excluding these cases, Nuance scored significantly more exams as appropriate than did Medicalis (80% versus 51%, P < .001) and predicted insurance outcome significantly more often (76% versus 58%, P < .001). Only when the Medicalis “indeterminate” and “not validated” categories were combined with the high- or moderate-utility categories did the 2 CDS systems have similar performance. Overall, 19% of examinations with low-utility ratings were reimbursed. Conversely, 0.8% of examinations with high- or moderate-utility ratings were denied reimbursement. ConclusionsThe chief difference between the 2 CDS systems, and the strongest influence on outcomes, was how exams without relevant guidelines or with insufficient information were handled. Nuance augmented published guidelines with clinical best practice; Medicalis requested additional information utilizing pop-up windows. Thus, guideline implementation choices contributed to decision making and outcomes. User interface, specifically, the number of screens and completeness of indication choices, controlled CDS interactions and, coupled with guidance implementation, influenced willingness to use the CDS system.