RESPOND - Raising awareness of anemia in oncology centers through computer algorithm driven guidelines: Assessment of intraclass correlation metrics for the accuracy of algorithmic definitions

Abstract

19633 Background: Publication of evidence-based (EB) practice guidelines (EBPGs) does not assure their application in patient care. Computerized clinical decision support systems (CDSSs) incorporating EBPGs may promote EB decision-making about individual patients. Most CDSSs are knowledge-, not evidence-based. Effective CDSSs are those integrated into workflow, providing point of care guidance, and offering actionable recommendations. Few are validated in terms of algorithmic operationalization. RESPOND is a CDSS based on the 2006 update of the European Organisation for Research and Treatment of Cancer (EORTC) EBPGs for erythropoietic proteins in cancer-related anemia. Method: Descriptive study. Panel of five experts (three physicians, two nurses) rated the 27 algorithms derived from the EORTC EBPGs in terms of accuracy (binary: yes/no). If not judged accurate, experts were asked to explain and propose correction. Objective was to attain intraclass correlation coefficient (ICC) >0.90 for each operationalization in a maximum of three iterations. Table 1 . Example of operationalization of guideline Hb, hemoglobin Results: In iteration 1, three experts agreed with all 27 sets; one disagreed with one set and one with four sets; therefore ICC = 1.00 for 22 sets; ≤0.90 for five sets. Three sets concerned additional output to be generated, and one a substantive inaccuracy; all were corrected. One set concerned a suggestion not supported by evidence, hence not incorporated. In iteration 2, all five experts agreed with all sets, thus ICC = 1.00 for 27 sets. Conclusions: The RESPOND CDSS being developed and validated is based on algorithms that accurately specify, in the form of conditional dependency rules, the EORTC EBPGs for erythropoietic proteins. [Table: see text] [Table: see text]