Measuring adverse event rates in oncology inpatients using trigger tools: An assessment of measurement properties.

Abstract

235 Background: The Institute for Healthcare Improvement developed the Global Trigger Tool (GTT) for measuring harm rates over time in 2006. Few studies have been published on the measurement properties of GTT and no studies have assessed the value of adding specific modules to the generic GTT. We, therefore, aimed to determine inter-rater reliability, and to evaluate the effect of interrater variation on the reliability of the generic GTT to detect adverse event rates over time. Furthermore to evaluate the effect of adding an oncology specific module to the generic GTT on number and category of adverse events (AEs) identified. Methods: A retrospective chart review was performed by two teams consisting of two primary reviewers. One team used the general GTT, and the other used an oncology specific GTT (GTTO); consisting of the general GTT module plus an additional oncology module. A random sample of 10 charts was selected every two weeks between all discharged patients from a Department of Oncology in 2010. n=240. All charts were reviewed by both teams using standard GTT methods and measures. AEs per 1,000 admission days and AEs per 100 admissions where used to draw Statistical process control (SPC) charts for evaluation of the patient safety process. Inter-rater variability between review teams was assessed calculating the Kappa Cohen coefficient using data from the identical general GTT module. Results: Reliability between the two teams of reviewers to identify an AE was moderate [K=0.45]. A comparison of SPC charts, using the results from the identical general module, showed that moderate agreement gave rise to different conclusions on the patient safety process between teams. Comparing GTT with GTTO we found no significant difference (CI 95% [0.06-0.08]) in number of identified AEs and an equal distribution of the events on harm categories. GTT and GTTO identified 56 and 57 AEs respectively. Conclusions: Moderate agreement between review teams gave rise to different conclusions on the patient safety process suggesting limited use of the GTT to track harm rates over time. Addition of an oncology specific module showed no significant effect on the total number or distribution of AEs on harm categories.