Accuracy of the Adverse Outcome Index: An Obstetrical Quality Measure

Abstract

In obstetrics, a nationally accepted set of quality indicators for patient safety was not available in the United States until the development of a set of 10 adverse outcome measures-the Adverse Outcome Index (AOI). The National Perinatal Information Center (NPIC) developed hospital discharge data-based algorithms combined with a small set of supplemental patient data for calculation of the AOI. A study was conducted to determine the specificity, sensitivity, positive predictive value (PPV), and negative predictive value (NPV) of the AOI by using the National Perinatal Information Center (NPIC) algorithm. A retrospective chart review of 4,252 obstetrical and neonatal charts from 2003 through 2007 was performed. NPIC definitions were compared with the "gold standard"-chart review. A total of 229 deliveries among the 4,000 randomly selected charts had at least one adverse outcome, reflecting an AOI of 5.7%. For detection of the 10 adverse outcomes within the AOI, the overall sensitivity of the AOI was 81.7%, specificity was 98.2%, PPV was 86.3%, and NPV was 97.4%. The Kappa value for agreement between the coded charts and the chart review was 0.82 (standard deviation=0.01, 95% confidence interval [CI]=0.80-0.85), which is considered very good. The AOI is highly reliant on accurate coding and provider documentation and requires validation with manual chart review. Concurrent chart review improves the accuracy of the AOI. Caution is advised when using the AOI as an exclusive measure of assessing obstetric quality because it may be heavily influenced by a single outcome measure; perineal laceration rates represented twice the frequency of all other outcomes combined. The AOI should be modified to better measure preventable adverse events and include a means of accounting for preexisting conditions.