Patient Safety Indicators Software Research Articles

The effect of health information technology implementation in Veterans Health Administration hospitals on patient outcomes

BackgroundThe impact of health information technology (HIT) in hospitals is dependent in large part on how it is used by nurses. This study examines the impact of HIT on the quality of care in hospitals in the Veterans Health Administration (VA), focusing on nurse-sensitive outcomes from 1995 to 2005. MethodsData were obtained from VA databases and original data collection. Fixed-effects Poisson regression was used, with the dependent variables measured using the Agency for Healthcare Research and Quality Inpatient Quality Indicators and Patient Safety Indicators software. Dummy variables indicated when each facility began and completed implementation of each type of HIT. Other explanatory variables included hospital volume, patient characteristics, nurse characteristics, and a quadratic time trend. ResultsThe start of computerized patient record implementation was associated with significantly lower mortality for two diagnoses but significantly higher pressure ulcer rates, and full implementation was associated with significantly more hospital-acquired infections. The start of bar-code medication administration implementation was linked to significantly lower mortality for one diagnosis, but full implementation was not linked to any change in patient outcomes. ConclusionsThe commencement of HIT implementation had mixed effects on patient outcomes, and the completion of implementation had little or no effect on outcomes. ImplicationsThis longitudinal study provides little support for the perception of VA staff and leaders that HIT has improved mortality rates or nurse-sensitive patient outcomes. Future research should examine patient outcomes associated with specific care processes affected by HIT.

Detecting adverse events in surgery: comparing events detected by the Veterans Health Administration Surgical Quality Improvement Program and the Patient Safety Indicators

BackgroundThe Patient Safety Indicators (PSIs) use administrative data to screen for select adverse events (AEs). In this study, VA Surgical Quality Improvement Program (VASQIP) chart review data were used as the gold standard to measure the criterion validity of 5 surgical PSIs. Independent chart review was also used to determine reasons for PSI errors. MethodsThe sensitivity, specificity, and positive predictive value of PSI software version 4.1a were calculated among Veterans Health Administration hospitalizations (2003–2007) reviewed by VASQIP (n = 268,771). Nurses re-reviewed a sample of hospitalizations for which PSI and VASQIP AE detection disagreed. ResultsSensitivities ranged from 31% to 68%, specificities from 99.1% to 99.8%, and positive predictive values from 31% to 72%. Reviewers found that coding errors accounted for some PSI-VASQIP disagreement; some disagreement was also the result of differences in AE definitions. ConclusionsThese results suggest that the PSIs have moderate criterion validity; however, some surgical PSIs detect different AEs than VASQIP. Future research should explore using both methods to evaluate surgical quality.

How Accurate is the AHRQ Patient Safety Indicator for Hospital-Acquired Pressure Ulcer in a National Sample of Records?

In 2008, we conducted a retrospective cross-sectional study to determine the test characteristics of the Agency for Healthcare Research and Quality patient safety indicator (PSI) for hospital-acquired pressure ulcer (PU). We sampled 1,995 inpatient records that met PSI 3 criteria and 4,007 records assigned to 14 DRGs with the highest empirical rates of PSI 3, which did not meet PSI 3 criteria, from 32 U.S. academic hospitals. We estimated the positive predictive value (PPV), sensitivity, and specificity of PSI 3 using both the software version contemporary to the hospitalizations (v3.1) and an approximation of the current version (v4.4). Of records that met PSI 3 version 3.1 criteria, 572 (PPV 28.3%; 95% CI 23.6-32.9%) were true positive. PU that was present on admission (POA) accounted for 76% of the false-positive records. Estimated sensitivity was 48.2% (95% CI 41.0-55.3%) and specificity 71.4% (95% CI 68.3-74.5%). Reclassifying records based on reported POA information and PU stage to approximate version 4.4 of PSI 3 improved sensitivity (78.6%; 95% CI 62.7-94.5%) and specificity (98.0; 95% CI 97.1-98.9%). In conclusion, accounting for POA information and PU staging to approximate newer versions of the PSI software (v4.3) moderately improves validity.

Comparison of the Agency for Healthcare Research and Quality Patient Safety Indicator Rates Among Veteran Dual Users

This study compares rates of 11 Agency for Healthcare Research and Quality Patient Safety Indicators (PSIs) among 266 203 veteran dual users (ie, those with hospitalizations in both the Veterans Health Administration [VA] and the private sector through Medicare fee-for-service coverage) during 2002 to 2007. PSI risk-adjusted rates were calculated using the PSI software (version 3.1a). Rates of pressure ulcer, central venous catheter-related bloodstream infections, and postoperative sepsis, areas in which the VA has focused quality improvement efforts, were found to be significantly lower in the VA than in the private sector. VA had significantly higher rates for 7 of the remaining 8 PSIs, although the rates of only 2 PSIs (postoperative hemorrhage/hematoma and accidental puncture or laceration) remained higher in the VA after sensitivity analyses were conducted. A better understanding of system-level differences in coding practices and patient severity, poorly documented in administrative data, is needed before conclusions about differences in quality can be drawn.

Using AHRQ Patient Safety Indicators to Detect Postdischarge Adverse Events in the Veterans Health Administration

Patient safety indicators (PSIs) use inpatient administrative data to flag cases with potentially preventable adverse events (AEs) attributable to hospital care. This study explored how many AEs the PSIs identified in the 30 days post discharge. PSI software was run on Veterans Health Administration 2003-2007 administrative data for 10 recently validated PSIs. Among PSI-eligible index hospitalizations not flagged with an AE, this study evaluated how many AEs occurred within 1 to 14 and 15 to 30 days post discharge using inpatient and outpatient administrative data. Considering all PSI-eligible index hospitalizations, 11 141 postdischarge AEs were identified, compared with 40 578 inpatient-flagged AEs. More than 60% of postdischarge AEs were detected within 14 days of discharge. The majority of postdischarge AEs were decubitus ulcers and postoperative pulmonary embolisms or deep vein thromboses. Extending PSI algorithms to the postdischarge period may provide a more complete picture of hospital quality. Future work should use chart review to validate postdischarge PSI events.

Examining the Impact of the AHRQ Patient Safety Indicators (PSIs) on the Veterans Health Administration

By focusing primarily on outcomes in the inpatient setting one may overlook serious adverse events that may occur after discharge (eg, readmissions, mortality) as well as opportunities for improving outpatient care. Our overall objective was to examine whether experiencing an Agency for Healthcare Research and Quality Patient Safety Indicator (PSI) event in an index medical or surgical hospitalization increased the likelihood of readmission. We applied the Agency for Healthcare Research and Quality PSI software (version 4.1.a) to 2003-2007 Veterans Health Administration inpatient discharge data to generate risk-adjusted PSI rates for 9 individual PSIs and 4 aggregate PSI measures: any PSI event and composite PSIs reflecting "Technical Care," "Continuity of Care," and both surgical and medical care (Mixed). We estimated separate logistic regression models to predict the likelihood of 30-day readmission for individual PSIs, any PSI event, and the 3 composites, adjusting for age, sex, comorbidities, and the occurrence of other PSI(s). The odds of readmission were 23% higher for index hospitalizations with any PSI event compared with those with no event [confidence interval (CI), 1.19-1.26], and ranged from 22% higher for Iatrogenic Pneumothorax (CI, 1.03-1.45) to 61% higher for Postoperative Wound Dehiscence (CI, 1.27-2.05). For the composites, the odds of readmission ranged from 15% higher for the Technical Care composite (CI, 1.08-1.22) to 37% higher for the Continuity of Care composite (CI, 1.26-1.50). Our results suggest that interventions that focus on minimizing preventable inpatient safety events as well as improving coordination of care between and across settings may decrease the likelihood of readmission.

Is Development of Postoperative Venous Thromboembolism Related to Thromboprophylaxis Use? A Case-Control Study in the Veterans Health Administration

Observational studies continue to report thromboprophylaxis underuse for postoperative pulmonary embolism/deep vein thrombosis (pPE/DVT) despite the long-standing existence of prevention guidelines. However, data are limited on whether thromboprophylaxis use differs between patients developing pPE/DVT versus those who do not or on why prophylaxis is withheld. Administrative data (2002-2007) from 28 Veterans Health Administration hospitals were screened for discharges with (1) pPE/DVT as flagged by the Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicator software and (2) pharmacoprophylaxis-recommended procedures, and the medical records were reviewed to ascertain true pPE/DVT cases. Controls were selected by matching cases by hospital, age, sex, diagnosis-related group, and predicted probability for developing pPE/DVT, and who underwent a pharmacoprophylaxis-recommended procedure. Records were assessed for "appropriate pharmacoprophylaxis use," defined primarily per American College of Chest Physicians (ACCP) guidelines, and reasons for anticoagulant nonuse. The 116 case-control pairs were similar in terms of demographics, surgery type, ACCP risk category, and appropriate pharmacoprophylaxis rates overall. Of the highest-risk patients, respective pharmacoprophylaxis rates among cases and controls were 88% versus 92% among hip/knee replacements and 31% versus 48% among cancer patients. Of the cases and controls who did not receive appropriate pharmacoprophylaxis, only about 25% had documented contraindications. Reviewers identified contraindications in 14% of cases and 9% of controls. Similarities in preventive pPE/DVT practice between cases and controls suggest that pPE/DVTs occur despite implementation of guideline-adherent practices.

Validating the Patient Safety Indicators in the Veterans Health Administration

The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) use administrative data to detect potentially preventable in-hospital adverse events. However, few studies have determined how accurately the PSIs identify true safety events. We examined the criterion validity, specifically the positive predictive value (PPV), of 12 selected PSIs using clinical data abstracted from the Veterans Health Administration (VA) electronic medical record as the gold standard. We identified PSI-flagged cases from 28 representative hospitals by applying the AHRQ PSI software (v.3.1a) to VA fiscal year 2003 to 2007 administrative data. Trained nurse-abstractors used standardized abstraction tools to review a random sample of flagged medical records (112 records per PSI) for the presence of true adverse events. Interrater reliability was assessed. We evaluated PPVs and associated 95% confidence intervals of each PSI and examined false positive (FP) cases to determine why they were incorrectly flagged and gain insight into how each PSI might be improved. PPVs ranged from 28% (95% CI, 15%-43%) for Postoperative Hip Fracture to 87% (95% CI, 79%-92%) for Postoperative Wound Dehiscence. Common reasons for FPs included conditions that were present on admission (POA), coding errors, and lack of coding specificity. PSIs with the lowest PPVs had the highest proportion of FPs owing to POA. Overall, PPVs were moderate for most of the PSIs. Implementing POA codes and using more specific ICD-9-CM codes would improve their validity. Our results suggest that additional coding improvements are needed before the PSIs evaluated herein are used for hospital reporting or pay for performance.

How Valid is the AHRQ Patient Safety Indicator “Postoperative Hemorrhage or Hematoma”?

Postoperative hemorrhage or hematoma (PHH), an Agency for Healthcare Research and Quality Patient Safety Indicator, uses administrative data to detect cases of potentially preventable postsurgical bleeding requiring a reparative procedure. How accurately it identifies true events is unknown. We therefore determined PHH's positive predictive value. Using Patient Safety Indicator software (v.3.1a) and fiscal year 2003-2007 discharge data from 28 Veterans Health Administration hospitals, we identified 112 possible cases of PHH. Based on medical record abstraction, we characterized cases as true (TPs) or false positives (FPs), calculated positive predictive value, and analyzed FPs to ascertain reasons for incorrect identification and TPs to determine PHH-associated clinical consequences and risk factors. Eighty-four cases were TPs (positive predictive value, 75%; 95% CI, 66-83%); 63% had a hematoma diagnosis, 30% had a hemorrhage diagnosis, 7% had both. Reasons for FPs included events present on admission (29%); hemorrhage/hematoma identified and controlled during the original procedure rather than postoperatively (21%); or postoperative hemorrhage/hematoma that did not require a procedure (18%). Most TPs (82%) returned to the operating room for hemorrhage/hematoma management; 64% required blood products and 7% died in-hospital. The most common index procedures resulting in postoperative hemorrhage/hematoma were vascular (38%); 56% were performed by a physician-in-training (under supervision). We found no substantial association between physician training status or perioperative anticoagulant use and bleeding risk. PHH's accuracy could be improved by coding enhancements, such as adopting present on admission codes or associating a timing factor with codes dealing with bleeding control. The ability of PHH to identify events representing quality of care problems requires additional evaluation.

Validity of Selected Patient Safety Indicators: Opportunities and Concerns

The Agency for Healthcare Research and Quality (AHRQ) recently designed the Patient Safety Indicators (PSIs) to detect potential safety-related adverse events. The National Quality Forum has endorsed several of these ICD-9-CM-based indicators as quality-of-care measures. We examined the positive predictive value (PPV) of 3 surgical PSIs: postoperative pulmonary embolus and deep vein thrombosis (pPE/DVT), iatrogenic pneumothorax (iPTX), and accidental puncture and laceration (APL). We applied the AHRQ PSI software (v.3.1a) to fiscal year 2003 to 2007 Veterans Health Administration (VA) administrative data to identify (flag) patients suspected of having a pPE/DVT, iPTX, or APL. Two trained nurse abstractors reviewed a sample of 336 flagged medical records (112 records per PSI) using a standardized instrument. Inter-rater reliability was assessed. Of 2,343,088 admissions, 6,080 were flagged for pPE/DVT (0.26%), 1,402 for iPTX (0.06%), and 7,203 for APL (0.31%). For pPE/DVT, the PPV was 43% (95% CI, 34% to 53%); 21% of cases had inaccurate coding (eg, arterial not venous thrombosis); and 36% featured thromboembolism present on admission or preoperatively. For iPTX, the PPV was 73% (95% CI, 64% to 81%); 18% had inaccurate coding (eg, spontaneous pneumothorax), and 9% were pneumothoraces present on admission. For APL, the PPV was 85% (95% CI, 77% to 91%); 10% of cases had coding inaccuracies and 5% indicated injuries present on admission. However, 27% of true APLs were minor injuries requiring no surgical repair (eg, small serosal bowel tear). Inter-rater reliability was >90% for all 3 PSIs. Until coding revisions are implemented, these PSIs, especially pPE/DVT, should be used primarily for screening and case-finding. Their utility for public reporting and pay-for-performance needs to be reassessed.

Accuracy of Reporting of Venous Thromboembolism in Hospitalized Patients.

Abstract 557 Introduction:Venous thromboembolism (VTE), which includes deep vein thrombosis (DVT) and pulmonary embolism (PE), is an important clinical event that needs to be accurately reported. The Agency for Healthcare Research and Quality (AHRQ) has categorized VTE as a reportable patient safety indicator (PSI) and has distributed a software package for recording events such as VTE based upon diagnosis codes. At our institution, there was concern that VTEs identified by this software method may be over estimating the actual number. Objective:To verify the cases of VTE as identified by AHRQ methodology through a review of the medical record for each event reported. Methods:Through a review of the electronic medical record (EMR), the events recorded as a VTE by PSI software over a 7 month period from November 2008 to May 2009 were further characterized into categories of acute, remote or age indeterminate DVTs and were distinguished from events that were either isolated superficial vein thrombosis (SVT) or that were not VTE at all. Discharge summaries, hospital notes, operative reports and imaging reports were used to verify identified cases of VTE. VTE occurrences were grouped by the month of discharge of the patient. The PSI software utilized diagnosis codes that have been abstracted for a patient's hospitalization based upon physician documentation. Results:437 events were recorded as VTE over a 7 month period. 59 cases (13.5%) were incorrectly coded as VTE. 35 cases (8.0%) were superficial vein thrombosis mistakenly classified as DVT. 12 cases (2.7%) had remote or age indeterminate DVTs. 12 cases (2.7%) cases had no VTE and were considered further abstraction errors. After interim reporting of the first 4 months of VTE data, vascular laboratory study reports were amended so that superficial vein thrombosis would no longer be identified by this terminology and would subsequently be classified as superficial vein thrombophlebitis. The last three months of data collection had 9 cases (4.7%) incorrectly identified as DVTs that were actually SVTs versus 26 cases (10.6%) prior to implementation of the nomenclature change. Conclusion:Hospitals must be aware of reporting practices in their own institution as national protocols may not completely depict individual hospital occurrences. Careful attention to the manner in which the medical record reflects events can make substantial difference in accounting of VTE occurrence. Because of the lack of specificity of current diagnosis coding guidelines, an appropriate medical term was not sufficient for medical record abstractors to categorize an event correctly in this study. As hospitals become monitored for quality of care with these methodologies, the accuracy of reporting becomes paramount.Summary of VTE Data (Nov 08–May 09)NovDecJanFebMarAprMayVTE Cases Reported61556069825456Age Indeterminate/Remote DVTs1113222SVTs6686504Non-VTE3321120Total Mistaken Reporting of VTE10 (16.4%)10 (18.2%)11 (18.3%)10 (14.5%)8 (9.8%)4 (7.4%)6 (10.7%) [Display omitted] [Display omitted] Disclosures:No relevant conflicts of interest to declare.

Clinical Validation of the AHRQ Postoperative Venous Thromboembolism Patient Safety Indicator

The Agency for Healthcare Research and Quality (AHRQ) patient safety indicators (PSIs) screen for potentially preventable complications in hospitalized patients using hospital administrative data. The PSI for postoperative venous thromboembolism (VTE) relies on International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes for deep vein thrombosis (DVT) or pulmonary embolism (PE) in secondary diagnoses fields. In a clinical validation study of the PSI for postoperative VTE, natural language processing (NLP), supplemented by pharmacy and billing data, was used to identify VTE events missed by medical records coders. In a retrospective review of postsurgical discharges, charts were processed using the AHRQ PSI software. Cases were identified as possible false negatives by flagging charts for possible VTEs using pharmacy and billing data to identify all patients who were therapeutically anticoagulated or had placement of an inferior vena caval filter. All charts were reviewed by a physician blinded to screening results. Physician interpretation was considered the gold standard for VTE classification. The AHRQ PSI had a positive predictive value (PPV) of .545 (95% confidence interval [CI], .453-.634) and a negative predictive value (NPV) of .997 (95% CI, .995-.999). Sensitivity was .87 and specificity was .98. Secondary coding review suggested that all 9 false-negative results were miscoded; if they had been properly coded, the sensitivity would increase to 1.00. Most false-positive cases resulted from superficial venous clots identified by the PSI due to coding ambiguity. The VTE PSI performed well as a screening tool but generated a significant number of false-positive cases, a problem that could be substantially reduced with improved coding methods.

Do the AHRQ Patient Safety Indicators Flag Conditions That Are Present at the Time of Hospital Admission?

The Agency for Healthcare Research and Quality (AHRQ) developed 20 patient safety indicators (PSIs) to identify potentially preventable complications of acute inpatient care based on administrative data. The objective of this patient safety performance study was to assess the impact of cases flagged by each PSI for diagnoses that were actually present on admission on unadjusted PSI rates. The latest AHRQ PSI software, which allows users to produce 14 of the 20 PSIs for adult inpatients both without and with a "present on admission" (PoA) variable, was applied to administrative data for adult patients discharged from the University of Michigan Health System (UMHS) in 2006. The impact of the PoA values on unadjusted PSI rates was evaluated. Because of concerns about the accuracy of PoA values, results were compared with those of a prior analysis at UMHS that was similar but based on a review of medical records. Thirteen PSIs had at least 1 case in the numerator. Rates for all but 1 of the 13 were lower using the PoA values and the reduction was statistically significant for 5 PSIs: decubitus ulcer (P < 0.001), foreign body left in (P = 0.033), selected infections due to medical care (P < 0.001), postoperative physiologic and metabolic derangement (P = 0.039), and postoperative pulmonary embolism or deep vein thrombosis (P < 0.001). Results were consistent with those of the analysis of medical records. Unadjusted PSI rates at UMHS are substantially overstated, because the PSIs do not differentiate preexisting conditions from complications and therefore include false positive cases. Because of these findings and the lack of a broader study of the validity of the indicators, PSIs should not be used to profile hospital performance.

Comparing measures of patient safety for inpatient care provided to veterans within and outside the VA system in New York

Context:Previous studies have compared measures of patient safety for veterans using the VA system to the general population. Discrepancies in the results of those studies suggest that the choice of...

Is the Availability of Hospital IT Applications Associated with a Hospital’s Risk Adjusted Incidence Rate for Patient Safety Indicators: Results from 66 Georgia Hospitals

This study examines the associations between the availability of IT applications in a hospital and that hospital's risk adjusted incidence rate per 1,000 hospitalizations for Agency for Healthcare Research and Quality's (AHRQ) 15 Patient Safety Indicators (PSIs). The study population consists of a convenience sample of 66 community hospitals in Georgia that completed a Hospital IT survey by December 2003 and provided data to Georgia Hospital Discharge Data Set during 2004. AHRQ's PSI software was used to estimate risk adjusted incidence rates. Differences in means, Pearson correlation coefficients, and multivariate regression analysis were used to determine if the availability of IT applications were associated with better PSI outcomes. This study finds very little statistically significant correlation between the availability of IT applications and risk adjusted PSI incident rate per 1,000