National Quality Forum Measure Research Articles

Review of the National Quality Forum's Measure Endorsement Process.

The National Quality Forum (NQF) evaluates healthcare performance measures for endorsement based on a broad set of criteria. We extracted data from NQF technical reports released between spring 2018 and spring 2019. Measures were commonly stewarded by federal agencies (44.29%), evaluated for maintenance (67.14%), classified as outcome (42.14%) or process (39.29%) measures, and used a statistical model for risk adjustment (48.57%). For 80% of the measures reviewed, a patient advocate was present on the reviewing committee. Validity was evaluated using face validity (65.00%) or score-level empirical validity (67.14%), and reliability was frequently evaluated using score-level testing (71.43%). Although 91.56% of all reviewed measures were endorsed, most standing committee members voted moderate rather than high support on key assessment criteria like measure validity, measure reliability, feasibility of use, and whether the measure addresses a key performance gap. Results show that although the Consensus Development Process includes multidisciplinary stakeholder input and thorough evaluations of measures, continued work to identify and describe appropriate and robust methods for reliability and validity testing is needed. Further work is needed to study the extent to which stakeholder input is truly representative of diverse viewpoints and improve processes for considering social factors when risk adjusting.

Using a brief tool to assess healthy school environments: a pilot study.

HealthPartners developed a checklist, the School Environment Index (SEI), that it uses to help elementary schools identify opportunities to improve student nutrition and increase physical activity. The objective in this pilot study was to assess whether the SEI, as administered, can be used to measure the progress of these programs. The authors focused their evaluation on the National Quality Forum measure evaluation components of reliability and validity; feasibility; and use and usability to assess the SEI's performance. The authors used data from 214 SEIs completed by the 69 schools that participated in the school challenge in at least 1 of the years 2015 through 2019. Between 29 and 53 schools participated in a particular year. Cronbach's α was 0.79, intraclass correlation was 0.36 (95% CI, 0.22-0.53), and sensitivity to change was 0.41 (95% CI, 0.17-0.66) per 1-year change in the standardized SEI score. The median (interquartile range) time required to complete the survey was 11 (7-21) minutes. On only 8 surveys was an entire domain of the SEI skipped or only a single response to the domain recorded. The SEI shows adequate internal consistency and sensitivity to change in this pilot evaluation. It is also feasible and useful to identify opportunities to improve practices and policies related to student nutrition and physical activity in partnership with the participating elementary schools. However, it lacks reliability as used. Increasing the number of respondents per school might moderate the impact of individual respondents and thereby increase reliability.

Establishing quality indicators for pre-hospital advanced airway management: a modified nominal group technique consensus process

BackgroundPre-hospital advanced airway management is a complex intervention composed of numerous steps, interactions, and variables that can be delivered to a high standard in the pre-hospital setting. Standard research methods have struggled to evaluate this complex intervention because of considerable heterogeneity in patients, providers, and techniques. In this study, we aimed to develop a set of quality indicators to evaluate pre-hospital advanced airway management. MethodsWe used a modified nominal group technique consensus process comprising three email rounds and a consensus meeting among a group of 16 international experts. The final set of quality indicators was assessed for usability according to the National Quality Forum Measure Evaluation Criteria. ResultsSeventy-seven possible quality indicators were identified through a narrative literature review with a further 49 proposed by panel experts. A final set of 17 final quality indicators composed of three structure-, nine process-, and five outcome-related indicators, was identified through the consensus process. The quality indicators cover all steps of pre-hospital advanced airway management from preoxygenation and use of rapid sequence induction to the ventilatory state of the patient at hospital delivery, prior intubation experience of provider, success rates and complications. ConclusionsWe identified a set of quality indicators for pre-hospital advanced airway management that represent a practical tool to measure, report, analyse, and monitor quality and performance of this complex intervention.

Gaps in standardized postoperative pain management quality measures: A systematic review

BackgroundThe goal of this study was an assessment of availability postoperative pain management quality measures and National Quality Forum–endorsed measures. Postoperative pain is an important clinical timepoint because poor pain control can lead to patient suffering, chronic opiate use, and/or chronic pain. Quality measures can guide best practices, but it is unclear whether there are measures for managing pain after surgery. MethodsThe National Quality Forum Quality Positioning System, Agency for Healthcare Research and Quality Indicators, and Centers for Medicare and Medicaid Services Measures Inventory Tool databases were searched in November 2019. We conducted a systematic literature review to further identify quality measures in research publications, clinical practice guidelines, and gray literature for the period between March 11, 2015 and March 11, 2020. ResultsOur systematic review yielded 1,328 publications, of which 206 were pertinent. Nineteen pain management quality measures were identified from the quality measure databases, and 5 were endorsed by National Quality Forum. The National Quality Forum measures were not specific to postoperative pain management. Three of the non-endorsed measures were specific to postoperative pain. ConclusionThe dearth of published postoperative pain management quality measures, especially National Quality Forum–endorsed measures, highlights the need for more rigorous evidence and widely endorsed postoperative pain quality measures to guide best practices.

Quality Indicators for Avalanche Victim Management and Rescue.

Decisions in the management and rescue of avalanche victims are complex and must be made in difficult, sometimes dangerous, environments. Our goal was to identify indicators for quality measurement in the management and rescue of avalanche victims. The International Commission for Mountain Emergency Medicine (ICAR MedCom) convened a group of internal and external experts. We used brainstorming and a five-round modified nominal group technique to identify the most relevant quality indicators (QIs) according to the National Quality Forum Measure Evaluation Criteria. Using a consensus process, we identified a set of 23 QIs to measure the quality of the management and rescue of avalanche victims. These QIs may be a valuable tool for continuous quality improvement. They allow objective feedback to rescuers regarding clinical performance and identify areas that should be the foci of further quality improvement efforts in avalanche rescue.

Testing the Use of Data Drawn from the Electronic Health Record to Compare Quality.

Introduction:Health systems spend $1.5 billion annually reporting data on quality, but efficacy and utility for benchmarking are limited due, in part, to limitations of data sources. Our objective was to implement and evaluate measures of pediatric quality for three conditions using electronic health record (EHR)-derived data.Methods:PCORnet networks standardized EHR-derived data to a common data model. In 13 health systems from 2 networks for 2015, we implemented the National Quality Forum measures: % children with sickle cell anemia who received a transcranial Doppler; % children on antipsychotics who had metabolic screening; and % pediatric acute otitis media with amoxicillin prescribed. Manual chart review assessed measure accuracy.Results:Only 39% (N = 2,923) of 7,278 children on antipsychotics received metabolic screening (range: 20%–54%). If the measure indicated screening was performed, the chart agreed 88% of the time [95% confidence interval (CI): 81%–94%]; if it indicated screening was not done, the chart agreed 86% (95% CI: 78%–93%). Only 69% (N = 793) of 1,144 children received transcranial Doppler screening (range across sites: 49%–88%). If the measure indicated screening was performed, the chart agreed 98% of the time (95% CI: 94%–100%); if it indicated screening was not performed, the chart agreed 89% (95% CI: 82%–95%). For acute otitis media, chart review identified many qualifying cases missed by the National Quality Forum measure, which excluded a common diagnostic code.Conclusions:Measures of healthcare quality developed using EHR-derived data were valid and identified wide variation among network sites. This data can facilitate the identification and spread of best practices.

Changes in Health Care Following COVID-19

Changes in Health Care Following COVID-19

Selecting Patient-Reported Outcome Measures to Contribute to Primary Care Performance Measurement: a Mixed Methods Approach.

New models of primary care include patient-reported outcome measures (PROMs) to promote patient-centered care. PROMs provide information on patient functional status and well-being, can be used to enhance care quality, and are proposed for use in assessing performance. Our objective was to identify a short list of candidate PROMs for use in primary care practice and to serve as a basis for performance measures (PMs). We used qualitative and quantitative methods to identify relevant patient-reported outcome (PRO) domains for use in performance measurement (PRO-PM) and their associated PROMs. We collected data from key informant groups: patients (n = 13; one-on-one and group interviews; concept saturation analysis), clinical thought leaders (n = 9; group discussions; thematic analysis), primary care practices representatives (n = 37; six focus groups; thematic analysis), and primary care payer representatives (n = 10; 12-question survey; frequencies of responses). We merged the key informant group information with findings from environmental literature scans. We conducted a targeted evidence review of measurement properties for candidate PROMs. We used a scoping review and key informant groups to identify PROM evaluation criteria, which were linked to the National Quality Forum measure evaluation criteria. We developed a de novo schema to score candidate PROMs against our criteria. We identified four PRO domains and 10 candidate PROMs: 3 for depressive symptoms, 2 for physical function, 3 for self-efficacy, 2 for ability to participate. Five PROMs met ≥ 70% of the evidence criteria for three PRO domains: PHQ-9 or PROMIS Depression (depression), PF-10 or PROMIS-PF (physical functioning), and PROMIS Self-Efficacy for Managing Treatments and Medications (self-efficacy). The PROMIS Ability to Participate in Social Roles and Activities met 68% of our criteria and might be considered for inclusion. Existing evidence and key informant data identified 5 candidate PROMs to use in primary care. These instruments can be used to develop PRO-PMs.

The NQF Scientific Methods Panel

In the summer of 2017, the National Quality Forum (NQF) announced the formation of a Scientific Methods Panel (hereafter referred to as "the Panel") as part of a redesign of its endorsement process. NQF created the Panel in response to stakeholder request during a Kaizen improvement event held in May 2017. Given the Panel's role in the endorsement of performance measures used in national payment programs, the objective of this article is to describe the work of the Panel, and to describe its function in the larger context of the NQF measure endorsement process and in the measurement enterprise writ large. This article also serves as an introduction to a series of planned white papers being authored by the panel on specific technical issues in the area of health care performance measurement.

Clinical trial participation and aggressive care at the end of life in patients with ovarian cancer

ObjectivesIn non-gynecologic cancers, clinical trial participation has been associated with aggressive care at the end of life. The objective of this investigation was to examine how trial participation affects end...

2069. A Unique Approach to Outpatient Antibiotic Stewardship in Rural Southern Ohio

BackgroundA five county rural community in southern Ohio was identified as having significantly higher than average rates of antibiotic use. The hospital system serving this area, Southern Ohio Medical Center (SOMC), began initial efforts in antimicrobial stewardship focusing on inpatient prescribing. However, most antimicrobial consumption occurs in the outpatient setting. Early attempts to improve antibiotic prescribing focused on only provider education and resulted in little change. Providers felt they were performing well, or their patients were more complex and prescribing the antibiotics was warranted. SOMC partnered with the state Quality Improvement Organization, HSAG, to design an intervention to address these challenges.MethodsAll outpatient and emergency room encounters with acute bronchitis and upper respiratory infection (URI) (ICD-10 codes [J00, J06.9, and J20.X]) were included in the analysis. Using criteria from a National Quality Forum measure, concomitant diagnoses were excluded to identify encounters where an associated condition may indicate the case is more complex. A 6-month baseline and two additional 6-month remeasurement periods were analyzed. Providers were given letters, peer-to-peer antimicrobial data comparison, and in-person feedback with guideline-driven recommendations for these conditions.ResultsBaseline analysis indicated 50% of all encounters without a coded concomitant diagnosis resulted in antibiotic prescriptions. There was a reduction in the overall rate at each remeasurement period, to 34% and then 12%. This resulted in a 76% relative improvement rate (RIR) overall at the final remeasurement period. At baseline, the highest volume setting, urgent care, had a prescribing rate of 71%. Urgent-care prescriptions reduced each remeasurement to 45% and 13%, resulting in an 81% RIR.ConclusionImplementing a robust outpatient stewardship program in a rural nonacademic setting is not without unique challenges. By using peer comparison of provider performance data on prescribing habits in uncomplicated patients with URI and acute bronchitis in addition to education, the rate of appropriate antibiotic use improved. Disclosures All authors: No reported disclosures.

Contraceptive Quality Measures: Population-Level Protection Using the Contraceptive Protection Index [1H

INTRODUCTION: National Quality Forum (NQF) endorsed contraception metric is now available to assess the proportion of women with reproductive capacity who are using highly effective contraception. The NQF measure does not assess the protection of the population from pregnancy based on the effectiveness of the methods and the distribution of method choice by age. The Contraceptive Protection Index (CPI) does incorporate these variables and can be calculated using the NQF measure data and may be instructive in monitoring population health over time. METHODS: We compare two observational cohorts (2013 and 2017) use of contraceptive methods in 54 agencies with some 200 health centers across the state of California. CPI was calculated using efficacy rates from clinical trials. Changes in CPI were evaluated by age group, race, ethnicity, and type of health center (Federally Qualified Health Centers, Health Departments, Planned Parenthoods, university and community clinics). RESULTS: Between 2013 and 2017, CPI increased for all groups. Teens (15-17), young adults (18-20) and women over 44 years old demonstrated the largest shift in CPI (increasing 1.4, 1.3, 1.6 respectively). This due to combination of decreases in condom or no method use with concomitant increases in LARC use. Federally Qualified Health Centers and Health Departments demonstrated the largest increases in CPI (1.4 and 1.9 respectively). CONCLUSION: Much of the shift in contraceptive use occurred within the group already using highly effective methods. Larger shifts in CPI are observed when women move from no method or condom use to highly effective methods, especially LARC use.

Implementing the Bereaved Family Survey for an advanced cancer population.

76 Background: Despite being endorsed as a National Quality Forum measure, the Bereaved Family Survey (BFS), a tool to assess a patient’s end-of-life (EOL) experience, has not been broadly used outside the Veteran’s Affairs Health System or in the outpatient setting. We adapted the BFS for an advanced cancer population and implemented it at an academic health system to identify areas for quality improvement in EOL care. Methods: Between August 2016 and May 2017, we surveyed caregivers of advanced cancer decedents. We included English-speaking decedents > 18 years of age who were continuity patients of a health system oncologist and had documentation of metastatic disease in either problem list, oncology notes, or advanced imaging. Death was verified in the chart or by obituary found via internet search. Caregivers surveyed were the appointed healthcare agent in an advance directive or the first listed contact. Scoring was done via the top-box approach. Surveys were mailed 3 months after death with a follow-up post-card reminder and phone-call at 2 and 4 weeks, respectively. Results: Of 285 eligible decedents, 242 caregivers were mailed surveys with 83 completed (34% response rate) of whom 28% died in the hospital. Mean overall BFS score was 75 out of 100. Ratings were high for overall care quality (78% Always/Usually) and staff caring (74% Always) with lower ratings for communication (58% Always), emotional/spiritual support (37% Always with 30% not wanting support) and pain control (56% Always/Usually uncomfortable). Nearly 84% of respondents felt their loved one died in the right place and 67% felt staff caring for the patient provided a dignified death. 74% of patients were referred to hospice with 64% of caregivers stating the hospice referral was timely. Write-in responses mentioned concerns about caregiver support, the quality and timeliness of hospice care and communication between physician and patient/caregiver. Over half of statements were positive remarks about a physician or ancillary staff member. Conclusions: A post-mortem survey implemented for an advanced cancer population received a modest response rate but collected valuable information regarding gaps in care quality to drive quality improvement.

Comparing Rural and Urban Cervical and Breast Cancer Screening Rates in a Privately Insured Population

ABSTRACTLow preventive screening varies by region and contributes to poor outcomes for breast and cervical cancer. Previous comparative urban and rural research on preventive screening has focused on government programs. This study quantified and compared rural and urban preventive cancer screening rates for women who were privately insured. National Quality Forum measures were used to calculate rates for women within rural and urban parts of the same Hospital Referral Region (HRR) using claims data. Mammography screening rates for women age 24 to 69 years were 77.1% in 2011 and 76.1% in 2008. Compared to urban women, mammography screening rates for women visiting rural physicians were lower in 42%, higher in 2% and identical in 56% of HRRs. Cervical cancer screening rates for women age 21 to 64 years were 82.9% in 2011 and 83.5% in 2008. Cervical cancer screening rates among women who saw rural physicians were lower in 55%, higher in 4%, and identical in 42% of HRRs. HRRs where rural areas underperformed urban areas increased between 2008 and 2011 for both screenings. Moderate but notable differences in women’s preventive screening rates between rural and urban physicians highlight the need for practical solutions that increase use of screening services and reduce barriers to services in rural areas.

A Multidisciplinary Patient Navigation Program Improves Compliance With Adjuvant Breast Cancer Therapy in a Public Hospital.

Cancer health disparities affecting low-income and minority patients have been well documented to lead to poor outcomes. This report examines the impact of patient navigation on adherence to prescribed adjuvant breast cancer treatment. A multidisciplinary patient navigation program was initiated at a public safety net hospital to improve compliance with 3 National Quality Forum measures: (1) administration of combination chemotherapy for women with Stage (defined by the American Joint Committee on Cancer [AJCC]) T1c, II, or III hormone receptor-negative breast cancer within 120 days; (2) administration of endocrine therapy for women with AJCC Stage T1c, II, or III hormone receptor-positive breast cancer within 365 days; and (3) radiation therapy for women receiving breast-conserving surgery within one year. Implementation of a multidisciplinary patient navigation program reduced time to treatment and improved compliance with adjuvant therapy for breast cancer in an underserved minority community.

Using health information exchanges to calculate clinical quality measures: A study of barriers and facilitators

BackgroundA key motivation for the large national investment in electronic health record systems is to promote electronic reporting of quality measures that can be used as the basis for moving to value-based payment. Given the fragmented delivery system, robust quality reporting requires aggregating data across sites of care. Health information exchanges (HIEs) have emerged to facilitate exchange of clinical data across provider organizations and, therefore, should be well-positioned to support clinical quality measure reporting. MethodsBy interviewing representatives from 36 HIEs across the United States, we aimed to determine whether HIEs are capable of computing National Quality Forum measures for 6 cardiovascular disease preventive services. ResultsEleven HIEs (30%) reported computing at least one CQM; six computed one or more of the measures, and no HIE computed a measure in each of the 6 areas. Barriers to computing CQMs included data quality, completeness, sharing, and transmission issues; organizational structure, maturity, and sustainability issues; and vendor issues. ConclusionsThe ability to compute CQMs at the HIE level is still yet to be developed; currently, very few HIEs are able to do so for a variety of reasons. As HIE services expand and HIEs mature organizationally, the viability and utility of CQM reporting at the HIE level will increase. ImplicationsAs the healthcare system migrates towards a value-based payment system these broad challenges will need to be addressed. Level of EvidenceCross-sectional semi-structured qualitative interviews

Evaluation of spiritual needs assessments in oncology palliative care: Results of an improvement project.

161 Background: Addressing all facets of serious illness requires attention to patients’ spiritual concerns. At our Cancer Center, we sought to ensure that palliative care (PC) inpatients received evidence-based spiritual care. Methods: Chaplaincy Care, Supportive Care Medicine, and Quality and Safety Departments partnered to improve spiritual care using an iterative Plan-Do-Check-Act Cycle approach. Evidence-based spiritual practices were defined in accordance with the Association of Professional Chaplains and Clinical Practice Guidelines for Quality PC of the National Consensus Project. As a member of the Global Palliative Care Quality Alliance, we also used the web-based Quality Data Collection Tool (QDACT) to guide spiritual assessment. Electronic health record templates were revised to support complete and timely documentation. We monitored performance based on National Quality Forum Measure #1647 for discussion of spiritual/religious concerns. Performance was reviewed monthly by Leadership and with the Chaplaincy Staff. Opportunities to improve performance were discussed and adjustments in practice were made. Results: A total of 1609 oncology patients (average of 89/month) were seen by the inpatient PC team from November 2013–April 2015. With transparent performance reporting and resolution of barriers to spiritual assessment, an increase from 36 to 96% was noted for the percentage of patients with documented spiritual or religious concerns. Spiritual assessment using QDACT indicated that most patients report high spiritual well-being, spiritual peace, and a religious affiliation. Conclusions: As the result of our efforts, the majority of PC oncology patients at our center now have assessment and documentation of spiritual and religious concerns. A team-based approach to quality improvement successfully fostered the integration of spiritual care into oncology palliative care. [Table: see text]

The "comfortable dying" measure: how patient characteristics affect hospice pain management quality scores.

All hospices were required by the Centers for Medicare and Medicaid Services (CMS) to collect the "Comfortable Dying" measure in 2012 (National Quality Forum measure #0209). However, it is not known how scores on this measure are affected by patient characteristics. It is important to identify these characteristics so that a hospice's case mix can be taken into account when interpreting its scores. Our aim was to describe the implementation of the NQF #0209 measure in 10 hospices and to identify patient characteristics associated with scores. We conducted an electronic health record (EHR)-based retrospective cohort study of patients in 10 hospices in the United States. The main outcome measure was the proportion of patients with pain that made them uncomfortable whose pain was controlled within 48 hours. A total of 4157 patients were eligible for an initial pain assessment. Of those who reported pain (n=1992), 1152 (58%) reported having their pain controlled on the follow-up assessment. In a multivariable regression model, clustered by hospice, six variables were independently associated with pain control. These included age (adjusted odds ratio [OR] 1.02; 95% confidence interval [CI] 1.02-1.03, p=0.003), a cancer diagnosis (OR 1.37; 95% CI 1.20-1.53, p=0.008), initial care in an inpatient unit (OR 1.28; 95% CI 1.08-1.47, p=0.031), presence of a Foley catheter (OR 1.40; 95% CI 1.15-1.59, p=0.038), use of opioid medication (OR 1.34; 95% CI 1.03-1.74, p=0.027), and higher Palliative Performance Scale (PPS) score (OR 1.02; 95% CI 1.01-1.03, p<0.001). Presence of a Stage 2 pressure ulcer was independently associated with worse pain control (OR 0.63; 95% CI 0.31-0.96, p=0.012). Several patient characteristics are associated with #0209 pain scores. As hospices are increasingly required to report quality measures, it will be essential to understand how their scores are affected by case mix.

Radiation Dose Metrics in CT: Assessing Dose Using the National Quality Forum CT Patient Safety Measure

The National Quality Forum (NQF) is a nonprofit consensus organization that recently endorsed a measure focused on CT radiation doses. To comply, facilities must summarize the doses from consecutive scans within age and anatomic area strata and report the data in the medical record. Our purpose was to assess the time needed to assemble the data and to demonstrate how review of such data permits a facility to understand doses. To assemble the data we used for analysis, we used the dose monitoring software eXposure to automatically export dose metrics from consecutive scans in 2010 and 2012. For a subset of 50 exams, we also collected dose metrics manually, copying data directly from the PACS into an excel spreadsheet. Manual data collection for 50 scans required 2 hours and 15 minutes. eXposure compiled the data in under an hour. All dose metrics demonstrated a 30% to 50% reduction between 2010 and 2012. There was also a significant decline and a reduction in the variability of the doses over time. The NQF measure facilitates an institution's capacity to assess the doses they are using for CT as part of routine practice. The necessary data can be collected within a reasonable amount of time either with automatic software or manually. The collection and review of these data will allow facilities to compare their radiation dose distributions with national distributions and allow assessment of temporal trends in the doses they are using.

Challenges in Evaluating All-Cause Hospital Readmission Measures for Use as National Consensus Standards

The National Quality Forum (NQF) aims to improve the quality of health care for all Americans through fulfillment of its three-part mission. The NQF uses its formal Consensus Development Process to evaluate and endorse consensus standards, including performance measures, best practices, frameworks, and reporting guidelines. To understand the opportunities and challenges in endorsing measures addressing all-cause readmissions to hospitals for use as national voluntary consensus standards for accountability and quality-improvement purposes. Report of standards development process. The Consensus Development Process was used to evaluate 3 candidate standards using the NQF Measure Evaluation Criteria. A 21-member steering committee rated each standard according to the criteria and made initial endorsement recommendations for all measures. Through the evaluation of measures for endorsement, several overarching issues in measuring all-cause readmissions were identified, including statistical modeling and the usability of the measures for quality improvement and accountability. Additionally, it was decided that, for the first time, quality monitoring and accountability of readmissions will take place at the health-plan level. Measuring at various levels of accountability reinforces the idea that multiple stakeholders have a responsibility and a role to reduce readmissions. These NQF-endorsed measures are a major step in promoting better understanding of readmissions and a reduction in hospital readmission rates, when appropriate. These measures can help reduce the substantial financial and emotional stress that readmissions place on the health care system, and patients will be able to communicate hospital-level performance on this important quality indicator.