Reprint of: Development and Evaluation of a Global Malnutrition Composite Score.

Abstract

Quality measures proliferated in the late 1990s and early 2000s and were first tied to financial performance incentives with the establishment of quality reporting programs for hospitals and then physicians.1Marjoua Y. Bozic K.J. Brief history of quality movement in US healthcare.Curr Rev Musculoskelet Med. 2012; 5: 265-273Crossref PubMed Scopus (75) Google Scholar Quality measurement has since expanded to virtually all provider areas of health care in the United States. Despite this growth, one area where a major deficit persists has been nutrition care. This article outlines the process pursued by the Academy of Nutrition and Dietetics (Academy) and Avalere Health (Avalere) to develop the first of its kind electronically specified composite measure addressing malnutrition care for hospitalized adults.Quality Measurement in Malnutrition CareIn the United States, national surveillance data from 2016 indicates that as many as 8% of hospitalized adults have a diagnosis of malnutrition.2Barrett M.L. Bailey M.K. Owens P.L. Non-maternal and non-neonatal inpatient stays in the United States involving malnutrition, 2016.https://www.hcup-us.ahrq.gov/reports/HCUPMalnutritionHospReport_083018.pdfDate accessed: December 9, 2020Google Scholar However, previous studies suggest that malnutrition and malnutrition risk may actually be found in as many as 20% to 50% of hospitalized patients, indicating a significant gap in the identification of malnutrition.3Pereira G.F. Bulik C.M. Weaver M.A. Holland W.C. Platts-Mills T.F. Malnutrition among cognitively intact, noncritically ill older adults in the emergency department.Ann Emerg Med. 2015; 65: 85-91Abstract Full Text Full Text PDF PubMed Scopus (37) Google Scholar Malnutrition is a critical predictor for inpatient adverse outcomes given its association with 30-day readmissions, length of stay (LOS), complications, and mortality.2Barrett M.L. Bailey M.K. Owens P.L. Non-maternal and non-neonatal inpatient stays in the United States involving malnutrition, 2016.https://www.hcup-us.ahrq.gov/reports/HCUPMalnutritionHospReport_083018.pdfDate accessed: December 9, 2020Google Scholar,4Barker L.A. Gout B.S. Crowe T.C. Hospital malnutrition: prevalence, identification and impact on patients and the healthcare system.Int J Environ Res Public Health. 2011; 8: 514-527Crossref PubMed Scopus (527) Google Scholar Despite this major gap in identification, no public quality reporting programs include performance measures focused on nutrition care or malnutrition.Driven by the consistent and expanding evidence of the high prevalence of malnutrition in hospitalized patients across the United States, the Academy, along with Avalere and other stakeholders, developed and implemented the Malnutrition Quality Improvement Initiative (MQii). The MQii was established largely in response to the need for assessment quality of care provided to hospitalized patients who are malnourished or at risk of malnutrition.5Academy of Nutrition and Dietetics Malnutrition Quality Improvement Initiative.https://www.eatrightpro.org/practice/practice-resources/clinical-malnutritionDate accessed: December 9, 2020Google Scholar Through a dual-pronged approach, the MQii supports quality improvement (QI) for malnutrition care based on a set of four malnutrition-focused electronic clinical quality measures and a complementary MQii Toolkit that includes resources guiding implementation of QI activities.6McCauley S.M. Barrocas A. Malone A. Hospital nutrition care betters patient clinical outcomes and reduces costs: the Malnutrition Quality Improvement Initiative story.J Acad Nutr Diet. 2019; 119: S11-S14Abstract Full Text Full Text PDF PubMed Scopus (14) Google Scholar,7Silver H.J. Pratt K.J. Bruno M. Lynch J. Mitchell K. McCauley S.M. Effectiveness of the Malnutrition Quality Improvement Initiative on practitioners malnutrition knowledge and screening, diagnosis, and timeliness of malnutrition-related care provided to older adults admitted to a tertiary care facility: a pilot study.J Acad Nutr Diet. 2018; 118: 101-109Abstract Full Text Full Text PDF PubMed Scopus (34) Google Scholar A multistakeholder collaboration identified measure gaps in malnutrition care, which were translated into a set of individual electronic clinical quality measures (eCQMs). As part of the measure evaluation process, a technical expert panel had also been convened to weigh in on the initial measure concepts from both a clinical and technical perspective regarding data feasibility. These eCQMs were subsequently piloted at a large hospital in the Midwest, and the testing results demonstrated that the measures were usable for identifying key improvement areas in malnutrition care related to identifying risk, assessing for clinical malnutrition, developing the appropriate care plan, and ensuring the diagnosis of malnutrition is documented to support follow up care.8Nepple K.G. Tobert C.M. Valladares A.F. Mitchell K. Yadrick M. Enhancing identification and management of hospitalized patients who are malnourished: a pilot evaluation of electronic quality improvement measures.J Acad Nutr Diet. 2019; 119: S32-S39Abstract Full Text Full Text PDF PubMed Scopus (10) Google Scholar The individual eCQMs that were tested are outlined in Figure 1.Figure 1Individual malnutrition-focused electronic clinical quality measures (eCQMs).eCQMMeasure nameMeasure description1Completion of a malnutrition screening within 24 h of admissionPatients aged ≥18 y who received a malnutrition screening and results are documented in their medical record within 24 h of their admission to the hospital2Completion of a nutrition assessment for patients identified as at risk for malnutrition within 24 h of a malnutrition screeningPatients aged ≥65 y who were identified to be at risk of malnutrition from a screening were provided a nutrition assessment within 24 h of the screening3Nutrition care plan for patients identified as malnourished after a completed nutrition assessmentPatients aged ≥65 y who were assessed and found to be malnourished should also have a documented nutrition care plan in their medical record4Appropriate documentation of a malnutrition diagnosisPatients aged ≥65 y who were assessed and found to be malnourished should have a physician-confirmed diagnosis of malnutrition documented in their medical record to ensure care plan implementation and transfer of necessary medical information upon discharge Open table in a new tab The initial pilot testing of these novel malnutrition-focused eCQMs demonstrated that it was feasible to collect the data from existing hospital electronic health record systems, and that the measures met minimum reliability and validity testing requirements as established by expert consensus.9National Quality ForumMeasure evaluation criteria and guidance for evaluating measures for endorsement.http://www.qualityforum.org/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=88439Date accessed: December 9, 2020Google Scholar Subsequently, the tested measures were adopted by a national learning collaborative of hospitals all implementing the principles of the MQii. A group of 27 US hospitals reported use of the four eCQMs to guide various QI projects focused on improving care provided to hospitalized patients who are malnourished or at risk of malnutrition.10Valladares A.F. Kilgore K.M. Partridge J. Sulo S. Kerr K.W. McCauley S. How a malnutrition quality improvement initiative furthers malnutrition measurement and care: results from a hospital learning collaborative.JPEN J Parenter Enteral Nutr. 2021; 45: 366-371Crossref PubMed Scopus (11) Google Scholar The participating collaborative hospitals reported changes in measure performance based on implementation of cyclical quality improvement initiatives at their respective institutions. With this new aggregate data, multivariate analyses were conducted to identify the relationships between performance on these implemented eCQMs with patient outcomes of 30-day readmission and LOS. The study results concluded that the measures could be successfully implemented in a cohort of diverse hospitals in the United States. Furthermore, the study demonstrated that when supported by QI tools, the hospitals were able to see meaningful improvements in measure performance. In addition, the multivariate analysis demonstrated that all four measures were significantly associated with outcomes of 30-day readmissions and patient LOS.10Valladares A.F. Kilgore K.M. Partridge J. Sulo S. Kerr K.W. McCauley S. How a malnutrition quality improvement initiative furthers malnutrition measurement and care: results from a hospital learning collaborative.JPEN J Parenter Enteral Nutr. 2021; 45: 366-371Crossref PubMed Scopus (11) Google ScholarThe Global Malnutrition Composite ScoreThese initial studies were crucial in establishing the evidentiary basis for the four malnutrition-focused eCQMs, which were being adopted across dozens of hospital systems throughout the country. As work with the Centers for Medicare and Medicaid Services (CMS) continued, an external panel of experts from CMS provided feedback to develop a composite measure—The Global Malnutrition Composite Score (GMCS)—constructed from the four individual eCQMs. CMS defines a composite measure as a performance measure representing a “combination of two or more component measures, each of which individually reflects quality of care, into a single performance measure with a single score.”11CMS Measures Management System Blueprint v14.1.http://cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/MMS-Blueprint.htmlDate accessed: December 9, 2020Google Scholar Composite measures facilitate the grouping of multiple quality of care constructs into a single value that more comprehensively assesses quality. The intent is that the composite performance score can be influenced in some way by each component score and have a summary score that can reflect the totality of the components. The feedback produced by a composite measure condenses a broader range of metrics that would be more challenging to otherwise assess comprehensively.Best practices for composite measure development11CMS Measures Management System Blueprint v14.1.http://cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/MMS-Blueprint.htmlDate accessed: December 9, 2020Google Scholar include ensuring all included component measures have a common orientation or focus (eg, domains of a specific area of health care quality) that may be broad in nature (overall quality of care in a disease state) or narrow (adherence to a specific set of guidance), composite component measures are justified by clinical evidence, presence of a demonstrated a gap in care or outcomes, and component measures are empirically evaluated for reliability and validity.Learning from the experience of building and testing the individual malnutrition-focused eCQMs referenced above, the Academy and Avalere (the measure development team) launched a composite measure development process to implement the feedback received from the external review.Developing the GMCSThe measure development team studied existing composite measures and identified an initial framework and objective for the eventual composite measure. Informed by the experience of the hospitals implementing the individual malnutrition-focused eCQMs, the development team determined the focus of the proposed composite measure would be on optimal malnutrition care for adults aged 65 years and older who are admitted to inpatient service and receive care appropriate to their level of malnutrition risk and/or malnutrition diagnosis if identified.The GMCS includes four component measures that are first scored separately as proportion measures.12Academy of Nutrition and DieteticsElectronic clinical quality measures: Global Malnutrition Composite Score.https://www.eatrightpro.org/-/media/eatrightpro-files/pr1ctice/quality-management/quality-improvement/malnutritioncompositescorehumanreadable.html?la=en&hash=078561776426C7E32CC7C5F8D3AC2574F53958ADDate accessed: December 9, 2020Google Scholar The four component measures (Figure 2) represent slight variations from the original individual eCQMs (Figure 1). The composite measure components were established using empirical testing by determining which individual components would most contribute to a sound overall composite score. The overall composite score is derived from averaging the individual performance scores for the following component measures:1.Screening for malnutrition risk at admission;2.Completing a nutrition assessment for patients who screened for risk of malnutrition;3.Appropriate documentation of malnutrition diagnosis in the patient’s medical record when this is indicated by the assessment findings; and4.Development of a nutrition care plan for malnourished patients, including the recommended treatment plan.Figure 2Global malnutrition composite score component measure details.Component measure nameDenominatorNumeratorScreening for Malnutrition Risk at AdmissionAll patients in the measure population with a documented malnutrition screening no more than 48 h before admission to the hospitalAll patients in the measure population who are documented as at risk for malnutrition via the completed malnutrition screeningCompletion of a Nutrition Assessment for Patients who Screened for Risk of MalnutritionPatients from the measure population who are documented as at risk for malnutrition via the completed malnutrition screeningPatients at risk of malnutrition who have a completed nutrition assessment documentedAppropriate Documentation of Malnutrition Diagnosis for Patients Identified with MalnutritionPatients from the measure population who have a completed nutrition assessment documented with findings of moderate or severe malnutritionPatients who have been identified as moderately or severely malnourished by the nutrition assessment who also have a documented medical diagnosis of malnutrition in their medical recordDevelopment of a Nutrition Care Plan for Malnourished PatientsPatients from the measure population who have a documented medical diagnosis of malnutrition in their medical recordPatients with a documented medical diagnosis of malnutrition in their medical record who have a documented nutrition care plan with treatment recommendations to address malnutrition Open table in a new tab The process for risk identification, diagnosis, and treatment of malnutrition necessitates a multidisciplinary care team that begins with identification of an initial risk population for more thorough assessment by a registered dietitian nutritionist (RDN). An RDN, in turn, provides the necessary treatment recommendations to address nutritional status and the clinical indicators that inform a medical diagnosis of malnutrition documented by a physician. As described above, the four component measures individually only provide a fraction of the necessary information on quality of care for patients at risk of malnutrition. For example, knowing which patients have been assessed out of those who were initially identified as at risk, but not knowing whether or not the appropriate proportion of patients were screened upon admission would be an insufficient assessment of quality of care. Therefore, the composite measure offers a more comprehensive assessment of follow-through on best practices for patients at risk of malnutrition. One analogous example can be found in a composite measure developed to better assess quality of care for patients with type 1 diabetes. Although glycated hemoglobin level is the traditional marker for glycemic control, guidelines promote several other steps to care for patients with type 1 diabetes and track their outcomes more comprehensively. Therefore, the reported composite measure includes three domains of care with corresponding component measures: management tools, diabetes care assessment, and complications risk.13Indyk J.A. Buckingham D. Obrynba K.S. Servick C. Gandhi K.K. Kramer A. et al.The Type 1 Diabetes Composite Score: an innovative metric for measuring patient care outcomes beyond hemoglobin A1c.Pediatr Qual Saf. 2020; 5: e354https://doi.org/10.1097/pq9.0000000000000354Crossref PubMed Google ScholarComposite measures reflect the individual component measures of which they are comprised. Consequently, a composite measure’s validity and usefulness is dependent on the accuracy of the individual measures as well as the methodology for combining the individual measures into a combined measure. The National Quality Forum outlines four key steps to developing a composite measure:14National Quality ForumComposite performance measure evaluation guidance.https://www.qualityforum.org/Publications/2013/04/Composite_Performance_Measure_Evaluation_Guidance.aspxDate accessed: January 20, 2021Google Scholar1.Defining the composite measure’s purpose and theoretical framework,2.Selecting the appropriate individual component measures for inclusion,3.Establishing the methodology for combining the selected component measures, and4.Empirically testing the validity and reliability of the overall compositeSimilar to individual performance measures, composite measures must also be tested to demonstrate reliability (that the measure is well defined and can be implemented consistently within and across measured entities), and validity (that the measure logic and scoring accurately captures the intent of the measure).11CMS Measures Management System Blueprint v14.1.http://cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/MMS-Blueprint.htmlDate accessed: December 9, 2020Google Scholar In the case that individual components have not been empirically tested, developers should first ensure that the individual components are empirically sound and robust before considering their inclusion in the composite. After being evaluated individually, the components can be incorporated in one of the many aggregation approaches outlined above and tested as a set to ensure reliability and validity in predetermined combination.GMCS Development MethodologyThe findings from the outcome analyses were used to inform the inclusion and exclusion of data elements and modifications to the existing individual malnutrition eCQMs in a newly proposed GMCS.10Valladares A.F. Kilgore K.M. Partridge J. Sulo S. Kerr K.W. McCauley S. How a malnutrition quality improvement initiative furthers malnutrition measurement and care: results from a hospital learning collaborative.JPEN J Parenter Enteral Nutr. 2021; 45: 366-371Crossref PubMed Scopus (11) Google Scholar As determined by the empirical validity testing outlined in the Composite Measure Evaluation section below, each of the main components of this measure is strongly correlated with outcomes that have been empirically associated with malnutrition, including 30-day readmissions and hospital LOS. The measure development team identified that each component was correlated in a significant way to both malnutrition as a clinical outcome as well as the sequelae of untreated malnutrition, including readmissions and longer LOS. Components that were excluded from the proposed GMCS included process-oriented timing intervals for admission-to-screening and screening-to-assessment to reduce methodological complexity of the GMCS.Rationale for Measure ScoringThe approach to measure scoring is considerably important for composite measures because they are a mathematical computation of multiple individual metrics. In practice, there are several ways that composite measures may be scored via the included components. Common methods include all-or-none scoring where a binary outcome of performance met or not met only occurs when performance is met on all components, any-or-none scoring where the performance on the composite is met if at least one of the component measure’s performance criteria is met, opportunity scoring is a patient-based scoring method that is based on a particular number of care events being met for a patient in the measure, linear scoring where the composite score is based on a sum of component scores in which the performance is met, and weighted scoring where each component score is assigned a weight factor and the overall composite score is the sum of the weighted scores.The measure development team proposed constructing the composite measure as an arithmetic average of the four components weighed equally, given that all components were significantly correlated to the important outcomes of malnutrition, 30-day readmissions, and LOS. In clinical practice, all four steps are critical components of the nutrition care process. Patients who are diagnosed and treated for malnutrition by a care team are often first identified by a nutrition screening for malnutrition risk around the time of admission. Next, based on the screening results, the patient is referred to an RDN for assessment and recommendations for malnutrition diagnosis and nutrition intervention. The names, denominators, and numerators of the final composite measure components are outlined in Figure 2. Each measure component is a proportion with a possible performance score of 0 to 100%. After each component score is calculated, an unweighted average of all four scores is completed to determine the final composite score with a total score ranging from 0 to 100%.Composite Measure Evaluation MethodologyAs outlined by National Quality Forum’s Measure Evaluation Criteria and Guidance,14National Quality ForumComposite performance measure evaluation guidance.https://www.qualityforum.org/Publications/2013/04/Composite_Performance_Measure_Evaluation_Guidance.aspxDate accessed: January 20, 2021Google Scholar all performance measures should be tested for reliability and validity to ensure they are precise or repeatable (indicating reliability), and they accurately reflect quality of care provided and can identify differences in quality (indicating validity). To that end, a large analytic patient dataset was assembled from data reported by 56 acute care hospitals across 10 states (N = 179,336). Data were collected at the encounter level and included information on each patient encounter: LOS, discharge status, and 30-day readmission flag, screening for malnutrition risk, nutrition assessment and subsequent nutrition care plan development by an RDN, and diagnosis of malnutrition by the attending physician. Data quality was a concern for the time-to-screening data point for patients above the 99th percentile and were therefore excluded from the analysis. The capture of screening data longer than 48 hours before admission was also not included in the dataset because they are considered not to be clinically reliable.Reliability TestingThe goal of reliability testing of performance measures is to demonstrate that the calculated composite score can detect true differences between measured entities (eg, hospitals or clinicians) from random measurement error.14National Quality ForumComposite performance measure evaluation guidance.https://www.qualityforum.org/Publications/2013/04/Composite_Performance_Measure_Evaluation_Guidance.aspxDate accessed: January 20, 2021Google Scholar Mathematically, a reliability coefficient is estimated as the ratio of true variance over the sum of the true variance and error variance. The value is constrained to fall between 0 and 1, with values closer to 1 indicating stronger reliability. In most instances, a reliability index of 0.70 or greater is regarded as acceptable for drawing conclusions about groups, although a value of 0.80 or higher is desirable. In the present study, GMCS reliability was assessed by fitting an intercept-only generalized linear mixed model (GLMM) to the composite score data. The GLMM variance components were then used to estimate the intraclass correlation coefficient (ICC), a variant of the Pearson correlation coefficient that is widely used to assess the reliability of group- and cluster-based measurements.15Adams J.L. The reliability of provider profiling: a tutorial.https://www.rand.org/pubs/technical_reports/TR653.htmlDate accessed: December 31, 2020Google ScholarValidity Testing at Composite Score LevelComposite measures validity testing seeks to confirm that the composite measure score is highly associated with outcomes that have been supported by the evidence (30-day readmissions and LOS). The overall composite measure was first tested for construct validity at the score level by constructing a hierarchical linear regression model. The hierarchical linear regression model was conducted to demonstrate that the predictability of the model significantly improved when the components in aggregate were included into the model over standard predictors of these outcomes such as patient characteristics and primary diagnoses. A stepwise approach was taken to measure the explanatory power of the malnutrition-associated measure components (ability of the components to explain the outcome). Initially, the hospital 30-day readmissions and LOS models were estimated using only the demographic and clinical variables. Later, the models were re-estimated, including the malnutrition variables. This approach allowed the measure development team to estimate the incremental improvement in goodness-of-fit from including the malnutrition variables. Model goodness-of-fit was reported as adjusted-R2 for the hospital LOS model and the concordance statistic (c-statistic) for the 30-day readmissions model.A secondary analysis was conducted to specifically assess the association between the main clinical end point of the composite measure (nutrition care plans for patients with a diagnosis of malnutrition) and the outcomes most associated with malnutrition (30-day readmissions and LOS). The analysis intended to understand the association of having a nutrition care plan with a malnutrition diagnosis vs not having a nutrition care plan.Validity Testing for Component Measures—Critical Data ElementsValidity was also studied for the component measures by developing a generalized linear (logistic) regression model where medical diagnosis of malnutrition was the response variable and screening (completion and result) and assessment variables (completion, timing, and result) were the predictor variables. An additional test was conducted to ensure the overall linear model for predicting diagnosis was also predictive of the nutrition care plan. The hypothesis for this test is that all predictor variables would be correlated to the outcome of malnutrition diagnosis and together would be a strong predictor of the malnutrition outcome, supporting the validity of including these components in the malnutrition composite.In addition to testing the components of the measure for validity toward the outcome of the composite measure, testing was completed to assess the correlation between the components and outcome of the composite measure with the clinical outcomes of patient LOS and 30-day readmissions. This phase of testing assessed the predictive relationship between the set of measure components and LOS and readmissions, adjusting for differences in patient characteristics. A generalized linear mixed model approach was utilized to conduct the analyses.Testing Composite Measure Denominator ExclusionsThe two main exclusions for this measure are a LOS <24 hours because those patients are not in the hospital long enough to receive proper assessment, and intervention care plan for malnutrition. Patients who are transferred or discharged to hospice16Schwartz DB, Posthauer ME, Maillet JO. Advancing nutrition and dietetics practice: dealing with ethical issues of nutrition and hydration. J Acad Nutr Diet. September 25, 2020 [Epub ahead of print]. https://doi.org/10.1016/j.jand.2020.07.028.Google Scholar,17Dorner B. Friedrich E.K. Position of the Academy of Nutrition and Dietetics: Individualized nutrition approaches for older adults: long-term care, post-acute care, and other settings.J Acad Nutr Diet. 2018; 118: 724-735Abstract Full Text Full Text PDF PubMed Scopus (32) Google Scholar have significantly different requirements for nutrition support18American Society for Parenteral and Enteral Nutrition (ASPEN) definition of terms, style, and conventions used in ASPEN board of directors–approved documents.http://www.nutritioncare.org/uploadedFiles/Documents/Guidelines_and_Clinical_Resources/ASPEN%20Definition%20of%20Terms,%20Style,%20and%20Conventions%20Used%20in%20ASPEN%20Board%20of%20Directors%E2%80%93Approved%20Documents.pdfDate accessed: December 9, 2020Google Scholar,19Academy-Avalere Health-Defeat Malnutrition Today dialogue proceedings/advancing patient-centered malnutrition care transitions March 2018.https://avalere.com/wp-content/uploads/2018/07/MQC_Malnutrition-Transitions-of-Care-Dialogue-Process-v23_singlepages.pdfDate accessed: December 9, 2020Google Scholar and those treatment plans are highly dependent on patient preferences.The measure development team tested measure exclusion criteria for both influence on the measure performance score and validity statistics for the individual malnutrition eCQMs when they were first developed.8Nepple K.G. Tobert C.M. Valladares A.F. Mitchell K. Yadrick M. Enhancing identification and management of hospitalized patients who are malnourished: a pilot evaluation of electronic quality improvement measures.J Acad Nutr Diet. 2019; 119: S32-S39Abstract Full Text Full Text PDF PubMed Scopus (10) Google Scholar The measure development team tested the measure specifications with a set of hypothetical measure exclusions that were determined by consensus agreement of the Technical Expert Panel (consisting of a group of clinical and technical experts whose guidance was sought by the measure developm