Statistical Comparison of Cardiac Output Measurement Methods: Advantages of an Error Grid Representation

Abstract

Valid data are essential for making medical decisions based on cardiac output (CO) and cardiac index (CI). CO and CI are used to monitor cardiovascular physiology and diseases such as heart failure. Invasive reference methods (RM) to estimate CO include thermodilution and various Fick measures. There is increasing interest in other devices that estimate CO less invasively and more continuously, including bioimpedance devices. However, accuracy of any CO estimation method is subject to substantial measurement error commonly reported to be ∼20%. Comparing a test method's (TM) estimated CO against that of RM is therefore challenging. Systematic bias and imprecision exist with all CO estimations; thus a sound analysis using appropriate statistical methods of comparison is essential. We describe several common approaches and propose an alternative comparison of CO estimates. Comparative Strengths & Weaknesses of Various Data Analyses: A key strength of using correlation, scatterplots, and linear regression analyses is that all are generally familiar. Intuitively, poor correlations, “random” scatterplots, and statistically insignificant slopes suggest poor TM-RM correspondence (poor accuracy). Linear regression quantifies bias (which correlation ignores) but is misleading as it assumes no RM measurement error. Deming regression (Linnet, 1990) accounts for measurement error in both RM and TM and allows one to assess systematic bias. However, neither linear nor Deming regression assess precision or repeatability. Bland-Altman plots are popular but fail to consider that RM is an established method. Error Grid Alternative: We present an alternative analysis of TM and RM correspondence using an error grid. We considered a simple ±20% error boundary. Based on considerations of the smallest clinically meaningful difference in CO and limitations of measurement accuracy, we examined an error bound of ±0.8 L/min for RM CO≤4 L/min. Clinical ramifications are most pronounced with smaller CO values, making data representation in this range essential. Percentage of data inside the grid boundary describes comparative TM accuracy vs. RM (both averaged over a prescribed number of measurements). Conclusions: Comparing two imperfect methods for CO measurement is a challenge and conventional analyses can be misleading. We propose an error grid based on clinical judgement, which can facilitate comparison of CO measurements. Valid data are essential for making medical decisions based on cardiac output (CO) and cardiac index (CI). CO and CI are used to monitor cardiovascular physiology and diseases such as heart failure. Invasive reference methods (RM) to estimate CO include thermodilution and various Fick measures. There is increasing interest in other devices that estimate CO less invasively and more continuously, including bioimpedance devices. However, accuracy of any CO estimation method is subject to substantial measurement error commonly reported to be ∼20%. Comparing a test method's (TM) estimated CO against that of RM is therefore challenging. Systematic bias and imprecision exist with all CO estimations; thus a sound analysis using appropriate statistical methods of comparison is essential. We describe several common approaches and propose an alternative comparison of CO estimates. Comparative Strengths & Weaknesses of Various Data Analyses: A key strength of using correlation, scatterplots, and linear regression analyses is that all are generally familiar. Intuitively, poor correlations, “random” scatterplots, and statistically insignificant slopes suggest poor TM-RM correspondence (poor accuracy). Linear regression quantifies bias (which correlation ignores) but is misleading as it assumes no RM measurement error. Deming regression (Linnet, 1990) accounts for measurement error in both RM and TM and allows one to assess systematic bias. However, neither linear nor Deming regression assess precision or repeatability. Bland-Altman plots are popular but fail to consider that RM is an established method. Error Grid Alternative: We present an alternative analysis of TM and RM correspondence using an error grid. We considered a simple ±20% error boundary. Based on considerations of the smallest clinically meaningful difference in CO and limitations of measurement accuracy, we examined an error bound of ±0.8 L/min for RM CO≤4 L/min. Clinical ramifications are most pronounced with smaller CO values, making data representation in this range essential. Percentage of data inside the grid boundary describes comparative TM accuracy vs. RM (both averaged over a prescribed number of measurements). Conclusions: Comparing two imperfect methods for CO measurement is a challenge and conventional analyses can be misleading. We propose an error grid based on clinical judgement, which can facilitate comparison of CO measurements.