The Quality Improvement Ratio: A Method to Identify Changes in Screening Rates

Abstract

Because many managed care organizations are using HEDIS (Health Plan Employer Data and Information Set) and other measurement sets for their report cards, these measurements are also used in improvement projects to show changes over time. One analytic strategy is to compare measurements between one period and another after an intervention has been made. Such an analysis, which is termed here the "conventional" or "two-group analysis" and which would use either a Chi-square test or Fisher's exact test, entails three problems. The conventional analysis is not statistically valid because there are members in common in the two groups; population changes can affect the conventional analysis and confound the identification of any change with time; and members who do not change their behavior obscure the changes that are occurring in the conventional analysis. The Quality Improvement Ratio (QuIR) does not suffer from those limitations; it is calculated only from members who change status. For example, in evaluating the effectiveness of an intervention to improve preventive health screening, the QuIR is calculated only from members becoming screened in the second period after being unscreened in the first period, or vice versa. The QuIR is the ratio of the number of members becoming screened in the second period divided by the number becoming unscreened. Methods to test the statistical significance of the QuIR are based on the standard test for paired binary data, McNemar's test. Whereas the conventional approach addresses whether there are differences in rates between periods, and analysis using the QuIR focuses directly on whether the rates are different because of an intervention.