Random Fluctuations and Validity in Measuring Disease Management Effectiveness for Small Populations

Abstract

One objective of a disease management (DM) program is the reduction of members' claims costs. A considerable amount of effort has been dedicated to standardizing the outcomes of DM measurement. An area that has not received as much attention is that of random fluctuations in measured outcomes and the related issue of the validity of outcomes subject to random fluctuation. From year to year, large random fluctuations in claims costs can increase or reduce actual savings from a DM program. Sponsors of DM programs want to know how large a group or sample is necessary to prevent the effect of random fluctuations from overwhelming the effect of claims reductions. In this paper, we measure the fluctuations in calculated DM savings in a large commercial population using an adjusted historical control methodology--the methodology that has become the industry standard and which is codified by DMAA's Guidelines. We then determine the sample size necessary to demonstrate DM program savings at different levels of confidence and model the effect on fluctuations in observed outcomes under different methods of choosing trend, different levels of truncation, and for different estimates of program savings. Some groups, particularly employers, will be smaller than the minimum size required for credible outcomes measurement. For groups smaller than this minimum size, we suggest a utilization-based outcomes measure that can be used as a proxy. For both claims- and utilization-based calculations, we provide confidence intervals to be placed around savings estimates. We do this for group sizes ranging from 1000 to 100,000 members.