Reliability Estimates of Clinical Measures Between Minimum Data Set and Online Survey Certification and Reporting Data of US Nursing Homes

Abstract

Evidence-based quality improvement in nursing homes relies heavily on administrative data. Minimum Data Set (MDS) and Online Survey Certification and Reporting (OSCAR) are the 2 largest sources of national nursing home administrative data. Although clinical measures in each of the data systems have frequently and independently been used to measure nursing home performance, no study has systematically examined their relative reliability. This study estimates the reliability of clinical measures between MDS and OSCAR data and discusses the utility of these databases in research. We analyzed 4 waves of national MDS and OSCAR data (1999-2002). A comparable group of 24 clinical indicators from both datasets was selected for the reliability test. Spearman rank-order correlation analyses were used to measure the test-retest reliability relationships. Across the 4 years, the majority of coefficients of clinical measures exhibit good reliability. There are 17 clinical measures (71% of the total measures) with correlation coefficients greater than 0.4, the minimally acceptable reliability threshold according to Morris. The highest correlation coefficients were associated with tube feeding at 0.883. Our findings indicate that the target clinical measures in OSCAR and MDS are generally consistent and reliable. However, researchers should use OSCAR self-reported clinical measures to measure nursing home outcomes in place of corresponding MDS clinical measures with caution because some indicators in the administrative data are more reliable than others.