Domain-Level Covariance Analysis for Multilevel Survey Data With Structured Nonresponse

Abstract

Health care quality surveys in the United States are administered to individual respondents (i.e., hospital patients, health plan members) to evaluate the performance of health care organizations (i.e., hospitals, health plans), which thus constitute estimation domains. For better understanding and more parsimonious reporting of dimensions of quality, we analyze relationships among quality measures at the domain level. Rather than specifying a full parametric model for the observed responses and the nonresponse patterns at the lower (patient) level, we first fit generalized variance–covariance functions that take into account nonresponse patterns in the survey responses, then specify a likelihood function for the domain mean responses using these generalized variance–covariance functions. This allows us to model directly the relationships among domain means for different items. Because the response scales are bounded, we assume that these means follow a truncated multivariate normal distribution. We calculate maximum likelihood (ML) estimates using the EM algorithm and sample under Bayesian models using Markov chain Monte Carlo. Finally, we perform factor analysis on the estimated or sampled between-domain covariance matrixes. Using posterior draws, we assess posterior distributions of the number of selected factors and the assignment of items to groups under conventional rules. We compare ML estimates of this factor structure with those from several Bayesian models with different prior distributions for the between-domain covariance. We present analyses of data from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey of Medicare Advantage health plans.