Impact of Multiple Matched Controls on Design Sensitivity in Observational Studies

Abstract

In an observational study, one treated subject may be matched for observed covariates to either one or several untreated controls. The common motivation for using several controls rather than one is to increase the power of a test of no effect under the doubtful assumption that matching for observed covariates suffices to remove bias from nonrandom treatment assignment. Does the choice between one or several matched controls affect the sensitivity of conclusions to violations of this doubtful assumption? With continuous responses, it is known that reducing the heterogeneity of matched pair differences reduces sensitivity to unmeasured biases, but increasing the sample size has a highly circumscribed effect on sensitivity to bias. Is the use of several controls rather than one analogous to a reduction in heterogeneity or to an increase in sample size? The issue is examined for Huber's m-statistics, including the t-test, the examination having three components: an example, asymptotic calculations using design sensitivity, and a simulation. Use of multiple controls with continuous responses yields a nontrivial reduction in sensitivity to unmeasured biases. An example looks at lead and cadmium in the blood of smokers from the 2008 National Health and Nutrition Examination Survey. A by-product of the discussion is a new result giving the design sensitivity for the permutation distribution of m-statistics.