Some Approximate Evidence Factors in Observational Studies

Abstract

An observational study or nonrandomized experiment has exact evidence factors if it permits several statistically independent tests of the same null hypothesis H0 of no treatment effect, where these several tests depend upon different assumptions about bias from nonrandom treatment assignment. In an observational study, we are typically uncertain about what assumptions truly describe treatment assignment. If independent tests each reject H0, and if each of these tests is valid under assumptions about treatment assignment that would invalidate the others, then rejection of H0 does not depend upon the truth of any one of these assumptions. Although exact evidence factors do exist, the requirement of exact independence may limit the class of test statistics in ways that reduce Pitman efficiency and design sensitivity, so one might wish to be free of these limitations. A much larger class of statistics permits approximate evidence factors which are not independent, but which preserve some of the important consequences of independence. Each of the several tests may be subjected to a sensitivity analysis, and the tests may be combined to yield a single inference. Results are proved for a large class of statistics and they are illustrated by following a suggestion of Maritz which uses the randomization distribution of tests based on statistics that are equated to zero in defining Huber’s m-estimates. A study of damage to DNA from occupational exposures to chromium is used as an example.