Observational (Non-Randomized) Studies

Abstract

In 1965, the statistician William G. Cochran defined an observational study as an attempt to draw inferences about the effects caused by a treatment, a policy, a program, an exposure, or an intervention in a context in which randomized experimentation is not possible for practical or ethical reasons. A “treatment” or “control” condition is something that could, in principle, be imposed or withheld; however, in most observational studies, there are overwhelming ethical or practical barriers to imposing the treatment on human subjects for experimental purposes. For instance, the treatment may be harmful, such as the sudden death of a spouse and its possible effects on depression, or a potentially traumatic event and its possible effects as a cause of post-traumatic stress syndrome. Alternatively, the treatment might be a prison sentence that is imposed by a court, in a typical context in which it is neither reasonable nor realistic to alter the prison sentence for research purposes. Observational studies became distinct from experiments when Sir Ronald Fisher invented randomized experimentation. In a randomized experiment, individuals are assigned to treatment or control by a truly random mechanism, perhaps by flips of a fair coin, or more commonly today by random numbers generated by a computer. Randomized treatment assignment ensures that there is no systematic reason that certain types of people receive treatment and other types receive control; it is simply luck. The benefits of randomized treatment assignment are absent in observational studies. Treated individuals and controls may differ systematically prior to treatment, so a difference in outcomes observed after treatment may not be an effect caused by the treatment. Some pretreatment differences are visible in data; others are not. Treated individuals might typically be somewhat older than controls, so treated individuals might be matched to controls of the same age to remove or adjust for that difference in age. There are many methods of adjusting for measured covariates, and matching is the simplest of these. Treated and control individuals may also differ prior to treatment in terms of covariates that were not measured, and it is not possible to match or adjust for an unmeasured covariate. Most of the controversy that commonly attends an observational study consists of debate about possible biases from the failure to adjust for some unmeasured covariate. Most of the creative effort in designing a successful observational study focuses on addressing possible biases from unmeasured covariates.