Principles of research: limitations of non-randomized studies

Abstract

This contribution is a sequel to an article in which I described the aim of evaluations of clinical effectiveness, the available study designs, threats to the validity of evaluations from selection and information bias and, in some circumstances, the difficulty of recruiting to randomized controlled trials involving one or more surgical treatments. This contribution should be read with (and not be considered a replacement for) the previous article. I will describe some limitations of controlled non-randomized trials or cohort studies to compare the effectiveness of two or more treatments. These limitations should be considered carefully when choosing a non-randomized study design for an evaluation or critically appraising a study to decide whether or not to act on the evidence it provides. I will not consider case-control, cross-sectional or uncontrolled studies (i.e. case series) or non-randomized cohort studies for other purposes (e.g. for studying aetiology), prognosis or evaluating diagnostic accuracy.