Principles of Research: Limitations of Non-randomized Studies

Abstract

This contribution is a sequel to a previous article ( Surgery 2000; 18(4): 100-4), in which I described the aim of evaluations of clinical effectiveness, the available study designs, threats to the validity of evaluations from selection and information bias and, in some circumstances, the difficulty of recruiting to randomized controlled trials (RCTs) involving one or more surgical treatments. This contribution should be read in conjunction with (and not be considered a replacement for) the previous article. I will describe some limitations of controlled non-randomized trials or cohort studies (NRS) to compare the effectiveness of two or more treatments. These limitations should be considered carefully when: • choosing a non-randomized study design for an evaluation • critically appraising a study to decide whether or not to act on the evidence it provides. I will not consider case-control, cross-sectional or uncontrolled studies (i.e. case series), or the use of non-randomized cohort studies for other purposes (e.g. for studying aetiology), prognosis or evaluating diagnostic accuracy.