Bias—Is it a problem, and what should we do?

Abstract

Observational studies are prone to two types of errors: random and systematic. Random error arises as a result of variation between samples that might be drawn in a study and can be reduced by increasing the sample size. Systematic error arises from problems with the study design or the methods used to obtain the study data and is not influenced by sample size. Over the last 20 years, veterinary epidemiologists have made great progress in dealing more effectively with random error (particularly through the use of multilevel models) but paid relatively little attention to systematic error. Systematic errors can arise from unmeasured confounders, selection bias and information bias. Unmeasured confounders include both factors which are known to be confounders but which were not measured in a study and factors which are not known to be confounders. Confounders can bias results toward or away from the null. The impact of selection bias can also be difficult to predict and can be negligible or large. Although the direction of information bias is generally toward the null, this cannot be guaranteed and its impact might be very large. Methods of dealing with systematic errors include: qualitative assessment, quantitative bias analysis and incorporation of bias parameters into the statistical analyses.