Biases in studies of spatial patterns in insect herbivory

Abstract

AbstractThe properties of the human mind are responsible for a number of biases that affect the quality of scientific research. However, scientists working in the fields of ecology and environmental science rarely take these biases into account. We conducted a meta‐analysis of data extracted from 125 publications comparing woody plant damage by defoliating insects in different environments in order to understand the extent to which our knowledge on spatial patterns in herbivory is affected by various biases. We asked which research methods are most prone to biases and whether these biases lead to overestimation of the effects under study. The effect sizes (ESs) decreased with increases in the numbers of plant species involved in the study, with 61% lower ESs for herbivory estimated on all plants growing in study plots compared to herbivory on selected species. ESs also depended on the leaf sampling procedure: when all leaves from a tree or branch were sampled for measurements of herbivory or when random or systematic selection protocols were applied, ESs were 74% smaller than in cases of more subjective haphazard selection. In addition, ESs were 97% and 135% greater when the person conducting sampling and measuring was aware of the research hypothesis or sample origin, when compared with situations when the observer was blinded to these factors. The impacts of cognitive biases on the study outcomes significantly decreased with the increase in publication year; however, this pattern emerged mostly due to high‐ranked journals and was non‐significant for other journals. Using the studies of spatial patterns in herbivory as an example, we showed that our ecological and environmental knowledge is considerably biased due to an unconscious tendency of researchers to find support for their hypotheses and expectations, which generally leads to overestimation of the effects under study. Cognitive biases can be avoided by using different methods, such as applying randomization procedures in sampling and blinding of research hypotheses and sample origins. These measures should be seen as obligatory; otherwise, accumulation of the biased results in primary studies may ultimately lead to false general conclusions in subsequent research synthesis.