Confirmation bias in studies of fluctuating asymmetry

Abstract

Fluctuating asymmetry (FA) represents small, non-directional deviations from perfect symmetry in morphological characters. FA is generally accepted to increase in response to stress; therefore, FA is frequently used in ecological studies as an index of stress experienced by an organism, in particular due to environmental pollution. We experimentally tested the hypothesis that the outcomes of studies based on FA measurements may have been influenced by confirmation bias, i.e. the tendency of humans to seek out evidence in a manner that confirms their hypotheses and beliefs. We collected 100 leaves of downy birch (Betula pubescens) from a single tree, grouped them haphazardly into ten samples, scanned every sample, and then asked each of 31 scientists (experienced in studying FA) to measure FA from the scanned images of all 100 leaves. Three groups of participants were provided with false information about the origin of each sample and about the hypothesis to be tested, and one group (control) was provided with true information. The participants who believed that the leaves originated from a heavily polluted site reported significantly higher values of FA when compared to the participants who believed that the leaves were collected from an unpolluted site. When the participants were told that half the samples originated from a polluted site and half from a clean site and were asked to attribute each sample to either of these sites based on leaf FA, the differences in FA between samples classified as ‘polluted’ and ‘unpolluted’ were much higher than the differences obtained from measurements of the same sets of samples made by the control group. We conclude that when scientists expected to find high FA in some samples, the results of their measurements confirmed their expectations. This effect, classified as confirmation bias, may considerably influence the outcomes of the research on FA. This confirmation bias can be avoided by using a blind method, where the person conducting measurements is not aware of the origin of samples being measured. We argue that the use of blind methods is critically important for any study addressing environmental or genetic impacts on FA.