What citation tests really tell us about bias in academic publishing

Abstract

The concept of status bias in the academic publishing process has been studied for decades, and economists have often used citation tests to address the question: if editors are biased in favour of high-status economists, their published papers will tend to be of lower quality and thus receive fewer citations. While this literature has tended to find bias against high-status economists rather than in their favour, I argue that these citation tests are quite uninformative, as their interpretation depends on the very strong assumption that authors are unbiased when choosing which papers to cite. I then use publication and citation data to estimate a model of the publication process, and find that the results depend strongly on this same (untested) assumption: the data are consistent with editor bias in favour of high-status economists if I assume that authors are also significantly biased in favour of high-status economists when choosing which papers to cite. My findings indicate important shortcomings of citation tests, as well as the need to find new approaches to study the question of status bias.