Matched control groups for modeling events in citation data: An illustration of nobel prize effects in citation networks

Abstract

Bibliometric data are frequently used to study the effects of events, such as the honoring of a scholar with an award, and to investigate changes of citation impact over time. However, the number of yearly citations depends upon time for multiple reasons: a) general time trends in citation data, b) changing coverage of databases, c) individual citation life‐cycles, and d) selection on citation impact. Hence, it is often ill‐advised to simply compare the average number of citations before and after an event to estimate its causal effect. Using a recent publication in this journal on the potential citation chain reaction of a Nobel Prize, we demonstrate that a simple pre‐post comparison can lead to biased and misleading results. We propose using matched control groups to improve causal inference and illustrate that the inclusion of a tailor‐made synthetic control group in the statistical analysis helps to avoid methodological artifacts. Our results suggest that there is neither a Nobel Prize effect as regards citation impact of the Nobel laureate under investigation nor a related chain reaction in the citation network, as suggested in the original study. Finally, we explain that these methodological recommendations extend far beyond the study of Nobel Prize effects in citation data.