A bibliometric analysis of six economics research groups: A comparison with peer review

Abstract

The research performance of research units in economics has been evaluated by simultaneous efforts of peers and bibliometricians, with extensive interactive comparison of results afterwards. We studied trends in productivity and impact of six economics research groups in the period 1980–1988. These groups participate in a large (above one million pounds) research programme of a national Research Council. Research performance of the groups was compared to the world average by means of the Journal Citation Score method. In order to investigate the influence of one key scientists (the “star effect‘) we applied a sensitivity analysis to the performance of the research groups by elimination of the papers (and subsequent citations) of one key member. Furthermore, to provide insight into the fields to which a group directs its work, and the fields in which a group has its most important contributions, comparisons were made of publishing and citing journal packets. Similarly, citations to the work of the research groups were analysed for country of origin. We compared the results of the bibliometric part of this study with those of a simultaneous peer review study. The bibliometric study yielded clear and meaningful results, notwithstanding the increasingly applied nature of the research groups. Results from peer review and bibliometric studies appear to be complementary and mutually supportive. The participants of the bibliometrics-peer review “confrontation” meeting regarded the exercise as most valuable, with lessons for the Research Council both for the future of research programmes and the form of evaluation used for large awards.