Inflated Impact Factors? The True Impact of Evolutionary Papers in Non-Evolutionary Journals

Erik Postma,Tom Tregenza

doi:10.1371/journal.pone.0000999

Abstract

Amongst the numerous problems associated with the use of impact factors as a measure of quality are the systematic differences in impact factors that exist among scientific fields. While in theory this can be circumvented by limiting comparisons to journals within the same field, for a diverse and multidisciplinary field like evolutionary biology, in which the majority of papers are published in journals that publish both evolutionary and non-evolutionary papers, this is impossible. However, a journal's overall impact factor may well be a poor predictor for the impact of its evolutionary papers. The extremely high impact factors of some multidisciplinary journals, for example, are by many believed to be driven mostly by publications from other fields. Despite plenty of speculation, however, we know as yet very little about the true impact of evolutionary papers in journals not specifically classified as evolutionary. Here I present, for a wide range of journals, an analysis of the number of evolutionary papers they publish and their average impact. I show that there are large differences in impact among evolutionary and non-evolutionary papers within journals; while the impact of evolutionary papers published in multidisciplinary journals is substantially overestimated by their overall impact factor, the impact of evolutionary papers in many of the more specialized, non-evolutionary journals is significantly underestimated. This suggests that, for evolutionary biologists, publishing in high-impact multidisciplinary journals should not receive as much weight as it does now, while evolutionary papers in more narrowly defined journals are currently undervalued. Importantly, however, their ranking remains largely unaffected. While journal impact factors may thus indeed provide a meaningful qualitative measure of impact, a fair quantitative comparison requires a more sophisticated journal classification system, together with multiple field-specific impact statistics per journal.

Highlights

Despite the fact that most scientists, funding organisations, promotion committees and journal editors are very much aware of the numerous problems associated with the use of journal impact factors as a measure of scientific quality or even impact [1,2,3], impact factors continue to be amongst the most commonly used measures of journal quality, and thereby of individual papers and their authors
Of the about 5000 articles published between 1996 and 2006 with the term ‘‘sexual selection’’ in either the title, keywords or abstract, only one-fifth was published in journals that are classified by ISI Scientific under ‘Evolutionary Biology’
The relative impact of ‘evolutionary’ articles published in EVOLUTION and JOURNAL OF EVOLUTIONARY BIOLOGY is significantly greater than zero. This implies that the ten keywords used to classify an article as ‘evolutionary’ may not be representative of the whole field of evolutionary biology

Summary

Introduction

Despite the fact that most scientists, funding organisations, promotion committees and journal editors are very much aware of the numerous problems associated with the use of journal impact factors as a measure of scientific quality or even impact [1,2,3], impact factors continue to be amongst the most commonly used measures of journal quality, and thereby of individual papers and their authors. One of the main problems associated with the use of impact factors as an objective measure of either quality or impact are the large and systematic differences that exist among different scientific disciplines, with impact factors of evolutionary and ecological journals being on the low end of the spectrum [4,5]. It is these systematic differences among scientific disciplines, which are unrelated to the quality or the size of the field, that make it impossible to directly compare impact factors of journals from different fields [4,5,6]. Of the about 5000 articles published between 1996 and 2006 with the term ‘‘sexual selection’’ in either the title, keywords or abstract, only one-fifth was published in journals that are classified by ISI Scientific under ‘Evolutionary Biology’

Methods

Results

Conclusion