Impact Factor, Impact, and Smoke and Mirrors

Abstract

I believe that the editorial by Martin Tobin, “Thirty years of impact factor and the Journal,” which appeared in the August 15, 2004 issue of the AJRCCM, merits comment (1). I was particularly struck by the figure showing a dramatic upward trajectory in the impact factor in recent years. This, to me, bore an ominous resemblance to the increase in the NASDAQ stock index in the late 1990s, before its collapse in 2000. We, as physician-scientists, are trained to notice the unusual and to investigate further. As the author noted, the impact factor for a journal in year X is a ratio of the number of citations (in any journal) in year X to the number of articles (in the journal of interest) in years X-1 and X-2. Thus, the impact factor can be influenced by either decreasing the number of articles published or increasing the number of citations. There is evidence for both of these in the case of AJRCCM (using Thorax and Chest as reference points). Any journal can directly influence the number of articles published annually. From 1999 to 2003 the number of articles in Thorax remained stable (196 to 198) as did the number of articles in Chest (701 to 704). Over the same period of time, the number of articles published in AJRCCM decreased from 600 to 380. The trajectory for publications was increasing through 2000 when AJRCCM published 757 articles, making the 2003 number about a 50% decrease. Given the explosion of research in respiratory disease in recent years, it seems ironic that publications have been cut in half. Viewed another way, ATS members are now paying twice as much per article, which, in my mind, decreases the “impact” of the Journal. The other way to influence impact factor is to increase the number of citations to published articles. Although one would think this would primarily result from the publication of additional respected manuscripts or reviews, it is also possible with a process known as “self-citing,” which is occurring in AJRCCM at a much higher rate than in other journals. For example, of the 3,948 citations to 2002 AJRCCM articles, 1,122 (30.2%) were “self-cites.” Corresponding values from Thorax and Chest were 12.1% and 11.3%, respectively. This appears to be occurring for several reasons. First, the series “Year in Review in the Blue Journal,” which exclusively cites AJRCCM articles, artificially increases the impact factor. As an example, the 6 articles in the 2004 edition of this series cited the 380 AJRCCM articles for 2003 a total of 576 times, and will influence the 2004 impact factor (2–7). Second, authors invited to resubmit articles to AJRCCM receive the following statement: “In revising your manuscript, please update your references to ensure that you discuss your work in the context of the most recent research in this area. In particular, you need to compare your references against the articles in Year in Review 2002 and 2003 accessible through the File-Cabinet icon on the Homepage of AJRCCM.” This is apparently intended to ensure increasing citation of AJRCCM work. The net result of this can be seen in Figure 1; for AJRCCM in 2003, 44% of the 2002 articles referenced were “self-citations,” compared with 19% of 2003 Thorax article references. Other strange phenomena also occur. For example, the fourth leading citer of AJRCCM articles in 2003 was the French journal Revue des Maladies Respiratoires (with the European Respiratory Journal at number three and Thorax at number five). In 2003, 32% of the Revue des Maladies Respiratoires 2002 citations were of AJRCCM articles (341/1055), with the next highest percentages being Chest and European Respiratory Journal, both at 5%. Figure 1. Distribution pattern of citations for articles published in 2003 in (A ) AJRCCM and (B ) Thorax. The solid part of the bar represents “self citations,” which comprised, in 2002, for AJRCCM 44% (1,122/2523) of the citations and for Thorax 19% (91/491) of the citations (9).