Evaluation of impact factor using two different methods

Abstract

Sir, Impact factor is one of the most important tools in evaluating the quality of science journals. Perhaps, it is the only factor known to most researchers today and it has been used by many individuals and institutions. For instance, authors prefer to publish in high impact journals, editors make effort to increase the journal’s impact factor and academic institutions take impact factors into consideration for hiring, promotion or financial incentives. In addition, granting agencies use it to evaluate the quality of applicant’s publications, and governments rank academic institutions based on impact factors. Thomson Reuters, the owner of the Institute of Scientific Information (ISI), a company specialized in producing various research analysis tools, produces impact factors of numerous journals. ISI generates Journal Citation Reports (JCR), a database containing information about journals including the number of articles and reviews, and impact factors. Impact factor is calculated using a predefined formula. For example, impact factor for the year 2000 is calculated based on the number of citations to 1998 articles in the year 2000 (A), the number of citations to 1999 articles in the year 2000 (B), the number of articles published in the year 1998 (C ) and the number of articles published in the year 1999 (D). Impact factor for 2000 is therefore (A þ B)/(C þ D). In recent years, more tools have been developed to allow researchers to analyze citation indices for various journals. Among those tools is Scopus (2009) citation database produced by Elsevier which is a large citations and publications database. It has various analytical tools including citation tracker and information about individual articles. Using the information provided by Scopus, one can calculate impact factor of any journal including Human Reproduction. In general, Human Reproduction publishes (or has published in the past) original articles, reviews, letters, editorials, notes, conference papers and short surveys. In JCR, impact factor is calculated based on citations to research articles and reviews. Using the same type of articles, we calculated impact factors of Human Reproduction for the years 2000–2006 with Scopus database, and we compared the results with those obtained from JCR. We found discrepancies in the number of articles and review articles produced by JCR and Scopus (data not shown). The impact factors reported by JCR are also consistently lower than those using Scopus database (Fig. 1). It is unclear which articles were used by JCR to calculate the impact factor. Indeed, a few authors have suggested that these articles should be listed on JCR website (Rossner et al., 2008). Similarly, Scopus could not disclose their exact method of data collection of the number of articles for any journal (personal communication). Identification of articles used for impact factors would be useful to evaluate whether journals with high impact factor are definitely better than those with lower impact factor. High quality articles lead to many citations increasing the impact factor. However, ordinary and yet highly controversial articles might also attract a good number of authors to reply or perform a similar study and cite the paper (Rossner et al., 2008). As a result, controversial articles may increase the impact factor of a journal. The number of authors per article and the number of review articles might influence the impact factor as well. It is possible that multi-author articles receive a higher number of self-citations (Sala and Brooks, 2008). Also, review articles tend to be quoted more often than original research papers. We should consider developing new tools to assess the real impact of scientific journals and differentiate between positive and negative impacts, both of them might lead to an increase in the impact factor of a journal.