The Impact of Ophthalmology

Abstract

Each summer, publishers, associations, and libraries eagerly await the annual publication of the ISI Journal Citation Reports (JCR), a report card for the journals included in the Thompson Reuters ISI database. Among other data, the JCR lists the impact factor of over 7500 journals.1Thompson Reuters ISIJournal Citation Report, Product Description Page. Thompson Scientific, Philadelphia, PA2006Google Scholar For the purposes of the JCR, Thompson Reuters defines the journal impact factor (JIF) as “the average number of times articles from the journal published in the past 2 years have been cited in the current JCR year.”2Thompson Reuters ISIGlossary of Thompson Scientific terminology. Thompson Scientific, Philadelphia, PA2006Google Scholar (JIF = A/B when A = the number of times articles published in years 1 and 2 were cited in journals indexed by ISI during year 3; and B = the number of “citable items” [articles and reviews, but not correspondence] published in years 1 and 2.) An impact factor of 1.0 means that, on average, articles published in a journal 1 or 2 years ago have been cited 1 time. The latest JIF (published in July 2008) for Ophthalmology is 4.621, putting its impact factor ranking at number 2 in the discipline, but first among those journals that published over 100 articles during 2007. That is, the 641 articles published in Ophthalmology during 2005 and 2006 garnered 2962 citations from articles during 2007 (2962/641 = 4.621). In all, articles in Ophthalmology have been cited 20,280 times over the time period covered by the JCR published in July of 2008. The journal with the highest impact factor in the discipline of ophthalmology is Progress in Retina and Eye Research with a JIF of 7.725 (51 articles published in 2005 and 2006 collected 394 cites during 2007). The 2008 Ophthalmology IF, just released in June 2009, has increased a good deal more, to 5.296. The JIF is one of the most frequently used of the various bibliometric measures to inform decisions regarding journals, but, like any report card, the JIF does not tell the whole story. Many periodicals in the discipline are not tracked by the JCR. Ulrich's Periodical Directory (www.ulrichsweb.com, accessed July 12, 2008) lists over 250 active English language journals in the field of ophthalmology and visual science. These are as diverse as journals reporting the latest in laboratory science to the business and news formats. Of these 250 periodicals, the 2007 JCR includes 45 ophthalmology and vision science journals (approximately 18%). The Impact Factor does not tell how well circulated a journal may be, a particularly important measure for advertisers. It cannot measure the quality or impact of an individual article in a journal. It cannot measure clinical versus scientific impact of a journal. Clinical impact is particularly difficult to measure since an article may be used to guide decisions in the clinic even when it is not cited in the scholarly literature. It is generally unwise to expect any meaningful results by comparing the impact factors from journals in separate and distinct disciplines.3Coelho P.M. Antunes C.M. Costa H.M. et al.The use and misuse of the “impact factor” as a parameter for evaluation of scientific publication quality: a proposal to rationalize its application.Braz J Med Biol Res. 2003; 36: 1605-1612Crossref PubMed Scopus (35) Google Scholar, 4Sims J.L. McGhee C.N. Citation analysis and journal impact factors in ophthalmology and vision science journals.Clin Experiment Ophthalmol. 2003; 31: 14-22Crossref PubMed Scopus (67) Google Scholar These differences do not reflect the relative quality of research of disciplines, but rather differences in disciplinary citation practices. In the 2007 JCR, the discipline of ophthalmology had an aggregate impact factor of 2.197 and a median impact factor of 1.443. This aggregate impact factor (AIF) means that, on average, articles in the ophthalmic literature published 1 or 2 years ago have been cited slightly more than twice. Contrast this with an AIF of 5.6 for cell biology or 0.6 for mathematics. Even within a single field, there are characteristics that will influence the impact factor of a particular journal or discipline. Laboratory-based scientific journals tend to have higher JIFs than the clinical journals.5Hansson S. Impact factor as a misleading tool in evaluation of medical journals.Lancet. 1995; 346: 906Abstract PubMed Google Scholar Reviews are, typically, cited more frequently than original research, usually being cited in the introduction of both basic research and clinical articles.6Walter G. Bloch S. Hunt G. Fisher K. Counting on citations: a flawed way to measure quality.Med J Aust. 2003; 178: 280-281PubMed Google Scholar, 7Weale A.R. Bailey M. Lear P.A. The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor.BMC Med Res Methodol. 2004; 4: 14Crossref PubMed Scopus (125) Google Scholar It has been noted8Murali N.S. Murali H.R. Auethavekiat P. et al.Impact of FUTON and NAA bias on visibility of research.Mayo Clin Proc. 2004; 79: 1001-1006Abstract Full Text Full Text PDF PubMed Scopus (47) Google Scholar, 9Scully C. Lodge H. Impact factors and their significance; overrated or misused?.Br Dent J. 2005; 198: 391-393Crossref PubMed Scopus (38) Google Scholar that greater availability influences how often a resource is cited. Electronic access, free full-text availability and open access increases availability.8Murali N.S. Murali H.R. Auethavekiat P. et al.Impact of FUTON and NAA bias on visibility of research.Mayo Clin Proc. 2004; 79: 1001-1006Abstract Full Text Full Text PDF PubMed Scopus (47) Google Scholar Interestingly, of the top 10 JIF journals in ophthalmology, 2 are available solely in electronic format. An article in this issue by Lansingh and Carter10Lansingh V.C. Carter M.G. Does open access in Ophthalmology affect how articles are subsequently cited in research?.Ophthalmology. 2009; 116: 1425-1431Abstract Full Text Full Text PDF PubMed Scopus (33) Google Scholar discusses the potential impact that open access (OA) may have on citation patterns in ophthalmology. They found that although OA articles appear to garner more citations than those published in closed access journals, other factors make this difference insignificant. The size of a journal is also an important factor, a journal publishing 10 articles per year can have a high JIF even if only 3 or 4 papers are frequently cited. The converse is true for journals publishing a large number of articles. In fact, the JIF was created in part as an attempt to “normalize” for the number of articles published per year, as well as for the age of an article. Older articles, by nature of having been around longer, will collect more citations over time. Journals publishing more articles will have more articles available to be cited and will also collect more citations, thus a mere count of citations per article is not a meaningful comparison. With this in mind, it is sometimes useful to also look at the other measures published alongside the JIF in the JCR. Thomson Reuters ISI has always acknowledged the limits of citation analysis in general and the impact factor in particular and warns against inappropriate use. The JIF is a measure of journal quality and cannot meaningfully be used to evaluate the quality of a single article or an individual, nor that of an institution. One cannot extrapolate the impact of an article from the impact factor of the journal in which it is published. An article's citation count, and thus its impact, is not the same as the JIF. Simply because they both use the word “impact” does not make them the identical nor even comparable. The JIF is not a reasonable substitute for reading and evaluating the quality of individual articles. It cannot be stressed enough that the JIF does not measure the value of a single article or the impact of an author. Scopus, Google, and other search engines and databases are becoming more sophisticated and may in time be able to publish metrics that allow some evaluation at the article level. Some databases have begun making available on-the-fly analysis methods and can report metrics such as number of citations, average citations per item, and average citations per year for search results. These are much more meaningful numbers on an article level than the JIF but are not “normalized” for the age of the literature, nor do they account for such confounding factors such as negative citations (e.g., those citing an article in order to dispute it). Judging the quality and impact of the scientific literature is a difficult task. The JIF is undoubtedly useful when used for the purpose for which it was intended: evaluation at the level of the journal. In summary, the JIF may be helpful for potential authors to choose which journal to submit a manuscript to because it gives a general idea about the quality of a journal. The JIF is not a measure of the quality of a single article or individual. Journals with a greater print and/or electronic availability rank higher. The JIF is influenced both by topic, as well as by articles, published in the past 2 years, rather than those appearing in the most recent year. Journals publishing laboratory-based science vs. clinical papers rank higher as are those publishing reviews compared with original articles. ErratumOphthalmologyVol. 116Issue 9PreviewWith apologies from the authors, in the editorial entitled “The Impact of Ophthalmology” (Ophthalmology 2009;116:1423–4), Thompson Reuters should have appeared as Thomson Reuters. Full-Text PDF