Analyzing the impact of an author's publications

Abstract

Information professionals are called on to determine how best to measure the impact of an author's articles, and citation counts are often regarded as one method for obtaining a quantitative expression of the utilization and contribution of a particular published paper. As Meho states, citation analysis assumes that influential works or scientists are cited more often than others [1]. Egghe and Rousseau claim that citation counts are based on four important assumptions: an article's citation implies use of that document by the citing author; the citation reflects the merit (quality, significance, impact) of the article; the references are from the best possible works on the topic; and the cited articles are related in content to the one in which they are used [2]. Traditionally, the peer-review process has been used to assess article quality. Currently, there is a global trend toward the development, refinement, and increased use of quantitative metrics, particularly those resulting in “quantifiable, post publication quality assessment” [1, 3, 4]. However, determining impact by citation analysis can be controversial; in some cases, works are cited to point out errors and inaccuracies in the research. Additionally, long articles are often cited more frequently, and some reference lists contain erroneous citations, which can skew results. Finally, journal visibility and prestige affects dissemination, and self-citation can artificially inflate citation counts [1, 3, 5–8]. Despite these concerns, citation analysis remains a useful tool for assessing faculty research publication. The journal impact factor (JIF) was developed to facilitate comparison between citation rates of journals and evolved as a measurement of journal quality on the assumption that a higher citation rate equaled a higher quality journal [9]. This assumption causes concerns, as Amin and Mabe indicated, because it is often used as the “chief quantitative measure of the quality of a journal, its research papers, and the researchers who wrote the paper” [10]. Many authors have noted other factors that affect the actual impact factor number: (1) research field, (2) type of journal, (3) average number of authors per paper, (4) size of the journal, and (5) two-year measurement window. Other limitations are that JIFs are biased toward US publishers, a small percentage of articles is highly cited, and the JIF may be easily manipulated [1, 3, 10–12]. Also of note is the fact that a journal may not yet be indexed in Web of Science (WOS) or tracked in the Journal Citation Reports (JCR) database long enough to have an impact factor. For these reasons, many have cautioned against using JIF to judge the quality or impact of individual papers or authors [9, 13]. Vieira and Faraino, however, used JCR to analyze the research record of their institution's list of faculty publications [14]. They pointed out that JCR can be an important research tool in indicating how faculty authors were citing the literature. More recently, Saha et al. found a strong correlation between the quality ratings of surveyed physicians of nine general medicine journals and their impact factors [15], while Yue et al. found that clinical and research neurologists' ratings of journal quality also correlated with impact factors [16]. Rice et al. provided critical information about the statistical formulas used to calculate the reliability and validity of citation data [17].