Use Of Citation Counts Research Articles

Self-referencing rates in biological disciplines.

The use of citation counts (among other bibliometrics) as a facet of academic research evaluation can influence citation behavior in scientific publications. One possible unintended consequence of this bibliometric is excessive self-referencing, where an author favors referencing their own publications over related publications from different research groups. Peer reviewers are often prompted by journals to determine whether references listed in the manuscript under review are unbiased, but there is no consensus on what is considered "excessive" self-referencing. Here, self-referencing rates are examined across multiple journals in the fields of biology, genetics, computational biology, medicine, pathology, and cell biology. Median self-referencing rates are between 8-13% across a range of journals within these disciplines. However, self-referencing rates vary as a function of total number of references, number of authors, author status/rank, author position, and total number of publications for each author. Importantly, these relationships exhibit interdisciplinary and journal-dependent differences that are not captured by examining broader fields in aggregate (e.g., Biology, Chemistry, Physics, etc.). These results provide useful statistical guidelines for authors, editors, reviewers, and journals when considering referencing practices for individual publications, and highlight the effects of additional factors influencing self-referencing rates.

Invention disclosures and the slowdown of scientific knowledge

Abstract Invention disclosures are one measure of new scientific knowledge that represents and predicts the future scientific research output of a US federal laboratory. In this article, we document a negative shift in the production function for new scientific knowledge as measured by invention disclosures at one federal laboratory, the National Institute of Standards and Technology, over the first 16 years of the new millennium. We find a negative shift of the production function for new scientific knowledge, and that shift might reflect the coincidence of the information and communication technology revolution that enabled fast science, and the evaluation of research with uncritical use of citation counts that created incentives to focus on incremental research in crowded research topics.

Assessing the publication impact using citation data from both Scopus and WoS databases: an approach validated in 15 research fields

In a recent paper ( https://doi.org/10.1007/s11192-020-03386-9 ) we proposed a model to estimate the citations of an article in a database (Scopus/Web of Science) in which it is not indexed using the percentile rank of the database (Web of Science/Scopus) in which it is indexed. In this study we supplement the previous work with three advances: (1) by using 15 different research fields, corresponding to over 1 million papers, since we previously used only four fields; (2) by measuring the agreement between the percentile ranks in both databases using Lin’s concordance correlation coefficient, since this coefficient has not been used previously to measure this agreement, but as a test with a sample of 15,400 papers to compare the actual and estimated number of citations; and (3) by using a robust data cleaning procedure. The results revealed a substantial concordance between percentile ranks of papers indexed in these two databases in all the research fields studied, and that this concordance is even stronger for high percentile values. This level of concordance suggests that we can consider the percentile of a paper in a database in which it is not indexed as being equal to the percentile of this paper in a database in which it is indexed. In other words, we increased the reliability of our previous conclusions that the percentile rank can be used as a citation database-normalization. The results of this study contribute to improve the use of citation counts in bibliometric studies, and to calculate research indicators when we need to use both bibliographic databases.

Using citations correctly in wildlife literature

Using citations correctly in wildlife literature

Innovation or imitation: The diffusion of citations

Citations in scientific literature are important both for tracking the historical development of scientific ideas and for forecasting research trends. However, the diffusion mechanisms underlying the citation process remain poorly understood, despite the frequent and longstanding use of citation counts for assessment purposes within the scientific community. Here, we extend the study of citation dynamics to a more general diffusion process to understand how citation growth associates with different diffusion patterns. Using a classic diffusion model, we quantify and illustrate specific diffusion mechanisms which have been proven to exert a significant impact on the growth and decay of citation counts. Experiments reveal a positive relation between the “low p and low q” pattern and high scientific impact. A sharp citation peak produced by rapid change of citation counts, however, has a negative effect on future impact. In addition, we have suggested a simple indicator, saturation level, to roughly estimate an individual article's current stage in the life cycle and its potential to attract future attention. The proposed approach can also be extended to higher levels of aggregation (e.g., individual scientists, journals, institutions), providing further insights into the practice of scientific evaluation.

Citation Metrics and Impact Factors Fail as Measures of Scientific Quality, in Particular in Taxonomy, and are Biased by Biological Discipline and by Geographic and Taxonomic Factors

Based on my experience and analyses of citation frequencies of papers published in various biological journals, I discuss the use of citation counts and journal Impact Factors as measures of scientific quality. I conclude that citation frequency differs not only among biological sub-disciplines such as ecology and taxonomy, but also among taxonomic papers on highly similar topics, and that these latter differences largely depend on the organismal group (cryptogams vs. vascular plants and species-rich vs small genera and families) and the geographic region concerned. As a result, journals with a broad biological scope face difficulties in competing in terms of Impact Factors, and thus in competing for the best manuscripts, with journals specialized on particular sub-disciplines that attract many citations. However, since papers on “hot” topics also tend to be short-lived, these differences, at least in part, would be much reduced if citations were counted for longer periods of time or were compensated based on the average age of cited papers in the particular field. In particular in taxonomy, factors such as originality and innovativeness, generally regarded as measures of scientific quality, appear to be inversely correlated with citation frequency. I, therefore, conclude that scientific editors would be able to significantly increase the Impact Factors of taxonomic journals, not by choosing for publication the most scientifically valuable and supreme articles, but by picking manuscripts based the taxonomic position and geographic distribution of the organisms concerned. Still, the author sincerely hopes that no editor will ever take such measures!

Drivers of citations: An analysis of publications in “top” accounting journals

Abstract Citation counts are increasingly used to assess the impact and quality of articles, journals, researchers, and research institutions. At the same time, there is an ongoing debate about their usefulness as a performance measure. To understand the actual drivers of this measure in the accounting literature, we empirically investigated which factors affect the citation rates of publications in five “top“ journals of the field using a three-step regression to accommodate different theoretical perspectives on citations. Our results show that citation behavior is influenced by both article and author characteristics, suggesting that not only the universalistic but also the particularistic perspective on science is associated with citation behavior in the accounting field. Contrary to previous studies, we identify several particularistic variables as significant, relating to the visibility and position of researchers in the scientific system and to how they promote their research. In addition, the strong effects of the universalistic variables of subject area and research method point to the possibility that the use of citation counts for evaluating performance might create a bias in favor of certain types of research. Overall, our results should urge caution against the uncritical use of citation counts as an indicator of quality and a performance measure.

Quantifying the evolution of citation cascades

ABSTRACTCitation is an important measurement in science which provides valuable clues to the historical development and trend forecast of science. However, the evolution of citation structure remains poorly understood despite long‐period and frequent use of citation counts for assessment purposes within scientific community. We observe citation structure from a cascade perspective and construct a large and long‐range citation graph based on real data. Two metrics are used to quantify the structural virality and cascade size. Preliminary results show that (1) age plays an important role for the evolution of citation cascades; (2) average depth tends to be influenced by both lifespan and the whole volume of scientific literature and grows much slower than cascade size; (3) there exists a stable positive relationship between average depth and cascade size, and it seems that an ultimate status exists where most papers eventually have large values for both metrics.

A Reverse Engineering Approach to the Suppression of Citation Biases Reveals Universal Properties of Citation Distributions

The large amount of information contained in bibliographic databases has recently boosted the use of citations, and other indicators based on citation numbers, as tools for the quantitative assessment of scientific research. Citations counts are often interpreted as proxies for the scientific influence of papers, journals, scholars, and institutions. However, a rigorous and scientifically grounded methodology for a correct use of citation counts is still missing. In particular, cross-disciplinary comparisons in terms of raw citation counts systematically favors scientific disciplines with higher citation and publication rates. Here we perform an exhaustive study of the citation patterns of millions of papers, and derive a simple transformation of citation counts able to suppress the disproportionate citation counts among scientific domains. We find that the transformation is well described by a power-law function, and that the parameter values of the transformation are typical features of each scientific discipline. Universal properties of citation patterns descend therefore from the fact that citation distributions for papers in a specific field are all part of the same family of univariate distributions.

Evaluations of individual scientists and research institutions

Evaluations of individual scientists and research institutions

Earlier Web usage statistics as predictors of later citation impact

The use of citation counts to assess the impact of research articles is well established. However, the citation impact of an article can only be measured several years after it has been published. ...

Earlier Web usage statistics as predictors of later citation impact

AbstractThe use of citation counts to assess the impact of research articles is well established. However, the citation impact of an article can only be measured several years after it has been published. As research articles are increasingly accessed through the Web, the number of times an article is downloaded can be instantly recorded and counted. One would expect the number of times an article is read to be related both to the number of times it is cited and to how old the article is. The authors analyze how short‐term Web usage impact predicts medium‐term citation impact. The physics e‐print archive—arXiv.org—is used to test this.

The most-cited statistical papers

We attempt to identify the 25 most-cited statistical papers, providing some brief commentary on each paper on our list. This list consists, to a great extent, of papers that are on non-parametric methods, have applications in the life sciences, or deal with the multiple comparisons problem. We also list the most-cited papers published in 1993 or later. In contrast to the overall most-cited papers, these are predominately papers on Bayesian methods and wavelets. We briefly discuss some of the issues involved in the use of citation counts.

Citations and Links as a Measure of Effectiveness of Online LIS Journals

Examines the use of citation counts and web links to evaluate online LIS journals. The Journal Impact Factor, a traditional metric for comparing journals, is based on the citation counts of a journal over a specified period of time, compared with the citable articles published in the journal. The Web Impact Factor (WIF) has been proposed as an equivalent metric for websites. The WIF is broadly defined as the ratio of links made to a site, compared to the number of information units (e.g. pages) at the site. The study compares the WIFs of a sample of online LIS journal websites with their conventional citation counts. Links to online journals are often considered to be equivalent to citations, but there are significant differences. Provides indications of how effective the online medium is for communicating LIS research and offers guidance for LIS authors and journal editors to help them make effective use of online journals.

Viewpoint: Selecting the 5 Most Important Papers in the First 50 Years of the Journal of Range Management

A graduate seminar to select the 5 most important papers pub lished in the first 50 years of the Journal of Range Management (J R M), 1948‐1997, cultivated an appreciation for the development of the discipline of rangeland science and management, and provided some historical perspective to judge the JRM. A review of textbooks, and papers describing early milestones and the use of citation counting were helpful in developing criteria to discriminate the importance of papers. The greatest disagreement among the 9 participants focused on the use of citation counts as a criterion: 2 students used only counts and 3 students refused to use counts. Eighteen papers received at least 1 vote as a top 5 paper, and 2 plant succession-vegetation monitoring papers were clearly the most popular. The exercise revealed that discontent with the JRM is not new. Although the JRM now covers a wider variety of topics, including both reductionist and synthetic works, some students felt that it was less encompassing of multi ple values of rangelands and the breadth of rangeland science than recent texts. The students found that the selection of impor tant papers expanded their understanding of the discipline and their resolve to publish in the JRM. Ideally, others will be chal lenged to perform this review for the benefit of students, the dis cipline, and the JRM.

Viewpoint: Selecting the 5 most important papers in the first 50 years of the Journal of Range Management

A graduate seminar to select the 5 most important papers published in the first 50 years of the Journal of Range Management (JRM), 1948–1997, cultivated an appreciation for the development of the discipline of rangeland science and management, and provided some historical perspective to judge the JRM. A review of textbooks, and papers describing early milestones and the use of citation counting were helpful in developing criteria to discriminate the importance of papers. The greatest disagreement among the 9 participants focused on the use of citation counts as a criterion: 2 students used only counts and 3 students refused to use counts. Eighteen papers received at least 1 vote as a top 5 paper, and 2 plant succession-vegetation monitoring papers were clearly the most popular. The exercise revealed that discontent with the JRM is not new. Although the JRM now covers a wider variety of topics, including both reductionist and synthetic works, some students felt that it was less encompassing of multiple values of rangelands and the breadth of rangeland science than recent texts. The students found that the selection of important papers expanded their understanding of the discipline and their resolve to publish in the JRM. Ideally, others will be challenged to perform this review for the benefit of students, the discipline, and the JRM.

Using citation counts as a measure of quality in science measuring what's measurable rather than what's valid

Empirical work in the social studies of science has progressed rapidly with the availability and development of the citation indexes. Citation counts have become a widely accepted measure of the quality of a scientific contribution. However, there are several problems involved in the use of citation counts as a measure of quality in science. First, citation counts are sensitive to popular trends in science. In this sense, they approximate a Nielsen rating for science. Second, the distribution of citations restricts their utility to separating the extremes. Third, citation counts are not sensitive to the ethical and moral dimensions of the quality of a scientific contribution. Fourth, citation counts underestimate the contribution of applied scientists. This paper examines these limitations.

Assessing Impact (Quality?) in Psychology: The Use of Citation Counts

Assessing Impact (Quality?) in Psychology: The Use of Citation Counts