Mean Citation Rate Research Articles

Exploring the relation between referencing practices and citation impact: A large‐scale study based on Web of Science data

In this large‐scale contribution, we deal with the relationship between properties of cited references of Web of Science articles and the field normalized citation rate of these articles. Using nearly 1 million articles, and three classification systems with different levels of granularity, we study the effects of number of cited references, share of references covered by Web of Science, mean age of references and mean citation rate of references on field normalized citation rate. To expose the relationship between the predictor variables and the response variable, we use quantile regression. We found that a higher number of references, a higher share of references to publications within Web of Science and references to more recent publications correlate with citation impact. A correlation was observed even when normalization was done with a finely grained classification system. The predictor variables affected citation impact to a larger extent at higher quantile levels. Regarding the relative importance of the predictor variables, citation impact of the cited references was in general the least important variable. Number of cited references carried most of the importance for both low and medium quantile levels, but this importance was lessened at the highest considered level.

Citations, mandates, and money: Author motivations to publish in chemistry hybrid open access journals

Hybrid open access refers to articles freely accessible via the Internet but which originate from an academic journal that provides most of its content via subscription. The effect of hybrid open access on citation counts and author behavior in the field of chemistry is something that has not been widely studied. We compared 814 open access articles and 27,621 subscription access articles published from 2006 through 2011 in American Chemical Society journals. As expected, the 2 comparison groups are not equal in all respects. Cumulative citation data were analyzed from years 2–5 following an article's publication date. A citation advantage for open access articles was correlated with the journal impact factor (IF) in low and medium IF journals, but not in high IF journals. Open access articles have a 24% higher mean citation rate than their subscription counterparts in low IF journals (confidence limits 8–42%, p = .0022) and similarly, a 26% higher mean citation rate in medium IF journals (confidence limits 14–40%, p < .001). Open access articles in high IF journals had no significant difference compared to subscription access articles (13% lower mean citation rate, confidence limits −27–3%, p = .10). These results are correlative, not causative, and may not be completely due to an open access effect. Authors of the open access articles were also surveyed to determine why they chose a hybrid open access option, paid the required article processing charge, and whether they believed it was money well spent. Authors primarily chose open access because of funding mandates; however, most considered the money well spent because open access increases information access to the scientific community and the general public, and potentially increases citations to their scholarship.

The analysis of temporal check-all-that-apply (TCATA) data

Temporal check-all-that-apply (TCATA) extends classical check-all-that-apply (CATA) by adding a temporal dimension to the evaluation. From a data analysis point of view, TCATA data are similar to Temporal Dominance of Sensations (TDS) data but differ in that more than one attribute can be selected at any time point. Procedures for analyzing TCATA data can hence be generalized from methods for CATA as well as for TDS.TCATA data can be organized in a matrix format with 0s and 1s similar to what has been described before for TDS, but with the relaxation that the column sums can exceed 1. Consequently, the same randomization tests as suggested for TDS are suitable for TCATA data. For TDS data, ad-hoc chance and significance limits are frequently used, a practice that we review critically. We also show that these do not generalize easily to TCATA data. Instead, we suggest comparing individual products to aggregate data across all other products in the test, with that aggregation being assumed to provide an estimate of what is to be expected in the respective product category. This approach does not rely on the number of attributes considered and is also recommended for TDS data. We further suggest a new way to visualize the respective results.The results are compared to the naïve approach of applying standard CATA analyses to the data by time point. Results are reasonably close to warrant very similar interpretation, such that applying CATA analyses by time point to TCATA data seems a potential ad-hoc alternative in practice for preliminary data evaluation.Additionally, panel agreement for TCATA data is obtained over time using Gwet’s AC1 coefficient and its 95% confidence bands, along with raw agreement and mean citation rate. Results provide insight into the level of panel agreement across the various time segments.We exemplify the suggested methods by means of data from a study on Syrah wine finish.

The size and impact of the elite set of publications in scientometric assessments

Several scientometric impact indicators [total citations, h, g, and ź-index, percentage rank position (PRP), weighted citation share (WCS)] of 190 elite papers of 15 members of the Hungarian Academy of Sciences active in three different fields were calculated. From the indices the PRP indicator proved to be independent of the citation practices in the fields. The PRP index of a journal paper can be calculated in per cent as unity minus (the rank number of the paper by citation frequency within the publishing journal minus one divided with the total number of papers in the journal) times hundred. The sum of the PRP index of the elite papers of a scientist may characterize his or her total publication performance. The size of the elite set of journal papers within the total was calculated by different methods. The h-index and g-index corresponds to the size of the elite, i.e. number of the elite papers according to the h-statistics and g-statistics, respectively. The number of papers in the ź-set is equal to the square root of total papers. The ź-index equals to one hundredth of citations to the ź-set papers. In the present paper the size of the elite set is determined as the number of papers in the h-set, g-set, or ź-set, and as 10 % of total papers, or number of papers cited 2, 3, or 5 times the mean citation rate (MCR) of the publishing journal. The ź-citation threshold model is presented for demonstrating how MCR and the distribution of citations over the papers may influence the size of the elite set and the corresponding ź-index. It was found that the scientific performances concluded from the ź-index obtained from elite sets of different size are in good agreement.

The Scientific Articles on Art Criticism

Research has been extremely involved in improving in the art criticism area. These improvements are reflected in scientific articles. This article purposed to investigate the 214 articles in art criticism to explore their main characteristics. These articles published in the Web of Science database of the Institute of Scientific Information (ISI) from the period of 1980 till 20 December 2013. Types of articles were article and review which is included in the study. The three top cited (more than 10 times citations) articles in art criticism were published in 1993 and 1999. The 214 articles mean citation rate was 0.87 (SD 2.38) times. Among the various fields, art (58.87%), arts humanities other topics (28.03%), both art and arts humanities other topics (5.14%), both art and education and educational research (2.33%), both art and history (1.40%), art, arts humanities other topics and literature (1.40%), both art and cultural studies (0.93%), both art and philosophy (0.93%), both art and literature (0.46%), and both arts humanities other topics and cultural studies (0.46%) were the most popular fields of research. The results showed that researches were done in the United States had highest citation which was written in English language.

Will Open Access Get Me Cited? An Analysis of the Efficacy of Open Access Publishing in Political Science

ABSTRACTThe digital revolution has made it easier for political scientists to share and access high-quality research online. However, many articles are stored in proprietary databases that some institutions cannot afford. High-quality, peer-reviewed, top-tier journal articles that have been made open access (OA) (i.e., freely available online) theoretically should be accessed and cited more easily than articles of similar quality that are available only to paying customers. Research into the efficacy of OA publishing thus far has focused mainly on the natural sciences, and the results have been mixed. Because OA has not been as widely adopted in the social sciences, disciplines such as political science have received little attention in the OA research. In this article, we seek to determine the efficacy of OA in political science. Our primary hypothesis is that OA articles will be cited at higher rates than articles that are toll access (TA), which means available only to paying customers. We test this hypothesis by analyzing the mean citation rates of OA and TA articles from eight top-ranked political science journals. We find that OA publication results in a clear citation advantage in political science publishing.

The impact of extreme observations in citation distributions

This article studies the role of extremely highly cited articles in two instances: the measurement of citation inequality and mean citation rates. Using a data set, acquired from Thomson Scientific, consisting of 4.4 million articles published in 1998–2003 in 22 broad fields with a 5-year citation window, the main results are the following. First, both within each of 22 broad fields and in the all-sciences case, citation inequality is strongly affected by the presence of a handful of extreme observations, particularly when it is measured by citation inequality indices that are very sensitive to citation differences in the upper tail of citation distributions. Second, the impact of extreme observations on citation averages is generally much smaller. The concluding section includes some practical lessons for students of citation inequality and/or users of high-impact indicators.

Comparative rank assessment of journal articles

To take into account the impact of the different bibliometric features of scientific fields and different size of both the publication set evaluated and the set used as reference standard, two new impact indicators are introduced. The Percentage Rank Position (PRP) indicator relates the ordinal rank position of the article assessed to the total number of papers in the publishing journal. The publications in the publishing journal are ranked by the decreasing citation frequency. The Relative Elite Rate (RER) indicator relates the number of citations obtained by the article assessed to the mean citation rate of the papers in the elite set of the publishing journal. The indices can be preferably calculated from the data of the publications in the elite set of journal papers of individuals, teams, institutes or countries. The number of papers in the elite set is calculated by the equation: P(πv)=(10logP)−10, where P is the total number of papers. The mean of the PRP and RER indicators of the journal papers assessed may be applied for comparing the eminence of publication sets across fields.

The Measurement of the Effect on Citation Inequality of Differences in Citation Practices across Scientific Fields

This paper has two aims: (i) to introduce a novel method for measuring which part of overall citation inequality can be attributed to differences in citation practices across scientific fields, and (ii) to implement an empirical strategy for making meaningful comparisons between the number of citations received by articles in 22 broad fields. The number of citations received by any article is seen as a function of the article’s scientific influence, and the field to which it belongs. A key assumption is that articles in the same quantile of any field citation distribution have the same degree of citation impact in their respective field. Using a dataset of 4.4 million articles published in 1998–2003 with a five-year citation window, we estimate that differences in citation practices between the 22 fields account for 14% of overall citation inequality. Our empirical strategy is based on the strong similarities found in the behavior of citation distributions. We obtain three main results. Firstly, we estimate a set of average-based indicators, called exchange rates, to express the citations received by any article in a large interval in terms of the citations received in a reference situation. Secondly, using our exchange rates as normalization factors of the raw citation data reduces the effect of differences in citation practices to, approximately, 2% of overall citation inequality in the normalized citation distributions. Thirdly, we provide an empirical explanation of why the usual normalization procedure based on the fields’ mean citation rates is found to be equally successful.

100 top-cited scientific papers in limb prosthetics

Research has tremendously contributed to the developments in both practical and fundamental aspects of limb prosthetics. These advancements are reflected in scientific articles, particularly in the most cited papers. This article aimed to identify the 100 top-cited articles in the field of limb prosthetics and to investigate their main characteristics. Articles related to the field of limb prosthetics and published in the Web of Knowledge database of the Institute for Scientific Information (ISI) from the period of 1980 to 2012. The 100 most cited articles in limb prosthetics were selected based on the citation index report. All types of articles except for proceedings and letters were included in the study. The study design and level of evidence were determined using Sackett’s initial rules of evidence. The level of evidence was categorized either as a systematic review or meta-analysis, randomized controlled trial, cohort study, case–control study, case series, expert opinion, or design and development. The top cited articles in prosthetics were published from 1980 to 2012 with a citation range of 11 to 90 times since publication. The mean citation rate was 24.43 (SD 16.7) times. Eighty-four percent of the articles were original publications and were most commonly prospective (76%) and case series studies (67%) that used human subjects (96%) providing level 4 evidence. Among the various fields, rehabilitation (47%), orthopedics (29%), and sport sciences (28%) were the most common fields of study. The study established that studies conducted in North America and were written in English had the highest citations. Top cited articles primarily dealt with lower limb prosthetics, specifically, on transtibial and transradial prosthetic limbs. Majority of the articles were experimental studies.

Multiplicative and fractional strategies when journals are assigned to several subfields

In many data sets, articles are classified into subfields through the journals in which they have been published. The problem is that while many journals are assigned to a single subfield, many others are assigned to several. This article discusses a multiplicative and a fractional strategy to deal with this situation. The empirical part studies different aspects of citation distributions under the two strategies, namely: the number of articles, the mean citation rate, the broad shape of the distribution, their characterization in terms of size- and scale-invariant indicators of high and low impact, and the presence of extreme distributions, that is, distributions that behave very differently from the rest. We found that, despite large differences in the number of articles according to both strategies, the similarity of the citation characteristics of articles published in journals assigned to one or several subfields guarantees that choosing one of the two strategies may not lead to a radically different picture in practical applications. Nevertheless, the characterization of citation excellence through a high-impact indicator may considerably differ depending on that choice.

The end of the “European Paradox”

This paper evaluates the European Paradox according to which Europe plays a leading world role in terms of scientific excellence, measured in terms of the number of publications, but lacks the entrepreneurial capacity of the US to transform this excellent performance into innovation, growth, and jobs. Citation distributions for the US, the European Union (EU), and the Rest of the World are evaluated using a pair of high- and low-impact indicators, as well as the mean citation rate (MCR). The dataset consists of 3.6 million articles published in 1998---2002 with a common 5-year citation window. The analysis is carried at a low aggregation level, namely, the 219 sub-fields identified with the Web of Science categories distinguished by Thomson Scientific. The problems posed by international co-authorship and the multiple assignments of articles to sub-fields are solved following a multiplicative strategy. We find that, although the EU has more publications than the US in 113 out of 219 sub-fields, the US is ahead of the EU in 189 sub-fields in terms of the high-impact indicator, and in 163 sub-fields in terms of the low-impact indicator. Finally, we verify that using the high-impact indicator the US/EU gap is usually greater than when using the MCR.

Sub-field normalization in the multiplicative case: Average-based citation indicators

This paper investigates the citation impact of three large geographical areas – the U.S., the European Union (EU), and the rest of the world (RW) – at different aggregation levels. The difficulty is that 42% of the 3.6 million articles in our Thomson Scientific dataset are assigned to several sub-fields among a set of 219 Web of Science categories. We follow a multiplicative approach in which every article is wholly counted as many times as it appears at each aggregation level. We compute the crown indicator and the Mean Normalized Citation Score (MNCS) using for the first time sub-field normalization procedures for the multiplicative case. We also compute a third indicator that does not correct for differences in citation practices across sub-fields. It is found that: (1) No geographical area is systematically favored (or penalized) by any of the two normalized indicators. (2) According to the MNCS, only in six out of 80 disciplines – but in none of 20 fields – is the EU ahead of the U.S. In contrast, the normalized U.S./EU gap is greater than 20% in 44 disciplines, 13 fields, and for all sciences as a whole. The dominance of the EU over the RW is even greater. (3) The U.S. appears to devote relatively more – and the RW less – publication effort to sub-fields with a high mean citation rate, which explains why the U.S./EU and EU/RW gaps for all sciences as a whole increase by 4.5 and 5.6 percentage points in the un-normalized case. The results with a fractional approach are very similar indeed.

The different impact on scientific output of academic-owned vs. academic-invented patents: evidence from a panel of Italian academic inventors

This paper explores, by means of descriptive and econometric methods, the relation between publishing and patenting at the individual scientist’s level in a large multi-faculty university for the period 1999–2005. It studies the relationship between academic patenting and publications count (per researcher per year) and mean citation rate, both at the pooled level and by a distinction between university- and firm-owned academic patents. Coherently with the current empirical literature, the results show a positive and significant effect of patenting on publications and citations, although with different intensity over time lags. Our paper goes beyond these results (well established in the literature) and detects the existence of differentiated impacts for three main types of academic patents: privately owned, publicly owned and other patents (including those individually owned and with mixed private–public ownership). According to our results, scientists patenting with industries are more productive in terms of scientific output after the patenting event.

Response to Watson R (2011) Commentary on Oermann MH, Shaw‐Kokot J, Knafl GJ & Dowell J (2010) Dissemination of research into clinical nursing literature. Journal of Clinical Nursing19, 3435–3442. Journal of Clinical Nursing20, 595–596

The authors would like to thank Watson (2011) for his insightful remarks on our paper (Oermann et al. 2010). Watson (2011) highlights two relevant issues from our research. The first issue is that of the tension between citation analysis and author impact. Of the two, citation analysis has the longest history and appears more widely accepted. Author impact, the h-index, is still developing. The second issue that Watson identifies involves citation patterns, which is getting increased attention in the literature. Our citation analysis examined the assimilation of selected nursing research articles into clinical literature. The articles selected for analysis were published between 1990–2000 and tracked from publication to 2009. During this period, journals moved from print only to print and electronic format. A 2010 study examined how electronic journal collections were reflected in citation patterns of authors in dentistry, nursing and pharmacy at a large urban university (De Groote & Barrett 2010). From 1996–2008, the number of citations to articles in print only journals in nursing decreased while the number of citations to papers available online increased. The authors suggested improved linking of full text articles directly into electronic databases possibly contributed to the increase. Based on De Groote and Barrett’s study, nursing faculty are citing papers published online more than those published only in print. In response to Watson (2011), we reanalysed the citations in our study to determine whether there was an increase in the number of citations after the journals began publishing in both print and electronic formats. The number of citations and time in years over the periods when the journals were available in print only and both print and electronic formats were analysed. Poisson regression was used to model numbers of citations as they are counts while converting them to citation rates per year (using the appropriate offset variable). Generalised estimating equations (GEE) techniques were used to model the correlation for an article’s citation rate before and after its journal began publishing papers electronically as well as in print. The GEE estimated intra-article correlation was high at 0·65, indicating that an article’s print citation rate was highly correlated with its print plus electronic citation rate. The mean citation rate increased significantly (p = 0·011) from its estimated print only value of 1·5 citations per year to its print plus electronic citation rate of 2·4 citations per year. Another factor influencing electronic access is the linking of electronic articles to databases. Some database vendors negotiate directly with publishers to create links. For example, in 1995, the Cumulative Index to Nursing and Allied Health Literature (CINAHL) began providing direct full text links to selected journals. There are currently over 700 linked journals. Another option, software such as Serials Solutions OpenURL linking, goes beyond a single database to include linking in all databases subscribed to by the library or institution. While there are many reasons authors cite other studies, we suggest that electronic access to articles influences citation patterns. This area deserves more analysis.

Reference standards and reference multipliers for the comparison of the citation impact of papers published in different time periods

AbstractIn this study, reference standards and reference multipliers are suggested as a means to compare the citation impact of earlier research publications in physics (from the period of “Little Science” in the early 20th century) with that of contemporary papers (from the period of “Big Science,” beginning around 1960). For the development of time‐specific reference standards, the authors determined (a) the mean citation rates of papers in selected physics journals as well as (b) the mean citation rates of all papers in physics published in 1900 (Little Science) and in 2000 (Big Science); this was accomplished by relying on the processes of field‐specific standardization in bibliometry. For the sake of developing reference multipliers with which the citation impact of earlier papers can be adjusted to the citation impact of contemporary papers, they combined the reference standards calculated for 1900 and 2000 into their ratio. The use of reference multipliers is demonstrated by means of two examples involving the time adjusted h index values for Max Planck and Albert Einstein.

Average-based versus high- and low-impact indicators for the evaluation of scientific distributions

Albarrán et al (2009a) introduced a novel methodology for the evaluation of citation distributions using a pair of high- and low-impact measures defined over the set of articles with citations below or above a critical citation level. Albarrán et al (2009b) presented the first empirical applications to a situation in which the world citation distribution in 22 scientific fields is partitioned into three geographical areas: the USA, the European Union, and the rest of the world. In this paper, we find that cardinal differences between the results obtained with our high-impact index and the mean citation rate are of a large order of magnitude. When, in addition, the percentage in the top 5% of most cited articles or the percentage of uncited articles are used, there are still important quantitative differences with respect to the high- and low-impact indicators advocated in our approach.

A research impact indicator for institutions

This paper introduces a new impact indicator for the research effort of a university, n h 3. The number of documents or the number of citations obtained by an institution are used frequently in international ranking of institutions. However, these are very dependent on the size and this is inducing mergers with the apparent sole goal of improving the research ranking. The alternative is to use the ratio of the two measures, the mean citation rate, that is size independent but it has been shown to fluctuate along the time as a consequence of its dependence on a very small number of documents with an extremely good citation performance. In the last few years, the popularity of the Hirsch index as an indicator of the research performance of individual researchers led to its application to journals and institutions. However, the original aim of this h index of giving a mixed measure of the number of documents published and their impact as measured by the citations collected along the time is totally undesirable for institutions as the overall size may be considered irrelevant for the impact evaluation of research. Furthermore, the h index when applied to institutions tends to retain a very small number of documents making all other research production irrelevant for this indicator. The n h 3 index proposed here is designed to measure solely the impact of research in a way that is independent of the size of the institution and is made relatively stable by making a 20-year estimate of the citations of the documents produced in a single year.

Mean citation rate per article in mathematics journals: Differences from the scientific model

AbstractThis paper analyzes the applicability of the article mean citation rate measures in theScience Citation Index Journal Citation Reports(SCI JCR) to the five JCR mathematical subject categories. These measures are the traditional 2‐year impact factor as well as the recently added 5‐year impact factor and 5‐year article influence score. Utilizing the 2008 SCI JCR, the paper compares the probability distributions of the measures in the mathematical categories to the probability distribution of a scientific model of impact factor distribution. The scientific model distribution is highly skewed, conforming to the negative binomial type, with much of the variance due to the important role of review articles in science. In contrast, the three article mean citation rate measures' distributions in the mathematical categories conformed to either the binomial or Poisson, indicating a high degree of randomness. Seeking reasons for this, the paper analyzes the bibliometric structure of Mathematics, finding it a disjointed discipline of isolated subfields with a weak central core of journals, reduced review function, and long cited half‐life placing most citations beyond the measures' time limits. These combine to reduce the measures' variance to one commensurate with random error. However, the measures were found capable of identifying important journals. Using data from surveys of the Louisiana State University (LSU) faculty, the paper finds a higher level of consensus among mathematicians and others on which are the important mathematics journals than the measures indicate, positing that much of the apparent randomness may be due to the measures' inapplicability to mathematical disciplines. Moreover, tests of the stability of impact factor ranks across a 5‐year time span suggested that the proper model for Mathematics is the negative binomial.

A comparison of the scientific performance of the U.S. and the European union at the turn of the 21st century

In this paper, scientific performance is identified with the impact that journal articles have through the citations they receive. In 15 disciplines, as well as in all sciences as a whole, the EU share of total publications is greater than that of the U.S. However, as soon as the citations received by these publications are taken into account the picture is completely reversed. Firstly, the EU share of total citations is still greater than the U.S. in only seven fields. Secondly, the mean citation rate in the U.S. is greater than in the EU in every one of the 22 fields studied. Thirdly, since standard indicators—such as normalized mean citation ratios—are silent about what takes place in different parts of the citation distribution, this paper compares the publication shares of the U.S. and the EU at every percentile of the world citation distribution in each field. It is found that in seven fields the initial gap between the U.S. and the EU widens as we advance towards the more cited articles, while in the remaining 15 fields—except for Agricultural Sciences—the U.S. always surpasses the EU when it counts, namely, at the upper tail of citation distributions. Finally, for all sciences as a whole the U.S. publication share becomes greater than that of the EU for the top 50% of the most highly cited articles. The data used refers to 3.6 million articles published in 1998–2002, and the more than 47 million citations they received in 1998–2007.