Citation Scores Research Articles

Commentary: The appearance of new medical cosmologies and the re-appearance of sick and healthy men and women: a comment on the merits of social theorizing

Cosmology, 1770–1870’ 1 is something of a classic. A search using the Thomson ISI Web of Knowledge Service reveals that, within this database, the article has some 133 citations and rate of citations have remained consistent over time. Furthermore, the impact is not limited to sociology; the top six subject areas that return the highest citation scores are ‘history and philosophy of science’ (29.3%); ‘social sciences, biomedical’ (23.3%); ‘public, environmental and occupational health’ (21.1%); ‘sociology’ (20.3%); ‘health care sciences and services’ (12.8%); and ‘medicine, general and internal’ (6.0%). It is intriguing that this conceptual paper should have such an enduring and broad significance. But herein lies its merit; as the sociologist Bryan Turner notes: ‘Ultimately sociology can better serve the practical problems and needs of patients by formulating sociological rather than medical questions.’ 2 This statement reflects Strauss’s well-known distinction between ‘sociology in medicine’ and ‘sociology of medicine’. 3 The former refers to research agendas that are professionally and institutionally driven, whereas the latter implies a more critical approach, wherein the philosophical and epistemological underpinnings of medical knowledge and medical practice are open to scrutiny. Jewson’s analysis certainly scrutinises the content of medical knowledge and reveals how it is contingent upon socio-economic relations between the patrons and producers. At a time when health and social researchers are increasingly encouraged to undertake empirically-based and policyorientated research it is useful to reflect on the

Libcitations: A measure for comparative assessment of book publications in the humanities and social sciences

AbstractBibliometric measures for evaluating research units in the book‐oriented humanities and social sciences are underdeveloped relative to those available for journal‐oriented science and technology. We therefore present a new measure designed for book‐oriented fields: the “libcitation count.” This is a count of the libraries holding a given book, as reported in a national or international union catalog. As librarians decide what to acquire for the audiences they serve, they jointly constitute an instrument for gauging the cultural impact of books. Their decisions are informed by knowledge not only of audiences but also of the book world (e.g., the reputations of authors and the prestige of publishers). From libcitation counts, measures can be derived for comparing research units. Here, we imagine a match‐up between the departments of history, philosophy, and political science at the University of New South Wales and the University of Sydney in Australia. We chose the 12 books from each department that had the highest libcitation counts in the Libraries Australia union catalog during 2000 to 2006. We present each book's raw libcitation count, its rank within its Library of Congress (LC) class, and its LC‐class normalized libcitation score. The latter is patterned on the item‐oriented field normalized citation score used in evaluative bibliometrics. Summary statistics based on these measures allow the departments to be compared for cultural impact. Our work has implications for programs such as Excellence in Research for Australia and the Research Assessment Exercise in the United Kingdom. It also has implications for data mining in OCLC's WorldCat.

The Poverty of Citation Databases: Data Mining Is Crucial for Fair Metrical Evaluation of Research Performance

For a long time, the journal impact factor has been used to evaluate the scientific performance of authors. It is increasingly recognized, however, that judging an author’s scientific performance should take into account that author’s scientific output, and not the output of other authors publishing in the same journal—the citation rates of papers in one journal can vary enormously, and the journal impact factor fails to that consider that variance. The number of citations an author attracts is a reliable measure of the attention the author receives from the scientific community, or, in other words, of the scientific impact of an author. (Attention is a lame arbiter of scientific quality, but that is a problem that cannot be solved by any simple metrics.) In 2005, Jorge E. Hirsch proposed a simple, elegant measure of an author’s impact: the h index, which is the number of an author’s papers (h) with at least h citations. Other author-based indexes have been proposed, such as the g index, which, given a set of papers ranked in decreasing order of the number of citations received, is the largest number such that the top g articles received together at least g 2 citations (Egghe 2006). The g index better takes into account the citation scores of top articles.

Citation Scores for Ancient Historians in the United States

This survey of citation scores provides a rough measure of the relative impact of scholarship published by forty-eight leading ancient historians in the United States.

Measuring the validity of early health technology assessment: Bibliometrics as a tool to indicate its scientific basis

The aim of this study was to assess whether publications of importance for improving the health system and its technologies are highly cited intrascientifically. Bibliometric assessment of the 596 publications used as sources in the fifty SBU Alerts from 2001 to 2004 from the Swedish Council on Technology Assessment in Health Care was carried out using the Thomson Scientific citation indexes. Normalized citation scores were calculated for all included studies. Additional factors such as the time and place of the research, subject categories, and journal source were analyzed. On average, the sources in SBU Alert have been cited eight times more than the world average consistently during the time period and across research areas. Articles used as its scientific basis are often published in a few, high impact, general medical journals. However, many of the articles are published in field-specific journals with low impact factors. Most articles used in SBU Alert are published by authors based in the United States or the United Kingdom. However, Swedish, Danish, and Dutch publications are overrepresented in its science base, whereas Japanese, Taiwanese, Indian, and Russian publications are underrepresented. Publications used as sources in a Swedish system for identification and early assessment of new methods in health care are also highly cited within the scientific community. This finding increases the appropriateness of using bibliometric indicators in evaluations of clinical research and suggests that decision makers through SBU Alert are getting scientifically sound advice.

The Hidden Perils of Citation Counting for Australasian Political Science

In a recent article in Australian Journal of Political Science, Dale and Goldfinch present ‘standard’ journal-based publication and citation rankings of Australasian political science departments designed to complement what they characterise as the multidisciplinary, historical, qualitative and humanistic political science of the region. However, the ‘highly cited’ articles in their top-ranked political science department belong to quantitative psychology. Through unravelling why their study favours the opposite of that which it was meant to detect, this paper alerts political scientists to the hidden perils of accepting ‘standard’ Institute of Scientific Information-based approaches to citation counting as valid measures of research ‘quality’. It exposes the veiled bibliometric assumption that the ‘best’ social science is quantitative research, notes that incongruous citation scores may inform the distribution of block funding and departmental appointment processes, and warns against using ‘standard’ data to unintentionally self-police the future shape of Australasian political science.

Evaluating Academic Journals using Impact Factor and Local Citation Score

This study presents a method for journal collection evaluation using citation analysis. Cost-per-use (CPU) for each title is used to measure cost-effectiveness with higher CPU scores indicating cost-effective titles. Use data are based on the impact factor and locally collected citation score of each title and is compared to the cost of managing and subscribing to journals. The study also conducts a case study to demonstrate how academic libraries could apply the method to local studies.

Characterizing creative scientists in nano-S&T: Productivity, multidisciplinarity, and network brokerage in a longitudinal perspective

While some believe that publication and citation scores are key predictors of breakthroughs in science, others claim that people who work at the intersection of scientific communities are more likely to be familiar with selecting and synthesizing alternatives into novel ideas. This paper contributes to this controversy by presenting a longitudinal comparison of highly creative scientists with equally productive researchers. The sample of creative scientists is identified by combining information on science awards and nominations by international peers covering research accomplishments in the mid-1990s. Results suggest that it is not only the sheer quantity of publications that causes scientists to produce creative pieces of work. Rather, their ability to effectively communicate with otherwise disconnected peers and to address a broader work spectrum also enhances their chances to be widely cited and to develop novel ideas.

Lifting the crown—citation z-score

Researchers worldwide are increasingly being assessed by the citation rates of their papers. These rates have potential impact on academic promotions and funding decisions. Currently there are several different ways that citation rates are being calculated, with the state of the art indicator being the crown indicator. This indicator has flaws and improvements could be considered. An item oriented field normalized citation score average ( c ¯ f ) is an incremental improvement as it differs from the crown indicator in so much as normalization takes place on the level of individual publication (or item) rather than on aggregated levels, and therefore assigns equal weight to each publication. The normalization on item level also makes it possible to calculate the second suggested indicator: total field normalized citation score (Σ c f). A more radical improvement (or complement) is suggested in the item oriented field normalized logarithm-based citation z-score average ( c ¯ f z [ ln ] or citation z-score). This indicator assigns equal weight to each included publication and takes the citation rate variability of different fields into account as well as the skewed distribution of citations over publications. Even though the citation z-score could be considered a considerable improvement it should not be used as a sole indicator of research performance. Instead it should be used as one of many indicators as input for informed peer review.

Whither Latin America? trends and challenges of science in Latin America

Science in Latin America has experienced vigorous growth in the past decade, as demonstrated by the fact that the Latin American share of the world's scientific publications increased from 1.8% in 1991-1995 to 3.4% in 1999-2003. Significant growth has also taken place in the numbers of PhDs in science and engineering (S&E) awarded in Latin American countries in recent years, including those in the natural sciences. Importantly, albeit at different rates, growth has been verified in almost all countries in the region, indicating a general effort to promote the development of S&E. In most research fields, however, the recognition or relative impact of Latin American science, as measured by the average number of citations received by published articles (CpP), is still below world averages and much lower than in developed nations. We show that average CpP values for a set of 34 representative developing and developed countries correlate significantly with gross expenditure in research and development (GERD), with gross domestic product (GDP) per capita and with the number of researchers per million inhabitants (RpM). Among those countries, Latin American nations present some of the lowest average values of CpP (<6), GERD (< or =1% of GDP) and RpM (<2,000). We also examined recent trends in scientific activity in Latin America, with focus on the natural sciences and on biochemistry and molecular biology (BMB). In terms of citation scores, publications in BMB compare favorably to those in other research fields within Latin America. At the same time, however, Latin American BMB is one of the areas for which relative impact--compared to developed nations or normalized to world averages--is lowest. These observations clearly indicate the need to establish effective policies to increase competitiveness in terms of the quality and international recognition of Latin American natural sciences in general, and BMB in particular, as opposed to merely increasing the absolute numbers of publications or the numbers of PhDs awarded in the region.

Theory and practise of the g-index

The g-index is introduced as an improvement of the h-index of Hirsch to measure the global citation performance of a set of articles. If this set is ranked in decreasing order of the number of citations that they received, the g-index is the (unique) largest number such that the top g articles received (together) at least g 2 citations. We prove the unique existence of g for any set of articles and we have that g ≥ h. The general Lotkaian theory of the g-index is presented and we show that g = (α-1 / α-2) α-1/α T 1/α where a> 2 is the Lotkaian exponent and where T denotes the total number of sources. We then present the g-index of the (still active) Price medallists for their complete careers up to 1972 and compare it with the h-index. It is shown that the g-index inherits all the good properties of the h-index and, in addition, better takes into account the citation scores of the top articles. This yields a better distinction between and order of the scientists from the point of view of visibility.

Delineating complex scientific fields by an hybrid lexical-citation method: An application to nanosciences

Relevance of bibliometric indicators on scientific areas critically depends on the quality of their delineation. Macro-level studies, often based on a selected list of journals, accept a high degree of fuzziness. Micro-level studies rely on sets of individual articles in order to reduce noise and enhance precision of retrieval. The most usual information retrieval process is based on lexical queries with various levels of sophistication. In the experiment on Nanosciences reported here, this process was used as a first step, to delineate a ‘seed’ of literature. It has strong limitations, especially for emerging or transversal fields. In a second step, the alternative approach of citation linkages, was used to expand the bibliography starting from lexical seed. The extension process presented is ruled by three parameters, two deal with the cited side (threshold on citation score, and specificity towards the field), one with the citing side (threshold on the number of relevant references) interplaying in the ‘referencing structure’ function (RSF) introduced in a previous work. This type of combination proves effective for delineating the transversal field of Nanosciences. Further improvements of the method are discussed.

Evaluación del impacto socioeconómico de la investigación en enfermería

This article first presents a brief analysis of nursing research in Spain. The analysis shows that, despite its potential benefits, Spanish nursing research is still an emergent discipline. Nevertheless, in recent years several interesting initiatives to change the current situation have been undertaken. Among these initiatives is the inclusion of courses on research methodology within undergraduate programs and the increase of nurses with first degrees and doctorates. Secondly, several characteristics that should be taken into account when evaluating nursing research are discussed. The use of indicators that complement standard criteria (i.e., impact factor) is strongly recommended. These new indicators (or indexes) should take into account issues such as the clinical applicability of research. Their implementation and use could help steer nurses away from researching more theoretical topics with the aim of gaining greater scientific impact to the detriment of topics with a higher social value and lower citation score. Finally, several methods for assessing the socioeconomic impact of nursing research are recommended. Among other research designs, we consider both case studies and surveys to be especially appropriate for this goal.

The Perception of Library and Information Science Journals by LIS Education Deans and ARL Library Directors: A Replication of the Kohl–Davis Study

Analyzing the collective opinion of presumed experts, often termed a perception study, is a frequently used approach for rating journals or evaluating education programs. Replicating the 1985 Kohl–Davis study, seventy-one library and information science (LIS) journals are ranked according to their mean rating on a 1 to 5 ordinal scale by deans of ALA-accredited education programs and by the directors of ARL libraries (surveyed during the summer of 2003). Comparison of the results with the 1985 study found considerable continuity in journal perceptions over the past two decades, but more so by directors than deans. A weak to moderate correlation was found between deans’ ratings and Journal Citation Reports citation scores, whereas the correlations between directors’ perceptions and citation data were weak to nonexistent. The findings confirm a hierarchy of prestige among LIS journals, but the hierarchical order differs somewhat between deans and directors.

Continuous, weighted Lorenz theory and applications to the study of fractional relative impact factors

This paper introduces weighted Lorenz curves of a continuous variable, extending the discrete theory as well as the non-weighted continuous model. Using publication scores (in function of time) as the weights and citation scores (in function of time) as the dependent variables, we can construct an “impact Lorenz curve” in which one can read the value of any fractional impact factor, i.e. an impact factor measured at the time that a certain fraction of the citations is obtained or measured at the time a certain fraction of the publications is obtained. General properties of such Lorenz curves are studied and special results are obtained in case the citation age curve and publication growth curve are exponential functions. If g is the growth rate and c is the aging rate we show that ln c ln 1 g determines the impact Lorenz curve and also we show that any two situations give rise to two non-intersecting (except in (0, 0) and (1, 1)) Lorenz curves. This means that, for two situations, if one fractional impact factor is larger than the other one, the same is true for all the other fractional impact factors. We show, by counterexample that this is not so for “classical” impact factors, where one goes back to fixed time periods. The paper also presents methods to determine the rates c and g from practical data and examples are given.

Going beyond journal classification for evaluation of research outputs

PurposeSeeks to characterise world astronomy research during the last decade by an analysis of papers in the Science Citation Index identified with a special filter and to study Indian output in order to identify the leading institutions and authors.Design/methodology/approachLists of specialist journals and title words of papers were selected to create a filter giving high precision and recall for astronomy papers. Some biology papers were erroneously retrieved because of ambiguous title words. Potential citation impact was determined from journal citation scores, and multiple regression was used to evaluate leading countries.FindingsTitle words added almost a quarter to the list of papers in specialist journals, and the final file contained over 96,000 papers. Potential impact increased with more authors per paper and more addresses; it was greater for papers from Canada, the UK and the USA, and less for papers from China, India and Russia; for other countries the effects of the author's location on potential impact were not statistically significant. Indian astronomy output has increased in potential impact, partly through greater international co‐authorship, but also through indigenous papers.Research limitations/implicationsThe study was confined to one subject area, and impact was determined on the basis of journals, not of individual papers.Practical implicationsUse of title words in addition to journal lists is essential to sub‐field definition in order to have high precision and recall. Because of the confounding effects of authorship numbers, it is necessary to use multiple regression analysis in order to see whether research from a given country is significantly better or worse than average.Originality/valueCharacterises world astronomy research during the last decade by an analysis of papers in the Science Citation Index identified with a special filter.

Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalisation

As citation practices strongly depend on fields, field normalisation is recognised as necessary for fair comparison of figures in bibliometrics and evaluation studies. However fields may be defined at various levels, from small research areas to broad academic disciplines, and thus normalisation values are expected to vary. The aim of this project was to test the stability of citation ratings of articles as the level of observation - hence the basis of normalisation - changes. A conventional classification of science based on ISI subject categories and their aggregates at various scales was used, namely at five levels: all science, large academic discipline, sub-discipline, speciality and journal. Among various normalisation methods, we selected a simple ranking method (quantiles), based on the citation score of the article in each particular aggregate (journal, speciality, etc.) it belonged to at each level. The study was conducted on articles in the full SCI range, for publication year 1998 with a four-year citation window. Stability is measured in three ways: overall comparison of article rankings; individual trajectory of articles; survival of the top-cited class across levels. Overall rank correlations on the observed empirical structure are benchmarked against two fictitious sets that keep the same embedded structure of articles but reassign citation scores either in a totally ordered or in a totally random distribution. These sets act respectively as a 'worst case' and 'best case' for the stability of citation ratings. The results show that: (a) the average citation rankings of articles substantially change with the level of observation (b) observation at the journal level is very particular, and the results differ greatly in all test circumstances from all the other levels of observation (c) the lack of cross-scale stability is confirmed when looking at the distribution of individual trajectories of articles across the levels; (d) when considering the top-cited fractions, a standard measure of excellence, it is found that the contents of the 'top-cited' set is completely dependent on the level of observation. The instability of impact measures should not be interpreted in terms of lack of robustness but rather as the co-existence of various perspectives each having their own form of legitimacy. A follow-up study will focus on the micro levels of observation and will be based on a structure built around bibliometric groupings rather than conventional groupings based on ISI subject categories.

Scale-Independent Bibliometric Indicators

(2005). Scale-Independent Bibliometric Indicators. Measurement: Interdisciplinary Research and Perspectives: Vol. 3, No. 1, pp. 24-28.

Citation‐enhanced indexing/abstracting databases

Provides an introductory look at what savvy users should know about the implications of having information about cited references in I/A records. Looks at the more sophisticated, link‐enabled cited references and the novel citation scores in full‐text collections, then discusses the alternatives for searching efficiently by elements of cited references: cited author, cited title, cited source and cited year in I/A databases and full‐text archives.

The Moral Standing of Non-human Primates: Why They Merit Special Consideration

Scientific experiments with non-human primates are viewed very controversially. Those who use non-human primates for scientific purposes contend that the results will be of great benefit to humans. They say that the distress to the animals is minimal and that, therefore, their experiments are ethically acceptable. On the other hand, there are those who oppose non-human primate experiments altogether, due to the close relationship between these animals and humans. In a literature study on non-human primate experiments in fundamental brain research, the outcome of research projects performed over the period of a decade was evaluated retrospectively. Objective criteria were applied both for determining the cost (internationally published distress scores, taking into account distress due to procurement, transportation and housing) and the benefit (evaluating publications that arose from the experiments, citation scores and so forth). Taking the example of electrophysiological recordings in the brain of the awake macaque, this balancing ended up very different from that proposed by the scientists, and did not support the conclusion that the experiments had been ethically acceptable.