Citation Bias Research Articles

Introduction to study design

AbstractMuch research is never published. The main cause of failure is the poor quality of the data, which does not allow valid conditions to be drawn. This problem can arise because of poor study design. The aims of this presentation are to discuss quality of data and the study designs developed to improve data validity. It is essential to be aware of the many sources of bias, which can fool the inexpert scientist at any stage of the project. These include selection bias, chanelling bias, interview bias, chronology bias, recall bias, citation bias, confounding, and others. These shortcomings can be avoided by proper study design. Different levels of evidence have been defined to categorize study designs. These levels include: 'expert opinion without critical appraisal'; case‐series; case control studies; cohort studies and randomized controlled trials, with systematic reviews in each category being superior to individual studies. Research should be undertaken only by individuals and teams who have a good understanding of study design. Adequate preparation before initiating the study should result in valid data and a good chance of publication in a high‐impact journal, if the research findings are indeed new and useful.

Publication bias: what is it? How do we measure it? How do we avoid it?

Publication occurs when results of published studies are systematically different from results of unpublished studies. The term dissemination bias has also been recommended to describe all forms of biases in the research-dissemination process, including outcome-reporting bias, time-lag bias, gray-literature bias, full-publication bias, language bias, citation bias, and media-attention bias. We can measure publication by comparing the results of published and unpublished studies addressing the same question. Following up cohorts of studies from inception and comparing publication levels in studies with statistically significant or positive results suggested greater odds of formal publication in those with such results, compared to those without. Within reviews, funnel plots and related statistical methods can be used to indicate presence or absence of publication bias, although these can be unreliable in many circumstances. Methods of avoiding publication bias, by identifying and including unpublished outcomes and unpublished studies, are discussed and evaluated. These include searching without limiting by outcome, searching prospective trials registers, searching informal sources, including meeting abstracts and PhD theses, searching regulatory body websites, contacting authors of included studies, and contacting pharmaceutical or medical device companies for further studies. Adding unpublished studies often alters effect sizes, but may not always eliminate publication bias. The compulsory registration of all clinical trials at inception is an important move forward, but it can be difficult for reviewers to access data from unpublished studies located this way. Publication may be reduced by journals by publishing high-quality studies regardless of novelty or unexciting results, and by publishing protocols or full-study data sets. No single step can be relied upon to fully overcome the complex actions involved in publication bias, and a multipronged approach is required by researchers, patients, journal editors, peer reviewers, research sponsors, research ethics committees, and regulatory and legislation authorities.

Gender Gap or Gender Bias in Peace Research? Publication Patterns and Citation Rates forJournal of Peace Research, 1983-2008

Many studies report lower academic productivity among women. But are women less likely to get their research published in the first place? The evidence for potential gender bias in publication and impact is mixed. This article examines the gender dimension of scientific publication in international relations (IR) based on submission data for Journal of Peace Research for the period 1983–2008. It examines the gender gap in submissions and explores whether the perceived merit of a research paper is affected by the gender of the authors and reviewers. It also investigates whether the gender of the first author influences citation counts. The data show a clear but declining gender gap. They do not indicate any significant gender bias in publication success or citations.

The Evolutionary Ecology of Nest Construction: Insight from Recent Fish Studies

Nests are built by a wide variety of animals as functional receptacles for developing eggs and offspring, and they play a critical role in the reproductive biology of many species. Traditionally, research on the ecology and evolution of nest building and construction behaviour has focused primarily on birds, and avian studies have dominated the literature. However, as researchers working on non-bird models have realised the importance of nest construction in evolutionary ecology, the number of studies published on the nesting behaviour of non-bird taxa has increased. An analysis of the literature reveals that fish have become major models for studying many aspects of nest building behaviour, but whereas studies of fish nest building behaviour frequently cite classical and contemporary bird studies, the findings of recent fish nesting research appears to be slower to be recognised by bird biologists. Further analysis reveals that this citation bias may arise because of the tendency of nest building studies to be published in taxon-specific, often local journals, and this may be especially the case for bird-focused studies. In this review, I summarise the recent literature on fish nesting behaviour, focusing on aspects that should be of mutual interest to fish and bird biologists. I hope that the review may be used by bird biologists to identify complementary and insightful nest-building research in fish, and that researchers with interests in the nest-building behaviour of animals across the taxonomic spectrum might use the review to focus on questions of mutual and general importance and interest.

Citation bias favoring statistically significant studies was present in medical research

ObjectiveStatistically significant studies may be cited more than negative studies on the same topic. We aimed to assess here whether such citation bias is present across the medical literature. Study Design and SettingWe conducted a cohort study of the association between statistical significance and citations. We selected all therapeutic intervention studies included in meta-analyses published between January and March 2010 in the Cochrane database, and retrieved citation counts of all individual studies using ISI Web of Knowledge. The association between the statistical significance of each study and the number of citations it received between 2008 and 2010 was assessed in mixed Poisson models. ResultsWe identified 89 research questions addressed in 458 eligible articles. Significant studies were cited twice as often as nonsignificant studies (multiplicative effect of significance: 2.14, 95% confidence interval: 1.38–3.33). This association was partly because of the higher impact factor of journals where significant studies are published (adjusted multiplicative effect of significance: 1.14, 95% confidence interval: 0.87–1.51). ConclusionA citation bias favoring significant results occurs in medical research. As a consequence, treatments may seem more effective to the readers of medical literature than they really are.

Personality Psychology in the First Decade of the New Millennium: A Bibliometric Portrait

Nine principal personality psychology journals— Journal of Personality and Social Psychology (JPSP), Journal of Personality (JP), Journal of Research in Personality (JRP), European Journal of Personality (EJP), Personality and Individual Differences (PAID), Personality and Social Psychology Bulletin (PSPB), Personality and Social Psychology Review (PSPR), Journal of Personality Assessment (JPA), and Journal of Personality Disorders (JPD)—have published 8510 research papers from 2001 to 2010. These papers have been cited 149 108 times (September 2011) by papers published in journals indexed in the Web of Science. Although personality psychologists from the US published the largest number of papers (4924, 57.9%) and had the largest number of citations (101 875, 68.3%), their relative contribution to personality literature has slightly diminished during the first decade of the new millennium. Unlike other countries, personality psychologists residing in the US demonstrated a strong country self–citation bias: They were about 14% more likely to cite papers which were written by their compatriots rather than non–US authors in three leading journals JPSP, PSPB, and PSPR. The intensity and pattern of citations indicate that personality psychology indeed occupies one of the core positions at the heart of psychological knowledge. Copyright © 2012 John Wiley & Sons, Ltd.

Dogmatic is problematic: Interpreting evidence for latitudinal gradients in herbivory and defense

It has long been believed that plant-animal interactions, including herbivory, are more intense toward the tropics. The more intense interactions in the tropics are thought to have selected for higher levels of defenses against herbivores. These ideas are fundamental to our understanding of global patterns in diversity, and for our understanding of plant-animal interactions. However, recent analyses have tended not to support the traditional hypothesis of higher herbivory and defenses at lower latitudes. Despite mounting empirical evidence, many ecologists have been slow to re-assess their beliefs. I show clear evidence for citation bias, with papers that support the traditional idea being cited over six times as often as papers that show higher herbivory at higher latitudes and over four times as often as papers showing higher defense at higher latitudes. I also highlight examples where interpretations that are counter to the available empirical evidence have been published in high profile journals. I suggest that providing rigorous empirical tests for ideas that have become widely established without appropriate testing should be a priority for ecologists. We need to make sure the objectivity of peer-reviewed science stands out from the mass of unchecked opinion available on the web.

Automated citation sentiment analysis: What can we learn from biomedical researchers

AbstractAutomated citation sentiment analysis is a newly emerged research topic inspired by traditional citation context analysis in scientometrics and applied linguistics. The main goals of current citation sentiment analysis are to develop new tools to model scientific literature and provide authoring support for researchers in tasks like literature review. In terms of developing authoring support tools, however, current studies have not taken into consideration the behavioral patterns of researchers' literature review practice, leaving user need assessment as a missing piece in current interdisciplinary research effort. This paper analyzed biomedical researchers' need by reviewing their publications on using manual citation sentiment analysis for detecting citation bias, and discussed the differences between biomedical researchers' approach and current automated citation sentiment analysis model. These differences are expected to inform the modeling of automated citation sentiment analysis.

Non-English journals and papers in physics and chemistry: bias in citations?

This study investigates, at the journal as well as the article level, if there is a difference in citations between English-language and non-English publications. The Web of Knowledge is used as data source. The investigation focuses on the fields of physics and chemistry. Using a precise definition of a "non-English journal", we filter out nine physics and thirty-four chemistry non-English journals, scattered over six physics and seven chemistry subfields. Average received citations per paper (CpP) of the non-English journal(s) are compared with the CpP of pure English journals, and this in the same subfield. We clearly observe that non-English journals are inferior--in number of citations received--to pure English journals and this in all physics and chemistry subfields studied. Further, twelve physics journals and ten chemistry journals were chosen as sample journals to compare the CpP of non-English papers with that of English language papers in the same journal. The result of this comparison is that for the majority of these journals and for most of the publication years the CpP of non-English papers is lower than that of the English language papers. Finally, analyzing linguistic characteristics of the citing literature confirms the own-language preference in non-English physics and chemistry journals.

Geographical knowledge diffusion and spatial diversity citation rank

This paper proposes a citation rank based on spatial diversity (SDCR) in terms of cities and countries, focusing on the measurement of the "spatial" aspect in citation networks. Our main goal is to solve the citation bias caused by different geographical locations of citations. We empirically investigate spatial properties of citing distances, citation patterns and spatial diversity to understand geographical knowledge diffusion, based on the data from "Transportation Science and Technology" subject category in the Web of Science (1966---2009). We also compare the proposed ranking method with other bibliometric measures, and conduct a case study to figure out the recent ranks of the well-established authors in Transportation research. It is found that the SDCR of a focal author is highly correlated with the sum of spatial diversity weights ("strength") of all his in-links, and it is better to set the damping factors smaller than 0.75 when ranking authors with various initial academic years by SDCR. The cases show that Hong Kong is becoming a cluster in Transportation research.

A Reverse Engineering Approach to the Suppression of Citation Biases Reveals Universal Properties of Citation Distributions

The large amount of information contained in bibliographic databases has recently boosted the use of citations, and other indicators based on citation numbers, as tools for the quantitative assessment of scientific research. Citations counts are often interpreted as proxies for the scientific influence of papers, journals, scholars, and institutions. However, a rigorous and scientifically grounded methodology for a correct use of citation counts is still missing. In particular, cross-disciplinary comparisons in terms of raw citation counts systematically favors scientific disciplines with higher citation and publication rates. Here we perform an exhaustive study of the citation patterns of millions of papers, and derive a simple transformation of citation counts able to suppress the disproportionate citation counts among scientific domains. We find that the transformation is well described by a power-law function, and that the parameter values of the transformation are typical features of each scientific discipline. Universal properties of citation patterns descend therefore from the fact that citation distributions for papers in a specific field are all part of the same family of univariate distributions.

Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor

The journal impact factor (JIF) proposed by Garfield in the year 1955 is one of the most prominent and common measures of the prestige, position, and importance of a scientific journal. The JIF may profit from its comprehensibility, robustness, methodological reproducibility, simplicity, and rapid availability, but it is at the expense of serious technical and methodological flaws. The paper discusses two core problems with the JIF: first, citations of documents are generally not normally distributed, and, furthermore, the distribution is affected by outliers, which has serious consequences for the use of the mean value in the JIF calculation. Second, the JIF is affected by bias factors that have nothing to do with the prestige or quality of a journal (e.g., document type). For solving these two problems, we suggest using McCall's area transformation and the Rubin Causal Model. Citation data for documents of all journals in the ISI Subject Category “Psychology, Mathematical” (Journal Citation Report) are used to illustrate the proposal.

Findings and implications from data mining the IMC review process

The computer science research paper review process is largely human and time-intensive. More worrisome, review processes are frequently questioned, and often non-transparent. This work advocates applying computer science methods and tools to the computer science review process. As an initial exploration, we data mine the submissions, bids, reviews, and decisions from a recent top-tier computer networking conference. We empirically test several common hypotheses, including the existence of readability, citation, call-for-paper adherence, and topical bias. From our findings, we hypothesize review process methods to improve fairness, efficiency, and transparency.

Testing the fairness of citation indicators for comparison across scientific domains: The case of fractional citation counts

Citation numbers are extensively used for assessing the quality of scientific research. The use of raw citation counts is generally misleading, especially when applied to cross-disciplinary comparisons, since the average number of citations received is strongly dependent on the scientific discipline of reference of the paper. Measuring and eliminating biases in citation patterns is crucial for a fair use of citation numbers. Several numerical indicators have been introduced with this aim, but so far a specific statistical test for estimating the fairness of these numerical indicators has not been developed. Here we present a statistical method aimed at estimating the effectiveness of numerical indicators in the suppression of citation biases. The method is simple to implement and can be easily generalized to various scenarios. As a practical example we test, in a controlled case, the fairness of fractional citation count, which has been recently proposed as a tool for cross-discipline comparison. We show that this indicator is not able to remove biases in citation patterns and performs much worse than the rescaling of citation counts with average values.

Taxonomic Chauvinism Revisited: Insight from Parental Care Research

Parental care (any non-genetic contribution by a parent that appears likely to increase the fitness of its offspring) is a widespread trait exhibited by a broad range of animal taxa. In addition to influencing the fitness of parent(s) and offspring, parental care may be inextricably involved in other evolutionary processes, such as sexual selection and the evolution of endothermy. Yet, recent work has demonstrated that bias related to taxonomy is prevalent across many biological disciplines, and research in parental care may be similarly burdened. Thus, I used parental care articles published in six leading journals of fundamental behavioral sciences (Animal Behaviour, Behavioral Ecology, Behavioral Ecology and Sociobiology, Ethology, Hormones and Behavior, and Physiology & Behavior) from 2001–2010 (n = 712) to examine the year-to-year dynamics of two types of bias related to taxonomy across animals: (1) taxonomic bias, which exists when research output is not proportional to the frequency of organisms in nature, and (2) taxonomic citation bias, which is a proxy for the breadth of a given article—specifically, the proportion of articles cited that refer solely to the studied taxon. I demonstrate that research on birds likely represents a disproportionate amount of parental care research and, thus, exhibits taxonomic bias. Parental care research on birds and mammals also refers to a relatively narrow range of taxonomic groups when discussing its context and, thus, exhibits taxonomic citation bias. Further, the levels of taxonomic bias and taxonomic citation bias have not declined over the past decade despite cautionary messages about similar bias in related disciplines— in fact, taxonomic bias may have increased. As in Bonnet et al. (2002), my results should not be interpreted as evidence of an ‘ornithological Mafia’ conspiring to suppress other taxonomic groups. Rather, I generate several rational hypotheses to determine why bias persists and to guide future work.

Triadic citations, country biases and patent value: the case of pharmaceuticals

Triadic patents minimise home bias effects in studies that focus on patent counts as a measure of innovative activity. Yet, biases in qualitative patent indicators have been largely neglected. This article advocates that forward patent citations, and triadic citations in particular, can illuminate further on home bias, self citations, and the speed of knowledge flows for drug patents published by the USPTO for the period 1980---2008. The evidence shows that triadic citations help to minimize the home bias in citations as well as to make patent quality more transparent. Also, it indicates that self citations and the age distribution of citations are important factors to consider when explaining cross-country differences in pharmaceutical citations.

The Hirsch index: more than a number?

The Hirsch index: more than a number?

Copper Status in Alzheimer′s Disease and Other Neurodegenerative Disorders: Genetics, Mechanisms, Neurophysiology, and Therapies

The present special issue is an attempt to explore some of the diverse aspects of copper involvement in the physiology of the brain. The contributions examine very hot topics, such as the relationship between copper and synaptic transmission or copper involvement in Alzheimer's disease (AD) and even include a computational model of copper binding to the amyloid precursor protein (S. Azini and R. Rouk 2011). In this model, the authors applied theoretical methods of computational chemistry to determine structure, binding affinities, and reduction potentials of Cu(II) and Cu(I) bound to models of the Asp1, Ala2, His6, and His13His14 regions of the amyloid beta peptide. A bulk of new studies has been recently published on the role of copper in the biological processes subtending synaptic transmission, and three contributions on this subject are included in the special issue. Specifically, one contribution describes a study in a cellular model of oocytes, which investigated the connection between copper and the prion protein in neurotransmission mediated by ATP-evoked currents (R. A. Lorca et al., 2011). A study on mouse hippocampal slices investigates copper inhibition of glutamatergic NMDA receptor-independent long-term potentiation (N. L. Salazar-Weber and J. P. Smith 2011), and a human model explores the effects of systemic copper on glutamatergic mediated excitability in the primary somatosensory cortex (F. Tecchio et al., 2011). Since the original papers that have been fuelling a heated debate in the literature about the pros and cons of copper on cognition are rather scattered and discordant, the aim of the editors was to select contributions that all together would provide the reader with an adequate overview of the issue but would still leave the reader free to form personal and hopefully critical opinions. In this section, some of the papers give a good overview of the nonunivocal in vitro, animal and human findings which demonstrate the involvement of copper in AD (D. Kaden et al. 2011). Also, in their paper, Y. Manso and colleagues (Y. Manso et al., 2011) review the preclinical and clinical literature dedicated to therapeutic interventions aimed at delaying or possibly reverting AD progression via the modulation of brain copper levels (Y. Manso et al., 2011). In the same section, some reports are preliminary but still original in their content, as, for example, the paper dealing with the influence of water quality on the cholesterol-fed rabbit model, which suggests that the origin of the brain neurofibrillary tangles is possibly systemic and somehow related to disturbances to the blood brain barrier mediated by copper ingested through the diet (D. L. Sparks et al., 2011), as well as the study by S. Bucossi and coworkers (S. Bucossi et al., 2011) which demonstrates an association between single nucleotide polymorphisms of the Wilson's disease gene and AD. T. U. Hoogenraad discusses further the link between these two neurological disorders (T. U. Hoogenraad 2011), which share the presence of abnormalities in toxic nonceruloplasmin (also named “free”) copper, proposing Zinc therapy as a valid tool against AD, as it has already proven to be in Wilson's disease. The perspective that comes out of his contribution (T. U. Hoogenraad 2011), for some maybe visionary or prophetic, is probably somehow adherent to the author's attitude, pragmatic, as can be expected from a physician who has dedicated his career to Wilson's disease clinical care, but also filled with enthusiasm, as demonstrated by his attitude to be prone to ideas of shifting paradigms: he was one of the pioneer of Zinc therapy in Wilson's disease. With a totally different approach, G. J. Brewer (G. J. Brewer 2011) discusses AD from an epidemiologic point of view, advancing the hypothesis that ingestion of inorganic copper from copper-made plumbing or via consumption of copper-rich dietary supplements may have a role in AD generation. Finally, the issue presents also a study connecting higher levels of copper to gender differences and cognitive status (J. F. Quinn et al., 2011). In conclusion, is copper good or bad in general for human health and in particular for cognition? Of course, the question is far from being answered in this special issue, but it is well discussed. These editors recommend reading carefully the diverse interpretations presented but also going back and read the original works which are at the basis of them. In fact, even though reliable, review articles often decrease the critical attitude of the reader, as recent publications about citation bias have pointed out [1]. Rosanna Squitti D. Larry Sparks Tjaard U. Hoogenraad George J. Brewer

Media Bias, Public Opinion, and Policy Liberalism from 1956 to 2004: A Second-Level Agenda-Setting Study

This study aims to explore second-level agenda-setting at the national level. In particular, it examines the relationships among the citation bias of the New York Times, national level public opinion, and Congressional policies from 1956 to 2004 in order to better understand mass media's role in national policymaking. In addition, it also tests one important intervening variable of the relationships among the three attribute agendas (the media agenda, the public agenda, and the policy agenda)—the president's policy liberalism.

Scientometric Analysis and Bibliography of Digit Ratio (2D:4D) Research, 1998–2008

A scientometric analysis of modern research on the second-to-fourth digit ratio (2D:4D), a widely studied putative marker for prenatal androgen action, is presented. In early 2009, this literature totalled more than 300 publications and, since its initiation in 1998, has grown at a rate slightly faster than linear. Key findings included evidence of publication bias and citation bias, incomplete coverage and outdatedness of existing reviews, and a dearth of meta-analyses in this field. 2D:4D research clusters noticeably in terms of researchers, institutions, countries, and journals involved. Although 2D:4D is an anthropometric trait, most of the research has been conducted at psychology departments, not anthropology departments. However, 2D:4D research has not been predominantly published in core and specialized journals of psychology, but rather in more broadly scoped journals of the behavioral sciences, biomedical social sciences, and neurosciences. Total citation numbers of 2D:4D papers for the most part were not larger than their citation counts within 2D:4D research, indicating that until now, only a few 2D:4D studies have attained broader interest outside this specific field. Comparative citation analyses show that 2D:4D research presently is commensurate in size and importance to evolutionary psychological jealousy research, but has grown faster than the latter field. In contrast, it is much smaller and has spread more slowly than research about the Implicit Association Test Fifteen conjectures about anticipated trends in 2D:4D research are outlined, appendixed by a first-time bibliography of the entirety of the published 2D:4D literature.