Citation Bias Research Articles

The Citation Diversity Statement: A Practice of Transparency, A Way of Life.

Appending a Citation Diversity Statement to a paper is a simple and effective way to increase awareness about citation bias and help mitigate it. Here, we describe why reducing citation bias is important and how to include a Citation Diversity Statement in your next publication.

Insight into citation bias.

Insight into citation bias.

EPID-32. IS THERE ANY EVIDENCE OF REPORTING AND CITATION BIAS AMONG COMPLETED PHASE III INTERVENTIONAL BRAIN TUMOR TRIALS?

Abstract OBJECTIVE Reporting and citation bias based on positive versus negative outcome among completed phase III interventional brain tumor trials (IBTTs) have not been previously reported. Thus, we aimed to assess the evidence. METHODS Clinicaltrials.gov was used to obtain all phase III IBTTs completed prior to December 31st 2016. Trials closed due to poor accrual or non-phase III were excluded. Data about the funding source, type of intervention, conducted at U.S or Non-U.S locations, patients enrolled, primary completion date, time to dissemination of results in months (either reported on clinicaltrials.gov or published in a journal), citations per year (from web of science) and outcome (positive or neutral/negative) were abstracted. Median time to dissemination was estimated using Kaplan-Meier estimates and a log rank test for statistical significance. The number of citations per year for positive and negative/neutral studies was compared using a t-test. RESULTS 65 studies were analyzed. The median time from primary completion to dissemination of results for trials with a positive outcome was 20 months (n = 20; 95% CI: 12–31) compared to 30 months for trials with a negative outcome (n=35; 95 % CI: 22 – 37) (p = 0.095). The average number of citations per year for trials with a positive outcome is 62.4 (n = 21; range 1.1 – 614.8) compared to 25.0 for trials with a negative outcome (n=35; range 0.14 - 158.8) (p = 0.213). There was no significant difference in time to dissemination based on the funding source, type of intervention or the location where the trial was conducted. CONCLUSION Positive phase III IBTTs results are disseminated earlier and are cited more frequently compared to negative trials. However, there is no statistically significant evidence of reporting or citation bias.

No evidence of citation bias as a determinant of STEM gender disparities in US biochemistry, genetics and molecular biology research

The lack of females in many Science Technology, Engineering and Mathematics (STEM) subjects in the USA is an ongoing concern, with many initiatives attempting to redress this imbalance. Some life sciences are apparently areas of relatively good practice, with higher proportions of female researchers than most other STEM subjects. This paper assesses gender differences in research contributions to 14 biochemistry, genetics or molecular biology specialisms in the USA 1996–2014/8, seeking evidence of trends in publishing and citation impact that may give insights into female success. With four exceptions (biochemistry, biophysics, biotechnology, and structural biology), the fields achieved or maintained at least 40% female first authors by 2018, with developmental biology and endocrinology both attaining female first author majorities. A regression analysis found close to gender parity overall in citation impact but a small male first author citation advantage in more fields than the opposite: an up to 3% increase in logged citation ratio to the world mean. This was partly due to males first authoring with larger teams. Fields with relatively many females did not favour female-led research with more citations, however.

Affirmative citation bias in scientific myth debunking: Athree-in-one case study.

Several uncorroborated, false, or misinterpreted conceptions have for years been widely distributed in academic publications, thus becoming scientific myths. How can such misconceptions persist and proliferate within the inimical environment of academic criticism? Examining 613 articles we demonstrate that the reception of three myth-exposing publications is skewed by an ‘affirmative citation bias’: The vast majority of articles citing the critical article will affirm the idea criticized. 468 affirmed the myth, 105 were neutral, while 40 took a negative stance. Once misconceptions proliferate wide and long enough, criticizing them not only becomes increasingly difficult, efforts may even contribute to the continued spreading of the myths.

Consensus and dissensus in comparative politics: Do comparativists agree on the goals, methods, and results of the field?

Are comparative political scientists divided over the goals, methods, and results of their field? This article attempts to answer this question, drawing on an original survey of US-based political scientists. The main conclusion is that there is relative consensus on the goals of research—comparativists favor broad generalizations and causal inference—but there is also acceptance of a variety of methodological approaches, both qualitative and quantitative, in pursuing this goal. Comparativists, however, show less agreement on substantive findings in the areas of democracy and democratic politics, economy and society, and political institutions. Interestingly, generational differences are relatively infrequent, but gender differences on issues such as rational choice and causal inference are more prominent, possibly contributing to gendered citation bias. The findings suggest that comparative politics may not have accumulated a large amount of agreed-upon knowledge, but that there is substantial agreement on the path forward.

Widespread Biases in Ecological and Evolutionary Studies

Abstract There has been widespread discussion of biases in the sciences. The extent of most forms of bias has scarcely been confronted with rigorous data. In the present article, we evaluated the potential for geographic, taxonomic, and citation biases in publications between temperate and tropical systems for nine broad topics in ecology and evolutionary biology. Across 1,800 papers sampled from 60,000 peer-reviewed, empirical studies, we found consistent patterns of bias in the form of increased numbers of studies in temperate systems. Tropical studies were nearly absent from some topics. Furthermore, there were strong taxonomic biases across topics and geographic regions, as well as evidence for citation biases in many topics. Our results indicate a strong geographic imbalance in publishing patterns and among different taxonomic groups across a wide range of topics. The task ahead is to address what these biases mean and how they influence the state of our knowledge in ecology and evolution.

The geography of scientific citations

Science’s main norms prescribe scientists to use citations as acknowledgements of cognitive content irrespective of geographical location. Previous studies, however, suggested that there is a considerable geographical bias in scientific citations. We argue that this geographical bias does not, in itself, falsify the notion that citations reflect acknowledgement of cognitive content, because cognitively related knowledge may be geographically concentrated as well. We analyse the role of organizational, regional and national co-location on citation likelihood for 5.5 million article pairs, and find that the geographical bias in citations is weak once cognitive relatedness is accounted for. Furthermore, we find that the effect of co-location on citation likelihood is strongest at the organizational level, weaker at the regional level, and weakest at the national level. In addition, we show that geographical co-location particularly increases the citation likelihood between two papers when knowledge relatedness between articles is low, suggesting that interdisciplinary research benefits most from co-location. Finally, we find that, when knowledge relatedness is high, the effect of geographical co-location on citation likelihood is non-existent. We discuss the implications regarding policies aimed to discourage strategic citations and to foster interdisciplinary research.

Selective citation in the literature on the hygiene hypothesis: a citation analysis on the association between infections and rhinitis

ObjectiveOur objective was to assess the occurrence and determinants of selective citation in scientific publications on Strachan’s original hygiene hypothesis. His hypothesis states that lack of exposure to infections in...

Citation bias in imaging research: are studies with higher diagnostic accuracy estimates cited more often?

To assess the risk of citation bias in imaging diagnostic accuracy research by evaluating whether studies with higher accuracy estimates are cited more frequently than those with lower accuracy estimates. We searched Medline for diagnostic accuracy meta-analyses published in imaging journals from January 2005 to April 2016. Primary studies from the meta-analyses were screened; those assessing the diagnostic accuracy of an imaging test and reporting sensitivity and specificity were eligible for inclusion. Studies not indexed in Web of Science, duplicates, and inaccessible articles were excluded. Topic (modality/subspecialty), study design, sample size, journal impact factor, publication date, times cited, sensitivity, and specificity were extracted for each study. Negative binomial regression was performed to evaluate the association of citation rate (times cited per month since publication) with Youden's index (sensitivity + specificity -1), highest sensitivity, and highest specificity, controlling for the potential confounding effects of modality, subspecialty, impact factor, study design, sample size, and source meta-analysis. There were 1016 primary studies included. A positive association between Youden's index and citation rate was present, with a regression coefficient of 0.33 (p = 0.016). The regression coefficient for sensitivity was 0.41 (p = 0.034), and for specificity, 0.32 (p = 0.15). A positive association exists between diagnostic accuracy estimates and citation rates, indicating that there is evidence of citation bias in imaging diagnostic accuracy literature. Overestimation of imaging test accuracy may contribute to patient harm from incorrect interpretation of test results. • Studies with higher accuracy estimates may be cited more frequently than those with lower accuracy estimates. • This citation bias could lead clinicians, reviews, and clinical practice guidelines to overestimate the accuracy of imaging tests, contributing to patient harm from incorrect interpretation of test results.

Citation bias in the literature on dietary trans fatty acids and serum cholesterol

ObjectiveBalanced citations are a necessary condition for a sound development of scientific knowledge, whereas selective citations may bias scientific consensus. In this study, we assess which determinants influenced the likelihood of being cited in the literature on trans fatty acids and cholesterol. Study Design and SettingWe conducted a citation network analysis of the literature concerning trans fats and low density cholesterol and high density cholesterol. Each publication was scored on various potential determinants of citation, such as study outcome, study design, sample size, journal impact factor, and funding source. We applied random effect logistic regression to identify determinants of citation. ResultsA network of 108 publications was identified, containing 5,041 potential citation paths and 669 utilized citation paths. Reporting statistically significant results was found to be a strong predictor of citation, together with sample size, journal impact factor, and the authority of the authors. ConclusionWithin the literature on trans fat intake and cholesterol, selective citations are based on several grounds. Especially the effect of reporting significant results on citation requires special attention because disproportionate attention is paid to publications suggesting a harmful effect of trans fat on cholesterol.

Order matters: Alphabetizing in-text citations biases citation rates.

Though citations are critical for communicating science and evaluating scholarly success, properties unrelated to the quality of the work-such as cognitive biases-can influence citation decisions. The primacy effect, in particular, is relevant to lists, which for in-text citations could result in citations earlier in the list receiving more attention than those later in the list. Therefore, how citations are ordered could influence which citations receive the most attention. Using a sample of 150,000 articles, we tested whether alphabetizing in-text citations biases readers into citing more often articles with first authors whose surnames begin with letters early in the alphabet. We found that surnames earlier in the alphabet were cited more often than those later in the alphabet when journals ordered citations alphabetically compared with chronologically or numerically. This effect seemed to be stronger in psychology journals (which have a culture of alphabetizing citations) compared with biology or geoscience journals (which primarily order chronologically or numerically) and was strongest among moderately and highly cited articles. Therefore, alphabetizing in-text citations biases citation decisions toward authors with surnames occurring early in the alphabet. These citation decisions result from an interaction between cognitive biases (more attention devoted to items earlier in a list) and the structure of the citation environment (the style in which citations are ordered). We suggest that journals using alphabetically ordered citations switch to chronological ordering to minimize this arbitrary alphabetical citation bias.

General introduction and outline of the special issue "Emerging advances and challenges in pesticide ecotoxicology".

The aim of this special issue was to bring together papers dealing with current challenges and trends in pesticide ecotoxicology. The special issue is composed of 25 papers into various topics, ranging from the development of more ecological ecotoxicological test approaches to an evaluation of citation bias in pesticide ecotoxicology literature. This paper provides a general introduction and an index of the various papers in the special issue. This special issue is trusted to provide a valuable contribution in addressing current challenges in pesticide ecotoxicology.

A network analysis of the propagation of evidence regarding the effectiveness of fat-controlled diets in the secondary prevention of coronary heart disease (CHD): Selective citation in reviews.

ObjectiveTo examine how the first randomised controlled trials (RCTs) evaluating the efficacy of cholesterol-lowering diets in the secondary prevention of coronary heart disease were interpreted in reviews of the literature prior to the National Institutes of Health consensus conference in 1984.DesignClaim-specific citation network analysis was used to study the network of citations between reviews and RCTs over a defined period (1969–1984). RCTs were identified and classified according to whether their conclusions supported or opposed the use of dietary fat modification/restriction in the secondary prevention of coronary heart disease. Each review published in this period that cited any of the RCTs was classified as supportive, neutral, or unsupportive to the use of dietary fat modification based on a quotation analysis of its evaluation of the findings of these RCTs. Citation bias and underutilisation were detected by applying a comparative density measure, in-degree centrality, and out-degree in a series of sub-graph analyses.ResultsIn total, 66 unique publications were identified (four RCTs—one supportive, three unsupportive; 62 reviews—28 supportive, 17 neutral, 17 unsupportive). On average, supportive reviews underutilised the available RCTs to a greater degree than other reviews. Amongst the supportive group, citation bias was common—23 (82%) reviews cited only the one RCT that was supportive.ConclusionMost reviews that disseminated a supportive evaluation of the results of RCTs in the context of secondary prevention cited only data that supported this position.

Measuring discursive influence across scholarship

Assessing scholarly influence is critical for understanding the collective system of scholarship and the history of academic inquiry. Influence is multifaceted, and citations reveal only part of it. Citation counts exhibit preferential attachment and follow a rigid "news cycle" that can miss sustained and indirect forms of influence. Building on dynamic topic models that track distributional shifts in discourse over time, we introduce a variant that incorporates features, such as authorship, affiliation, and publication venue, to assess how these contexts interact with content to shape future scholarship. We perform in-depth analyses on collections of physics research (500,000 abstracts; 102 years) and scholarship generally (JSTOR repository: 2 million full-text articles; 130 years). Our measure of document influence helps predict citations and shows how outcomes, such as winning a Nobel Prize or affiliation with a highly ranked institution, boost influence. Analysis of citations alongside discursive influence reveals that citations tend to credit authors who persist in their fields over time and discount credit for works that are influential over many topics or are "ahead of their time." In this way, our measures provide a way to acknowledge diverse contributions that take longer and travel farther to achieve scholarly appreciation, enabling us to correct citation biases and enhance sensitivity to the full spectrum of scholarly impact.

Evidence of citation bias in the pesticide ecotoxicology literature.

As scientists, we are tasked with letting evidence guide our conclusions. In the world of pesticides this takes on added importance as the data can influence ecological and human health outcomes and regulations, and even the manner in which we grow food. Yet, there seems to be a reticence to engage with the totality of the pesticide ecotoxicology literature, especially papers that report few or no effects or low risk to non-target organisms. We suspected that these studies would have fewer citations than studies that report significant effects or risk for the same compound, and this would be unrelated to the strength of the study, e.g., high quality studies with few or no effects would be cited less frequently than weaker studies that reported effects. To investigate this, we examined a subset of literature around the herbicide atrazine. We found that papers reporting an effect had significantly more citations per year than those that did not (p < 0.05). There was no significant relationship between the strength of the study and number of citations, but a general trend for weaker studies to have greater number of citations. The impact factor of journals was not positively correlated with the strength of the study methods, but studies that reported effects were published in journals with a greater mean impact factor than those that reported no effects (p < 0.05). This analysis reveals evidence of citation bias within the pesticide ecotoxicology literature, as well as bias by journals to publish studies that report effects, regardless of study quality.

Being influential or being misleading? Citation bias in psychiatric research and practice.

Being influential or being misleading? Citation bias in psychiatric research and practice.

Risk Evaluation Requires an Independent Mind.

Biomedical research pertaining to pathologies observed in adolescents can involve areas where patients can expect no immediate benefits. Here, Congress stipulates that local review boards are restricted to approving procedures posing no greater than minimal risk (45 CFR 46.404). An evaluation of risk embraces the current state of the art with regard to the safety and efficacy of procedures. A tendency of biomedical scholars to cite highly cited documents can introduce a bias in an argumentation in favor or against a given recommendation in the context that bias in citations can be correlated with an imprudent use of funds for research. We use choice examples to highlight the necessity of approaching any scholarly task with an independent mind.

Selective citation in the literature on swimming in chlorinated water and childhood asthma: a network analysis

BackgroundKnowledge development depends on an unbiased representation of the available evidence. Selective citation may distort this representation. Recently, some controversy emerged regarding the possible impact of swimming on childhood asthma, raising the question about the role of selective citation in this field. Our objective was to assess the occurrence and determinants of selective citation in scientific publications on the relationship between swimming in chlorinated pools and childhood asthma.MethodsWe identified scientific journal articles on this relationship via a systematic literature search. The following factors were taken into account: study outcome (authors’ conclusion, data-based conclusion), other content-related article characteristics (article type, sample size, research quality, specificity), content-unrelated article characteristics (language, publication title, funding source, number of authors, number of affiliations, number of references, journal impact factor), author characteristics (gender, country, affiliation), and citation characteristics (time to citation, authority, self-citation). To assess the impact of these factors on citation, we performed a series of univariate and adjusted random-effects logistic regressions, with potential citation path as unit of analysis.ResultsThirty-six articles were identified in this network, consisting of 570 potential citation paths of which 191 (34%) were realized. There was strong evidence that articles with at least one author in common, cited each other more often than articles that had no common authors (odds ratio (OR) 5.2, 95% confidence interval (CI) 3.1–8.8). Similarly, the chance of being cited was higher for articles that were empirical rather than narrative (OR 4.2, CI 2.6–6.7), that reported a large sample size (OR 5.8, CI 2.9–11.6), and that were written by authors with a high authority within the network (OR 4.1, CI 2.1–8.0). Further, there was some evidence for citation bias: articles that confirmed the relation between swimming and asthma were cited more often (OR 1.8, CI 1.1–2.9), but this finding was not robust.ConclusionsThere is clear evidence of selective citation in this research field, but the evidence for citation bias is not very strong.

Taxonomic bias in animal behaviour publications

Evidence suggests that certain taxonomic groups are more thoroughly studied than others across a wide range of biological disciplines. Such taxonomic biases have the potential to define our understanding of theory, and limit the generality of our insights. To assess the distribution of taxonomic representation in current and historical animal behaviour research, we constructed a data set containing article metrics and taxonomic information for all research articles published in the journal Animal Behaviour between 1953 and 2015. We found significant taxonomic bias, with chordate papers making up 70% of all publications in the past 15 years, despite accounting for less than 7% of all animal species. Within chordates, Animal Behaviour content is biased towards endotherms, with birds and mammals comprising more than 50% of all publications. In sum, six animal orders account for more than half of all publications, with the most commonly published order, Passeriformes, representing one in five articles. Our findings confirm that a relatively narrow group of ‘model’ taxa represent the vast majority of articles, and may have a disproportionate influence on our understanding of behavioural patterns and processes. Furthermore, we find evidence of a citation bias, with chordate studies receiving on average four citations more per paper than arthropod studies. While historical trends suggest that the publication gap between arthropods and chordates has been shrinking for the past 45 years, our findings show that a considerable bias still remains. These biases may originate from human preferences for certain animal types, but we argue that they are likely maintained by a mixture of taxonomic prejudices, cultural aspects of behavioural ecology as a field, and of academia in general. We suggest that the patterns are clear and their implications serious, and that it is time that both researchers and journals give serious consideration to addressing them.