Citation Content Research Articles

Citation Content Analysis of Master Degree Theses in Library and Information Technology for Improved Research

Citation Content Analysis of Master Degree Theses in Library and Information Technology for Improved Research

Assessing citation integrity in biomedical publications: corpus annotation and NLP models.

Citations have a fundamental role in scholarly communication and assessment. Citation accuracy and transparency is crucial for the integrity of scientific evidence. In this work, we focus on quotation errors, errors in citation content that can distort the scientific evidence and that are hard to detect for humans. We construct a corpus and propose natural language processing (NLP) methods to identify such errors in biomedical publications. We manually annotated 100 highly-cited biomedical publications (reference articles) and citations to them. The annotation involved labeling citation context in the citing article, relevant evidence sentences in the reference article, and the accuracy of the citation. A total of 3063 citation instances were annotated (39.18% with accuracy errors). For NLP, we combined a sentence retriever with a fine-tuned claim verification model to label citations as ACCURATE, NOT_ACCURATE, or IRRELEVANT. We also explored few-shot in-context learning with generative large language models. The best performing model-which uses citation sentences as citation context, the BM25 model with MonoT5 reranker for retrieving top-20 sentences, and a fine-tuned MultiVerS model for accuracy label classification-yielded 0.59 micro-F1 and 0.52 macro-F1 score. GPT-4 in-context learning performed better in identifying accurate citations, but it lagged for erroneous citations (0.65 micro-F1, 0.45 macro-F1). Citation quotation errors are often subtle, and it is currently challenging for NLP models to identify erroneous citations. With further improvements, the models could serve to improve citation quality and accuracy. We make the corpus and the best-performing NLP model publicly available at https://github.com/ScienceNLP-Lab/Citation-Integrity/.

Transdisciplinary fine-grained citation content analysis: A multi-task learning perspective for citation aspect and sentiment classification

The diffusion of citation knowledge is an important measure of transdisciplinary scientific impact and the diversity of transdisciplinary citation content (sentences). Moreover, combining citation sentiment (CS) and citation aspect (CA) can help researchers identify the attitudes, ideas, or positions reflected in the evolution of scientific elements (e.g., theories, techniques, and methods). This is because of their use by scholars from different disciplines, paving the way toward transdisciplinary penetration and the development of domain knowledge through the proliferation of cited knowledge. However, most studies mainly address citation aspect classification (CAC) and citation sentiment classification (CSC) separately, ignoring their shared features of interactions. In this study, we construct a dataset for transdisciplinary citation content analysis using citations and academic full texts from the Chinese Social Sciences Citation Index (CSSCI), which includes 14,832 manually-annotated citations. Thereafter, we utilized the developed dataset to conduct a transdisciplinary fine-grained citation content analysis by combining CAC and CSC. The objective of the CAC task was to classify transdisciplinary citations into theoretical concepts (TC), methodological techniques (MT), and data information (DI), whereas the CSC task classified citations into positive, negative, and neutral classes. Furthermore, we leveraged a multi-task learning (MTL) model to perform CAC and CSC jointly and then compared its performance to those of several widely-used deep learning models. Our model achieved 83.10 % accuracy for CAC and 80.46 % accuracy for CSC, demonstrating its superiority to single-task systems. This indicates the strong correlation between the CAC and CSC of transdisciplinary citation tasks, benefiting from each other when learned concurrently. This new method can be used as an auxiliary decision support system to extend the analysis of transdisciplinary citation content.

Citation content in literature review sections of research articles: A cross-paradigm comparison of design science and interpretivist research in information systems

Despite a long history of citation research, its focus has been on the linguistic and rhetorical characteristics of citations rather than their semantic nature. Using a move-based approach grounded in the CARS model (Swales, 1990), this paper studies the sorts of source ideas cited in different parts of literature reviews (LR) of Information Systems research articles following the design science research (DSR) and interpretivist research (IR) paradigms. Findings reveal three types of epistemically-oriented semantic content – theoretical, research, and methodological – and one type that is non-epistemically-framed, which vary quantitatively and qualitatively between the DSR and IR LRs. For example, both types of LRs cite more often in Move 1 Establishing a territory than in Moves 2 and 3 Establishing a niche and Occupying the niche, with theoretical terminologies, definitions, propositions being referred to more frequently in the IR writing than in the DSR writing. On the other hand, citations in the DSR writing often display prescriptive propositions, design concepts, artefacts, and methods which are absent from the IR writing. These findings not only help us better understand how citation is shaped and constrained by rhetorical moves and research paradigms but also provide implications for the teaching of citation content.

Opening research data contributes to the citations of related research articles: Evidence from Data in Brief

AbstractThis paper explores the effect of publishing a data paper in the Open Access journal Data in Brief (DIB) on the citation counts of the related research paper. Using regression analysis, citation content analysis and a survey method, we investigate whether research papers with a related data paper have higher citation counts and the potential reasons. After controlling variables that correlate with the citation counts, research papers with a related data paper were found to have higher citation counts than those published in the same issue of the same journal. Next, we explored the causal relationship between the two variables by surveying the corresponding authors of 618 papers who shared datasets in DIB from 2014 to 2021. The results show that the authors acknowledge the benefits of sharing data in DIB, including citation increase and career reputation enhancement. We further explored how the data papers in DIB increase the citations of the related research papers by using citation content analysis. We found that scientists co‐cite the data papers and their related research papers for the purpose of reusing the underlying data or portraying a better understanding of the underlying data and related research articles.

Uncovering the use and impact of mechanical weed control articles: A citation content analysis

AbstractThis study investigated the extent to which seven papers on mechanical weed control have been cited, understood and utilised in subsequent publications. Web of Science was used to identify the citing publications, and citation content analysis was conducted to investigate the cognitive links between the citing and cited publications. Cognition involves acquiring, understanding, and using knowledge. Citation of the seven publications in 305 publications was classified, and it was found that perfunctory citations (those that were routinely referenced, with little effort to understand or use content) accounted for 53% of all citations and 16% of the citing articles included citations that were not supported by the references. The most striking finding was that key content was rarely used in articles, despite being referenced in 42% of the published articles. It is recommended that more time be allocated by authors to understanding literature, as this would appear to be a matter of diminishing concern for the scientific community. For those who assume their research area has a better citation practice than found in this study, it is recommended that authors conduct a citation content analysis within their own research area to increase the focus on good literature practices.

Revisiting the Do-Live-Well Health Promotion Framework: A Citation Content Analysis.

Background. The Do-Live-Well (DLW) framework was first published in 2015 and aimed to fill a theoretical gap in the health promotion literature related to the links between occupational patterns and health. However, the extent of uptake and use of the framework since publication is unknown. Purpose. To explore and reflect on the adoption and application of DLW in the literature. Method. Citation content analysis of two seminal DLW publications was conducted from 2015 to November 2022 across six databases. Findings. Seventeen citations directly applied DLW to inform research (n = 10), practice (n = 5) and knowledge translation (n = 2). Implications. The findings highlight uptake of the framework in a range of settings, and how it can inform an occupation-based understanding of health and well-being. Ongoing knowledge dissemination, development of practice tools, and research to update evidence and examine relevance are needed to further advance the utility and application of the framework.

Systematic review and citation content analysis of the CHIME framework for mental health recovery processes: recommendations for developing influential conceptual frameworks

To identify design features of the CHIME conceptual framework of mental health recovery which are associated with high rates of citation. Systematic review of all citations of the Connectedness, Hope, Identity, Meaning, and Empowerment (CHIME) framework of mental health recovery. Papers citing CHIME were screened and extracted from three citation databases. Citation content analysis was used to investigate associations between nine CHIME design features. Citations were investigated across six forms of visibility: all citations; Anglophone vs non-Anglophone; academic vs non-academic; academic discipline; professional group; and clinical population. There were 915 eligible documents identified. Six CHIME framework design features met predefined thresholds for high levels of influence: (i) using a systematic review methodology for development, (ii) adopting a memorable acronym, (iii) having disaggregable components, and being unaligned to a (iv) particular discipline (i.e., transdisciplinary), (v) professional group, or (vi) diagnostic population. Documents from Anglophone countries were more likely to cite CHIME with reference to trans-professional (χ2=3.96, df=1, p=0.05) and ethnicity sub-group analysis (p=0.039) design features than non-Anglophone documents. Non-academic documents were more likely to cite the acronym design feature than academic papers (χ2=5.73, df=1, p=0.01). Public Health-related publications were more likely to cite CHIME within a trans-diagnostic framework (χ2=16.39, df=1, p<0.001) than other disciplines. The influence and impact of conceptual frameworks for recovery are increased when the framework is underpinned by a systematic review, includes disaggregable components which can be summarized using a memorable acronym, and when the framework is transdisciplinary, trans-professional, and trans-diagnostic.

Unvoicing a field's expertise: A two-pronged citation and language analysis

The field of Developmental Education (DE) draws from a distinct and multidisciplinary body of research and scholarship to facilitate students' transitions to college and to support their postsecondary academic success. However, highly cited scholars and policymakers external to the field perpetuate negative perceptions of DE, arguing for reform or elimination without inclusion of field experts. Through a combination of Citation Content Analysis and Transitivity Analysis, this study examined citation trends and verb transitivity to uncover the voices privileged as experts within an influential publication by the Community College Research Center and aimed to uncover how the authors (re)presented the DE field, literature, scholarship, its members, and its students. Findings revealed a single citation of an in-field DE scholar and repeated erasure of developmental educators and DE scholarship through passivation of DE professionals as social actors. Implications are discussed for the importance of including inside-field experts’ voices in discussions about their field as a necessary component of developing a critical praxis of integrating scholarship and practice in support of students.

Preprint citation practice in PLOS

The role of preprints in the scientific production and their part in citations have been growing over the past 10 years. In this paper we study preprint citations in several different aspects: the progression of preprint citations over time, their relative frequencies in relation to the IMRaD structure of articles, their distributions over time, per preprint database and per PLOS journal. We have processed the PLOS corpus that covers 7 journals and a total of about 240,000 articles up to January 2021, and produced a dataset of 8460 preprint citation contexts that cite 12 different preprint databases. Our results show that preprint citations are found with the highest frequency in the Method section of articles, though small variations exist with respect to journals. The PLOS Computational Biology journal stands out as it contains more than three times more preprint citations than any other PLOS journal. The relative parts of the different preprint databases are also examined. While ArXiv and bioRxiv are the most frequent citation sources, bioRxiv’s disciplinary nature can be observed as it is the source of more than 70% of preprint citations in PLOS Biology, PLOS Genetics and PLOS Pathogens. We have also compared the lexical content of preprint citation contexts to the citation content to peer-reviewed publications. Finally, by performing a lexicometric analysis, we have shown that preprint citation contexts differ significantly from citation contexts of peer-reviewed publications. This confirms that authors make use of different lexical content when citing preprints compared to the rest of citations.

Disciplinary and generic variation of citation use in research articles

For novice writers as well as EAP practitioners, citation use poses considerable challenges stemming from the writers’ limited understanding of disciplinary conventions, unwarranted source use, or rhetorical strategies. Although studies handling those challenges have reported typical citation functions in various disciplines, the explanation of functions in terms of content still confuses novice writers. Citation content per se is crucial to the specificity and reliability of the evidence presented. Drawing on a form-content-function integration, the present study examined the Introduction and the Results and Discussion sections in sixty research articles from physics, biology, education, and applied linguistics. We found that non-integral citations prevail across the four disciplines. We also found variations of citation contents and functions in the four disciplines and in part-genres. Specifying rhetorical functions and content in part-genres can thus enhance the evaluative power of source use.

Castells’s “Network Society Theory” to Human Geography —Analysis and Comparison Based on Chinese and Foreign Quotations

Based on 898 English documents and 363 Chinese documents citing the Rising of Network Society, it studied that the knowledge contribution of citation content analysis and citation context analysis methods, and the knowledge contribution of Chinese and foreign quotations to human geography. The study found that “mobile space” is the most quoted theoretical view in domestic and foreign literature, and the proportion of domestic research is significantly higher than foreign research; the focus of domestic and foreign research focuses on the external spatial form and its transformation, while foreign research pays more attention on the internal spatial dynamics of network society and three types of knowledge contributions, reflecting the influence of “network social theory” on human geography. Among them, critical references reveal the shortcomings of “network social theory” point out the abstraction of “spatial duality” the importance of local space, and the limitations of research data, methods, and time background, which provides new enlightenment for the future application and innovation of “network social theory” in the field of human geography.

Citation recommendation using semantic representation of cited papers’ relations and content

Citation recommendation can help researchers quickly find supplementary or alternative references in massive academic resources. Current research on citation recommendation mainly focuses on the citing papers, resulting in the enormous cited papers are ignored, including the relations among cited papers and their citation context cited in citing papers. Moreover, cited paper’s content is often denoted with its original title and abstract, which is hard to acquire and rarely considers different citation motivations. Furthermore, the most appropriate method for semantic representation of cited papers’ relations and content is uncertain. Therefore, this paper studies citation recommendation from the perspective of semantic representation of cited papers’ relations and content. Firstly, four forms of citation context are designed and extracted as cited papers’ content considering citation motivations, as well as co-citation relationships are extracted as cited papers’ relations. Secondly, 132 methods are designed for generating semantic vector of cited paper, including four network embedding methods, 16 methods by combining four text representation algorithms with four forms of citation content, and 112 fusion methods. Finally, similarity among cited papers is calculated for citation recommendation and a quantitative evaluation method based on link prediction is designed, to find the most appropriate form of citation content and the optimal method. The result shows that doc2vecC (Document to Vector through Corruption) with the form of CS&SS (Current Sentences and Surrounding Sentences) performs best, in which the AUC (Area Under Curve) and MAP (Macro Average Precision) reach 0.877 and 0.889 and have increased by 0.462 and 0.370 compared with the worst-performing method. This performance is slightly improved by parameters adjustment, and a case study is performed whose results have further proved the effectiveness of this method. In addition, among four forms of cited papers’ content, CS&SS performs best in almost all methods. Furthermore, the fusion methods not always perform better than the single methods, where doc2vecC (CS&SS) performs better than the best fusion method GCN (Graph Convolutional Network). These results not only prove the effectiveness of citation recommendation from the perspective of cited paper, but also provide helpful and useful suggestions for method selection and citation content selection. The data and conclusions can be extended to other text mining-related tasks. Simultaneously, it is a preliminary research which needs to be further studied in other domains using emerging semantic representation methods.

Referencing Made Easy: Reference Management Softwares

Reference management softwares are a powerful tool in the researchers' armamentarium. They primarily help in re-sequencing, re-styling and reformatting of the citation content in the research manuscripts. They also enable multi-user collaboration on research and allow the researcher to manage database searches and digital libraries. Using these softwares allows synchronization of cloud based digital libraries on multiple electronic devices enabling remote access, and also allows for management of online portfolios. We, herein, describe the basic principles, functions, and limitations of various reference management softwares.

Applying text similarity algorithm to analyze the triangular citation behavior of scientists

Triangular citation is a particular citation structure with research value and application significance, and the fundamental mechanism of its formation lies in the indirect citation between Literature A and Literature C. It is necessary to dig deeply into the citation context and citation motivation of indirect citation behavior to understand the internal mechanism of triangular citation. In this paper, the text-similarity algorithm was used to calculate the similarity of citation contents between B→A and C→A in 22,536 triangular citation data, and the followers C with high citation similarity were identified. Secondly, from the perspectives of language, literature type, interdisciplinary citation, and author self-citation, combined literature characteristics and followers’ identification results, the context and motivation of indirect citation behaviors imposed by the followers C were analyzed. The results show that followers C appear more frequently in triangular citations. At the same time, due to the influence of language differences, literature types, discipline differences or author self-citation, follower C will have lazy citation motivation when citing Literature A, partially or out-and-out requote Literature A from the citation content of Literature B, resulting in a high similarity between the citation content of C→A and B→A.

Characterizing references from different disciplines: A perspective of citation content analysis

Multidisciplinary cooperation is now common in research since social issues inevitably involve multiple disciplines. In research articles, reference information, especially citation content, is an important representation of communication among different disciplines. Analyzing the distribution characteristics of references from different disciplines in research articles is basic to detecting the sources of referred information and identifying contributions of different disciplines. This work takes articles in PLoS as the data and characterizes the references from different disciplines based on Citation Content Analysis (CCA). First, we download 210,334 full-text articles from PLoS and collect the information of the in-text citations. Then, we identify the discipline of each reference in these academic articles. To characterize the distribution of these references, we analyze three characteristics, namely, the number of citations, the average cited intensity and the average citation length. Finally, we conclude that the distributions of references from different disciplines are significantly different. Although most references come from Natural Science, Humanities and Social Sciences play important roles in the Introduction and Background sections of the articles. Basic disciplines, such as Mathematics, mainly provide research methods in the articles in PLoS. Citations mentioned in the Results and Discussion sections of articles are mainly in-discipline citations, such as citations from Nursing and Medicine in PLoS.

Literature research of Compendium of Materia Medica quoted from Bencao Tujing

According to the content of Compendium of Materia Medica quoted from Bencao Tujing(, Illustration Classics for Materia Medica), this paper analyzed Li Shizhen's treatment methods of Bencao Tujing, such as description mode, the first record of drug varieties, distinction of related drugs, the purpose of specific citation content, and the characteristics of citation literature. It is believed that Li Shizhen's citation on Bencao Tujing is comprehensive, scientific, and absorbed the essence of Bencao Tujing. It was a great contribution to Li Shizhen's successful compilation of Compendium of Materia Medica, and Li Shizhen insisted on the principle of treating repetition with deletions, treating deficiency with addition, and treating errors with correction. Li Shizhen corrected realistically the deficiency of "The pictures and annotations are different, the two are not matched with each other, or there are pictures without annotations, or there are articles without pictures,or there are right annotations with the wrong pictures" in Bencao Tujing.

Difference analysis of research topics in a specific domain based on different content levels

[Purpose/Significance] This paper aims to explore whether there are differences in the title and abstract, citation content, and full-text content on research topic, and analyze whether the topic content in the title and ab stract can reveal the research content of the full text and the effect of the citation content on the content of the citing literature, so as to provide theoretical support for analyzing the research content of the full text based on the title and abstract of the literature. [Method/Process] This paper conducts an empirical study using Chinese journal papers in the field of COVID-19, extracts feature words from the titles and abstracts, citations and full-text contents of the literature, uses the clustering algorithm to cluster the feature words, and then uses manual interpretation to identify the research topics, and conducts a comparative study to analyze the topic differences among the three parts. [Results/Conclusions] The results show that: the research topics are different in the title and abstract, citation content and full-text content of the literature;compared with the title and abstract, the full text contains more topic content, but the difference in the topic content is small, so the topic content in the title and abstract can be used to represent the research content of the full text;the content of the citation is related to the topic of the citing literature, and they can complement each other. © 2021 Agricultural Information Institute, Chinese Academy of Agricultural Sciences. All Rights Reserved.

The citation impact of social sciences and humanities upon patentable technology

This paper examines the citation impact of papers published in scientific-scholarly journals upon patentable technology, as reflected in examiner- or inventor-given references in granted patents. It analyses data created by SCImago Research Group, linking PATSTAT’s scientific non-patent references (SNPRs) to source documents indexed in Scopus. The frequency of patent citations to journal papers is calculated per discipline, year, institutional sector, journal subject category, and for “top” journals. PATSTAT/Scopus-based statistics are compared to those derived from Web of Science/USPTO linkage. A detailed assessment is presented of the technological impact of research publications in social sciences and humanities (SSH). Several subject fields perform well in terms of the number of citations from patents, especially Library and Information Science, Language and Linguistics, Education, and Law, but many of the most cited journals find themselves in the interface between SSH and biomedical or natural sciences. Analyses of the titles of citing patents and cited papers are presented that shed light upon the cognitive content of patent citations. It is proposed to develop more advanced indicators of citation impact of papers upon patents, and ways to combine citation counts with citation content and context analysis.

Citation content/context data as a source for research cooperation analysis

Using citation relationships, one can build three groups of papers: (1) the papers of a selected author; (2) those papers cited by the author; (3) papers citing the author. Authors of papers from these three groups can be presented as a fragment of a research cooperation network, because they use/cite research outputs of each other. Their papers' full texts and especially the contexts of their in-text citations contain some information about the character of this research cooperation. We present a concept of research cooperation, based on publications and the current results of the Cirtec project for building the research cooperation characteristics. This work is based on the processing of citation content/context data. The results include an on-line service for authors to monitor the citation content data extractions and three types of built indicators/parameters: co-citation statistics, spatial distribution of citations over papers' body and topic models for citation contexts.