Text Reuse Detection Research Articles

Mono-lingual text reuse detection for the Urdu language at lexical level

Text reuse is the process of creating new texts from pre-existing ones. In recent years, Urdu Text Reuse Detection (U-TRD) has garnered the attention of researchers due to the ready availability of digital text all over the internet, which can be copied or paraphrased from other sources without proper attribution, making it easier to reuse but challenging to detect. Previous studies have explored the issue of U-TRD at the phrasal, sentence/passage, and document levels, using benchmark corpora and methods. However, the problem of U-TRD has not been investigated at the lexical level in terms of corpora and methods. To address this research gap, our study has developed a large benchmark corpus manually annotated at the lexical level. This corpus consists of 22,184 text pairs categorized into two levels of rewrite: (1) Derived (8,660) and (2) Non-Derived (13,524). Additionally, our research has involved the development, application, evaluation, and comparison of a range of methods, including baseline methods (uni-gram overlap and word embedding-based methods), along with state-of-the-art transformer-based methods and feature-fusion-based methods, using the proposed UTRD-Lex-23 corpus. Our study concludes that one of our proposed feature-fusion methods outperforms all other methods. The model we propose, which combines seven different Sentence Transformers (ST) (each producing 768 dimension vectors) with one uni-gram (at word level) and sixteen different features extracted from four different Word Embedding (WE) based models (yielding 300 dimension vectors), achieves an F1 score of 0.70601 using 10-fold cross validation. To foster and promote research in Urdu (a low-resourced language) proposed corpus will be freely and publicly available for research purposes.

Textuality as amplification: reconsidering close reading and distant reading in cultural history

ABSTRACT This article discusses the idea of distant reading and explores the ways in which it can be conducted in research. It focuses especially on how distant reading can contribute to the study of cultural history, which is often regarded as a domain of close reading. The article argues that distant reading methods can successfully be applied in the analysis of cultural transmission in the past, where it is often essential to combine the study of textual signification with the idea of textuality as material flow. The article draws on an example from press history and especially discusses text reuse detection as a strategy for identifying textual amplification.

Urdu Text Reuse Detection at Phrasal level using Sentence Transformer-based approach

Urdu Text Reuse Detection at Phrasal level using Sentence Transformer-based approach

Cross-lingual Text Reuse Detection at Document Level for English-Urdu Language Pair

In recent years, the problem of Cross-Lingual Text Reuse Detection (CLTRD) has gained the interest of the research community due to the availability of large digital repositories and automatic Machine Translation (MT) systems. These systems are readily available and openly accessible, which makes it easier to reuse text across languages but hard to detect. In previous studies, different corpora and methods have been developed for CLTRD at the sentence/passage level for the English-Urdu language pair. However, there is a lack of large standard corpora and methods for CLTRD for the English-Urdu language pair at the document level. To overcome this limitation, the significant contribution of this study is the development of a large benchmark cross-lingual (English-Urdu) text reuse corpus, called the TREU (Text Reuse for English-Urdu) corpus. It contains English to Urdu real cases of text reuse at the document level. The corpus is manually labelled into three categories (Wholly Derived = 672, Partially Derived = 888, and Non Derived = 697) with the source text in English and the derived text in the Urdu language. Another contribution of this study is the evaluation of the TREU corpus using a diversified range of methods to show its usefulness and how it can be utilized in the development of automatic methods for measuring cross-lingual (English-Urdu) text reuse at the document level. The best evaluation results, for both binary ( F 1 = 0.78) and ternary ( F 1 = 0.66) classification tasks, are obtained using a combination of all Translation plus Mono-lingual Analysis (T+MA) based methods. The TREU corpus is publicly available to promote CLTRD research in an under-resourced language, i.e., Urdu.

Develop corpora and methods for cross-lingual text reuse detection for English Urdu language pair at lexical, syntactical, and phrasal levels

In recent years, Cross-Lingual Text Reuse Detection (CLTRD) has attracted the attention of the research community because large digital repositories and efficient Machine Translation systems are readily and freely available, which makes it easier to reuse text across the languages and very difficult to detect it. In the previous studies, the problem of CLTRD for the English-Urdu language pair has been explored at the sentence/passage and document level, and benchmark corpora and methods have been developed. However, there is a lack of benchmark corpora and methods for the CLTRD for the English-Urdu language pair at the lexical, syntactical, and phrasal levels. To fulfill this research gap, this study presents three large benchmark corpora for detecting the Cross-Lingual Text Reuse (CLTR) at three levels of rewrite (Wholly Derived (WD), Partially Derived (PD), and Non Derived (ND)). The CLEU-Lex, CLEU-Syn and CLEU-Phr corpora contain 66,485 (WD = 22,236, PD = 20,315 and ND = 23,934), 60,267 (WD = 20,007, PD = 16,979 and ND = 23,281) and 60,106 (WD = 23,862, PD = 15,878 and ND = 20,366) CLTR pairs respectively. As a secondary major contribution, we have applied the Cross-Lingual Word Embedding (CLWE), Cross-Lingual Semantic Tagger (CLST), and Cross-Lingual Sentence Transformer (CLSTR) based methods on our three proposed corpora for the CLTRD. Our extensive experimentation showed that for the binary classification task, the best results on the CLEU-Lex corpus were obtained using the cross-lingual sentence transformer ( $$F_{1}$$ = 0.80). For the CLEU-Syn and CLEU-Phr corpora, the best results were obtained using the cross-lingual sentence transformer and a combination of the CLWE, CLST and CLSTR methods ( $$F_{1}$$ = 0.92 on CLEU-Syn and $$F_{1}$$ = 0.94 on CLEU-Phr). For the ternary classification task, the best results on the CLEU-Lex corpus were obtained using the cross-lingual sentence transformer method ( $$F_{1}$$ = 0.69). For the CLEU-Syn corpus, the best results were obtained using a combination of the CLWE, CLST, and CLSTR methods ( $$F_{1}$$ = 0.82). For the CLEU-Phr corpus the best results were obtained using cross-lingual sentence transformer and combination of CLWE, CLST, and CLSTR methods ( $$F_{1}$$ = 0.78). To foster and promote research in Urdu (a low-resourced language) all the three proposed corpora are free and publicly available for research purposes.

Cross-Lingual Text Reuse Detection at sentence level for English–Urdu language pair

In recent years, the problem of Cross-Lingual Text Reuse Detection (X-TRD) has gained the interest of researchers due to the availability of large digital repositories and automatic translation systems. These systems are promptly available and openly accessible, which makes it easier to reuse text across the languages and hard to detect. In previous studies, different corpora and techniques have been developed for X-TRD at sentence/passage and document level for the English–Urdu language pair. However, there is a lack of large benchmark corpora and standard techniques for X-TRD for the English–Urdu language pair at the sentence level. To overcome this limitation, this study presents a large benchmark sentential cross-lingual (English–Urdu) corpus of 21,669 sentence pairs with simulated cases of X-TR, which are manually annotated at three levels of rewrite (Wholly Derived (WD) = 7,655, Partially Derived (PD) = 6,461, and Non Derived (ND) = 7,553). As a second major contribution, we have applied various state-of-the-art Cross-Lingual Sentence Transformers (CLST), and Translation plus Mono-lingual Analysis (T+MA) including N-gram Overlap (lexical), WordNet based techniques (semantic), mono-lingual word embedding-based techniques, and Kullback–Leibler Distance (KLD) (probabilistic) on our proposed sentential corpus for X-TRD. For the binary classification, the best results are obtained (F1 = 0.94) using a combination of all CLST and T+MA techniques and a combination of all T+MA techniques, whereas, for the ternary classification task, the best results are obtained (F1 = 0.84) using a combination of all CLST and T+MA techniques. The corpus will be publicly available to foster and promote research for X-TRD in an under-resourced language, such as the Urdu language.

Methodology for the implementation and use of text reuse detection systems in secondary education

The article analyses the situation with the lack of systematic control over the originality of texts of students’ written works in secondary schools, including individual projects, provided by the Russian Federal State Educational Standards. The authors point out that compliance with the academic program’s requirements to observe the norms and rules of citation and absence of plagiarism in the texts of individual projects is not ensured in practice in any way. In particular, schools have not developed evaluation criteria and procedures for monitoring. Also, there is no access to the necessary text reuse detection tools. The article substantiates the necessity for secondary education to check students’ for reused text by using specialized software on the example of the “Antiplagiat” system. As a result of the study, a methodology of introducing digital tools for text reuse detection in schools is presented. We developed such a methodology for secondary schools for the first time. It will allow avoiding the repetition of methodological errors made during the introduction of tools for text reuse detection in higher education, provide methodological support to teachers, and establish adequate criteria for assessing the originality of works, considering the specifics of the school.

Cross-lingual Text Reuse Detection Using Translation Plus Monolingual Analysis for English-Urdu Language Pair

Cross-Lingual Text Reuse Detection (CLTRD) has recently attracted the attention of the research community due to a large amount of digital text readily available for reuse in multiple languages through online digital repositories. In addition, efficient machine translation systems are freely and readily available to translate text from one language into another, which makes it quite easy to reuse text across languages, and consequently difficult to detect it. In the literature, the most prominent and widely used approach for CLTRD is Translation plus Monolingual Analysis (T+MA). To detect CLTR for English-Urdu language pair, T+MA has been used with lexical approaches, namely, N-gram Overlap, Longest Common Subsequence, and Greedy String Tiling. This clearly shows that T+MA has not been thoroughly explored for the English-Urdu language pair. To fulfill this gap, this study presents an in-depth and detailed comparison of 26 approaches that are based on T+MA. These approaches include semantic similarity approaches (semantic tagger based approaches, WordNet-based approaches), probabilistic approach (Kullback-Leibler distance approach), monolingual word embedding-based approaches siamese recurrent architecture, and monolingual sentence transformer-based approaches for English-Urdu language pair. The evaluation was carried out using the CLEU benchmark corpus, both for the binary and the ternary classification tasks. Our extensive experimentation shows that our proposed approach that is a combination of 26 approaches obtained an F 1 score of 0.77 and 0.61 for the binary and ternary classification tasks, respectively, and outperformed the previously reported approaches [ 41 ] ( F 1 = 0.73) for the binary and ( F 1 = 0.55) for the ternary classification tasks) on the CLEU corpus.

Readers' engagement through digital social reading on Twitter: the TwLetteratura case study

PurposeThe purpose of this article is to explore how participants with different motivations (educational or leisure), familiarity with the medium (newbies and active Twitter users), and participating instructions respond to a highly structured digital social reading (DSR) activity in terms of intensity of engagement and social interaction.Design/methodology/approachA case study involving students and teachers of 211 Italian high school classes and 242 other Twitter users, who generated a total of 18,962 tweets commenting on a literary text, was conducted. The authors performed both a quantitative analysis focusing on the number of tweets/retweets generated by participants and a network analysis exploiting the study of interactions between them. The authors also classified the tweets with respect to their originality, by using both automated text reuse detection approaches and manual categorization, to identify quotations, paraphrases and other forms of reader response.FindingsThe decoupling (both in space and time) of text read (in class) and comments (on Twitter) likely led users to mainly share text excerpts rather than original personal reactions to the story. There was almost no interaction outside the classroom, neither with other students nor with generic Twitter users, characterizing this project as a shared experience of “audiencing” a media event. The intensity of social interactions is more related to the breadth of the audience reached by the user-generated content and to a strong retweeting activity. In general, better familiarity with digital (social) media is related to an increase in the level of social interaction.Originality/valueThe authors analyzed one of the largest educational social reading projects ever realized, contributing to the still scarce empirical research about DSR. The authors employed state-of-the-art automated text reuse detection to classify reader response.

Evaluation of Fingerprint Selection Algorithms for Local Text Reuse Detection

Abstract Detection of local text reuse is central to a variety of applications, including plagiarism detection, origin detection, and information flow analysis. This paper evaluates and compares effectiveness of fingerprint selection algorithms for the source retrieval stage of local text reuse detection. In total, six algorithms are compared – Every p-th, 0 mod p, Winnowing, Hailstorm, Frequency-biased Winnowing (FBW), as well as the proposed modified version of FBW (MFBW). Most of the previously published studies in local text reuse detection are based on datasets having either artificially generated, long-sized, or unobfuscated text reuse. In this study, to evaluate performance of the algorithms, a new dataset has been built containing real text reuse cases from Bachelor and Master Theses (written in English in the field of computer science) where about half of the cases involve less than 1 % of document text while about two-thirds of the cases involve paraphrasing. In the performed experiments, the overall best detection quality is reached by Winnowing, 0 mod p, and MFBW. The proposed MFBW algorithm is a considerable improvement over FBW and becomes one of the best performing algorithms. The software developed for this study is freely available at the author’s website http://www.cs.rtu.lv/jekabsons/.

Temu kembali dokumen sumber rujukan dalam sistem daur ulang teks

The architecture of the text-reuse detection system consists of three main modules, i.e., source retrieval, text analysis, and knowledge-based postprocessing. Each module plays an important role in the accuracy rate of the detection outputs. Therefore, this research focuses on developing the source retrieval system in cases where the source documents have been obfuscated in different levels. Two steps of term weighting were applied to get such documents. The first was the local-word weighting, which has been applied to the test or reused documents to select query per text segments. The tf-idf term weighting was applied for indexing all documents in the corpus and as the basis for computing cosine similarity between the queries per segment and the documents in the corpus. A two-step filtering technique was applied to get the source document candidates. Using artificial cases of text reuse testing, the system achieves the same rates of precision and recall that are 0.967, while the recall rate for the simulated cases of reused text is 0.66.

History and Virtual Topology: The Nineteenth-Century Press as Material Flow

Drawing on a new materialist approach, this article discusses the concepts of the virtual and virtual topology, and their fruitfulness for historians' empirical work. It starts by following Gilles Deleuze's argument that the virtual, the transformative potential of the real, has to be distinguished from the possible, which is merely an imagined double of the given world. Embracing this premise, the article shows the potential of virtual topology to shed light on the transformation of a network in the past. It suggests that history is not only about actual and stable things, it is also a site of becomings. This idea is elaborated through an analysis that focuses on the changes in the nineteenth-century Finnish press. The virtual as a theoretical concept is combined with the methodological opportunities offered by recent developments in the digital humanities, in this case, text reuse detection.

Measuring Short Text Reuse for the Urdu Language

Text reuse occurs when one borrows the text (either verbatim or paraphrased) from an earlier written text. A large and increasing amount of digital text is easily and readily available, making it simpler to reuse but difficult to detect. As a result, automatic detection of text reuse has attracted the attention of the research community due to the wide variety of applications associated with it. To develop and evaluate automatic methods for text reuse detection, standard evaluation resources are required. In this paper, we propose one such resource for a significantly under-resourced language—Urdu, which is widely used in day to day communication and has a large digital footprint particularly in the Indian subcontinent. Our proposed Urdu short text reuse corpus contains 2684 short Urdu text pairs, manually labeled as verbatim (496), paraphrased (1329), and independently written (859). In addition, we describe an evaluation of the corpus using various state-of-the-art text reuse detection methods with binary and multi-classification settings and a set of four classifiers. Output results show that character n-gram overlap using J48 classifier outperform other methods for the Urdu short text reuse detection task.

COUNTER: corpus of Urdu news text reuse

Text reuse is the act of borrowing text from existing documents to create new texts. Freely available and easily accessible large online repositories are not only making reuse of text more common in society but also harder to detect. A major hindrance in the development and evaluation of existing/new mono-lingual text reuse detection methods, especially for South Asian languages, is the unavailability of standardized benchmark corpora. Amongst other things, a gold standard corpus enables researchers to directly compare existing state-of-the-art methods. In our study, we address this gap by developing a benchmark corpus for one of the widely spoken but under resourced languages i.e. Urdu. The COrpus of Urdu News TExt Reuse (COUNTER) corpus contains 1200 documents with real examples of text reuse from the field of journalism. It has been manually annotated at document level with three levels of reuse: wholly derived, partially derived and non derived. We also apply a number of similarity estimation methods on our corpus to show how it can be used for the development, evaluation and comparison of text reuse detection systems for the Urdu language. The corpus is a vital resource for the development and evaluation of text reuse detection systems in general and specifically for Urdu language.