Providing citations to support fact-checking: Contextualizing detection of sentences needing citation on small wikipedias

Abstract

Authoritative citations are critical to ensure information integrity, especially in encyclopedias like Wikipedia. To date, research on automating citation worthiness detection has largely focused on the most resourceful language, English Wikipedia, neglecting the applicability to smaller Wikipedias. In addition, previous research proposed models that analyze the content inherent to a sentence to determine its citation worthiness, overlooking the potential of additional context to improve the prediction. Addressing these gaps, our study proposes a transformer-based contextualized approach for smaller Wikipedias, presenting a novel method to compile high-quality datasets for the Albanian, Basque, and Catalan editions. We develop the Contextualized Citation Worthiness (CCW) model, employing sentence representations enriched with adjacent sentences and topic categories for enhanced contextual insight. Empirical experiments on three newly created datasets demonstrate significant performance improvements of our contextualized CCW model, with 6%, 3% and 6% absolute improvements over the baseline for Albanian, Basque and Catalan datasets, respectively. We conduct an in-depth analysis to understand the influence and extent to which preceding and succeeding context as well as topic categories contribute to the accuracy of citation-worthiness predictions. Our findings suggest that incorporating such contextual information aids in the automatic identification of sentences in need of citations, not least when both the preceding and succeeding context are incorporated. This has implications for supporting Wikipedia projects across low-resource languages, promoting better article validation and fact-checking.