Fact-checking: relevance assessment of references in the Polish political domain

Abstract

The prevalence of fake news could be observed in circumstances of emotion-causing events, like elections or pandemics. In fear of the potential impact, many fact-checking organisations were established. However, fact-checking requires a large amount of human labor, and hence there is a strong demand for complete automation of this process. Nevertheless, this milestone has not been achieved yet, even for English. The problem grows for the less popular languages that suffer from a scarcity of available resources. To address this problem for the Polish language domain, we propose a solution for automating one of the fact-checking stages - relevance assessment, which is crucial when searching for evidence. Leveraging recent advancements in natural language processing, we have acquired relevant data and developed classifiers of evidence relevance with respect to claims in Polish. Our approach can assess the evidence relevance with a performance at a level of a 0.778 F1-score.