Token-Level Fact Correction in Abstractive Summarization

Abstract

This paper addresses fact correction for abstractive summarization of which aim is to edit a system-generated summary into a new source-consistent summary. The summaries generated by abstractive summarization models often contain various kinds of factual errors. Thus, fact correction becomes essential to apply abstractive summarization to real-world applications. However, most existing methods for fact correction focus only on entity-level errors, which occasions the error correction methods to miss non-entity errors such as inconsistent tokens or mentions. Therefore, this paper presents a token-level fact correction that resolves inconsistencies of a system-generated summary at the token level. Since a token is the smallest meaning-bearing unit, all kinds of errors can be corrected if the errors are rectified at this level. The proposed fact corrector examines the consistency of a summary at the summary level like existing methods, but corrects the found inconsistencies at the token level. Thus, the proposed corrector consists of three modules of a summary fact checker, a token fact checker, and a fact emender. The summary fact checker inspects if a system-generated summary is factually consistent with a source text, the token fact checker finds out the tokens which cause inconsistency, and the fact emender actually replaces the inconsistency-causing tokens with correct tokens in the source text. Since these modules are closely related and affect one another, they are jointly trained to improve the performance of each module. The effectiveness of the proposed fact corrector is empirically proven from two viewpoints of consistency and summarization performance. For correcting inconsistencies in a summary, it is shown that the summaries by the proposed corrector are more factually consistent than those by its competitors. In addition, the proposed corrector outperforms the current state-of-the-art corrector even in automatic summarization performance.