Integration of speech recognition and machine translation: Speech recognition word lattice translation

Abstract

An important issue in speech translation is to minimize the negative effect of speech recognition errors on machine translation. We propose a novel statistical machine translation decoding algorithm for speech translation to improve speech translation quality. The algorithm can translate the speech recognition word lattice, where more hypotheses are utilized to bypass the misrecognized single-best hypothesis. The decoding involves converting the recognition word lattice to a translation word graph by a graph-based search, followed by a fine rescoring by an A* search. We show that a speech recognition confidence measure implemented by posterior probability is effective to improve speech translation. The proposed techniques were tested in a Japanese-to-English speech translation task, in which we measured the translation results in terms of a number of automatic evaluation metrics. The experimental results demonstrate a consistent and significant improvement in speech translation achieved by the proposed techniques.