Dual Learning: Theoretical Study and an Algorithmic Extension

Abstract

Dual learning has been successfully applied in many machine learning applications including machine translation, image-to-image transformation, etc. The high-level idea of dual learning is very intuitive: if we map an x from one domain to another and then map it back, we should recover the original x. Although its effectiveness has been empirically verified, the theoretical understanding of dual learning is still very limited. In this paper, we characterize sufficient conditions for dual learning to outperform vanilla translators. Based on our theoretical analysis, we further extend dual learning by introducing more related mappings and propose multi-step dual learning, in which we leverage feedback signals from additional domains to improve the qualities of the mappings. We show that multi-step dual learning has the potential to boost the performance of dual learning. Experiments on WMT 14 English $$\leftrightarrow$$ ↔ German, MultiUN English $$\leftrightarrow$$ ↔ French, and IWSLT’17 English $$\leftrightarrow$$ ↔ Chinese translations verify our theoretical findings on dual learning, and the results on the translations among English, French, and Spanish of MultiUN demonstrate the effectiveness of multi-step dual learning.