Abstract
Unsupervised domain adaptation addresses the problem of leveraging labeled data in a source domain to learn a well-performing model in a target domain where labels are unavailable. In this paper, we improve upon a recent theoretical work (Zhang et al., 2019b) and adopt the Margin Disparity Discrepancy (MDD) unsupervised domain adaptation algorithm to solve the cross-lingual text labeling problems. Experiments on cross-lingual document classification and NER demonstrate the proposed domain adaptation approach advances the state-of-the-art results by a large margin. Specifically, we improve MDD by efficiently optimizing the margin loss on the source domain via Virtual Adversarial Training (VAT). This bridges the gap between theory and the loss function used in the original work Zhang et al.(2019b), and thereby significantly boosts the performance. Our numerical results also indicate that VAT can remarkably improve the generalization performance of both domains for various domain adaptation approaches.
Highlights
Unsupervised domain adaptation provides an appealing solution to many applications where direct access to a massive amount of labeled data is prohibitive or very costly (Sun and Saenko, 2014; Vazquez et al, 2013; Stark et al, 2010; Keung et al, 2019)
We demonstrated that Margin Disparity Discrepancy (MDD) can generally outperform the current state-of-the-art model (Keung et al, 2019) by a large margin
We further improve MDD by identifying the gap between theory and the actual loss function being used in the original work (Zhang et al, 2019b)
Summary
Unsupervised domain adaptation provides an appealing solution to many applications where direct access to a massive amount of labeled data is prohibitive or very costly (Sun and Saenko, 2014; Vazquez et al, 2013; Stark et al, 2010; Keung et al, 2019). Many recent successes in unsupervised domain adaptation have been achieved by learning domain invariant features that are simultaneously being discriminative to the task in the source domain (Chen et al, 2018; Ganin and Lempitsky, 2014; Ganin et al, 2016; Tzeng et al, 2017) Following this line, Keung et al (2019) propose a language-adversarial training approach for cross-lingual document classification and NER. Instead of training a discriminator that predicts if the representations are from the source domain or the target domain (Keung et al, 2019; Ganin and Lempitsky, 2014; Ganin et al, 2016), Zhang et al (2019b) proposes to optimize an auxiliary classifier which, together with the classifier, minimizes the discrepancy between the two domains via adversarial training We apply this approach to cross-lingual text labeling tasks, which, as demonstrated, outperforms Keung et al (2019) by a large margin. This matches the theoretical insights (Ben-David et al, 2010; Zhang et al, 2019b) that the generalization of the target domain can be boosted as a consequence of the improvement in the source domain
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
