Abstract

Transfer learning (TL) seeks to improve the learning of a data-scarce target domain by using information from source domains. However, the source and target domains usually have different data distributions, which may lead to negative transfer. To alleviate this issue, we propose a Wasserstein Selective Transfer Learning (WSTL) method. Specifically, the proposed method considers a reinforced selector to select helpful data for transfer learning. We further use a Wasserstein-based discriminator to maximize the empirical distance between the selected source data and target data. The TL module is then trained to minimize the estimated Wasserstein distance in an adversarial manner and provides domain invariant features for the reinforced selector. We adopt an evaluation metric based on the performance of the TL module as delayed reward and a Wasserstein-based metric as immediate rewards to guide the reinforced selector learning. Compared with the competing TL approaches, the proposed method selects data samples that are closer to the target domain. It also provides better state features and reward signals that lead to better performance with faster convergence. Extensive experiments on three real-world text mining tasks demonstrate the effectiveness of the proposed method.

Highlights

  • Transfer learning (TL) is a type of classical machine learning methods to leverage information from data-rich source domains to help a data-scarce target domain (Pan and Yang, 2009)

  • Transfer learning based on deep neural networks, referred to as deep transfer learning (Ruder and Plank, 2017; Yosinski et al, 2014), has been widely used on various tasks in natural language processing

  • The reinforced source data selector serves as an agent and the selection process can be modeled as a Markov decision process which can be solved by reinforcement learning: The selector selects a subset of source data, feeds into the TL module with the target data and receives rewards for this action

Read more

Summary

Introduction

Transfer learning (TL) is a type of classical machine learning methods to leverage information from data-rich source domains to help a data-scarce target domain (Pan and Yang, 2009). The discriminator provides Wasserstein distancebased metric to serve as an immediate reward signal to help guide RL policy, which can solve the sparse reward problem In this way, the proposed method can select high-quality source data to help the target in an efficient and effective manner. 1) We proposed a Wasserstein distance based reinforced selective transfer learning method to select high-quality data efficiently and effectively to alleviate negative transfer. 1 Let ne and nt denote the number of source samples and target samples respectively, the empirical Wasserstein distance can be approximated by maximizing the discriminator loss Lwd when fφ is 1-Lipschitz: 2) The introduced Wasserstein discriminator provides better state representations and immediate reward signals to the reinforced selector, which leads to more stable training and better performance with faster convergence. Where Pvis sampled uniformly along straight lines between source and target representation pairs and λ is the penalty coefficient

TL with Wasserstein Discriminator formulated as: min ω
Reinforced Selective Training
Datasets and Implementation Details
Experiments on Review Helpfulness Prediction
Conclusions
Proof for Equation 7
Experiment Settings
Findings
Feature Visualization
Full Text
Paper version not known

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.