A hybrid transfer learning algorithm incorporating TrSVM with GASEN

Abstract

Traditional machine learning is generally committed to obtaining classifiers which are well-performed over unlabeled test data. This usually relies on two critical assumptions: firstly, sufficient labeled training data are available; secondly, training and testing data are drawn from the same distribution and the same feature space. Unfortunately, in most cases, the actual situation is difficult to meet the above conditions. Transfer learning scheme is naturally proposed to alleviate this problem. In order to get robust classifiers with relatively lower computational costs, we incorporate the rationale of Support Vector Machine (SVM) into transfer learning scheme and propose a novel SVM-based transfer learning model, abbreviated as TrSVM. In this method, support vector sets are extracted to represent the source domain. New training datasets are respectively constructed by combining each support vector set and target labeled dataset. On the basis of these training datasets, a number of new base classifiers can be acquired. Since performance of a classifiers ensemble is generally superior to that of individual classifiers, ensemble selection is utilized in our work. A hybrid transfer learning algorithm, integrating the Genetic Algorithm based Selective Ensemble (GASEN) with TrSVM, is proposed, and abbreviated as TrGASVM, naturally. GASEN is a genetic algorithm-based heuristic algorithm for solving combinatorial optimization problems. It can not only enhance the generalization ability of an ensemble, but also alleviate the local minimum problem of greedy ensemble pruning methods. Since TrGASVM is under frame of TrSVM and GASEN, it inevitably inherits the advantages of both algorithms. The reasonable incorporation of TrSVM with GASEN endows TrGASVM with favorable transfer learning capability, with its effectiveness being demonstrated by the experimental results on three real-world text classification datasets.