Cross-domain Text Classification Research Articles

TACIT: A Target-Agnostic Feature Disentanglement Framework for Cross-Domain Text Classification

Cross-domain text classification aims to transfer models from label-rich source domains to label-poor target domains, giving it a wide range of practical applications. Many approaches promote cross-domain generalization by capturing domaininvariant features. However, these methods rely on unlabeled samples provided by the target domains, which renders the model ineffective when the target domain is agnostic. Furthermore, the models are easily disturbed by shortcut learning in the source domain, which also hinders the improvement of domain generalization ability. To solve the aforementioned issues, this paper proposes TACIT, a target domain agnostic feature disentanglement framework which adaptively decouples robust and unrobust features by Variational Auto-Encoders. Additionally, to encourage the separation of unrobust features from robust features, we design a feature distillation task that compels unrobust features to approximate the output of the teacher. The teacher model is trained with a few easy samples that are easy to carry potential unknown shortcuts. Experimental results verify that our framework achieves comparable results to state-of-the-art baselines while utilizing only source domain data.

Transferred Low-rank Discriminative Sub-dictionary Learning Method for Information Identification and Understanding in IoT-based Natural Language Processing

As Internet of Things (IoT) technology progresses rapidly, there is an increasing demand for automatic identification and understanding of natural language data. However, data labeling requires large amounts of effort and cost. Most intelligent algorithms rely on the assumption of uniform distribution of data, which brings great challenge to IoT-based natural language processing. To solve this problem, this study develops a transferred low-rank discriminative sub-dictionary learning (TLDSL) method. The TLDSL method learns a shared subspace through the maximum mean discrepancy (MMD) strategy that minimizes the distribution difference of sparse coefficients between the source and target domains. By learning the common sub-dictionary of the two domains, TLDSL reveals the intrinsic connection and establishes a bridge between the two domains, thus completing the knowledge transfer. By introducing the sub-dictionary incoherence, TLDSL can avoid the atomic correlation between different sub-dictionaries. In addition, the sparse coefficients are constrained in low rank representation, which can improve the model discrimination ability while preserving the global data structure. Experiments show that the TLDSL method can be effectively performed on cross-domain text classification and handwritten digit recognition.

Semi-Supervised Sentiment Classification and Emotion Distribution Learning Across Domains

In this study, sentiment classification and emotion distribution learning across domains are both formulated as a semi-supervised domain adaptation problem, which utilizes a small amount of labeled documents in the target domain for model training. By introducing a shared matrix that captures the stable association between document clusters and word clusters, non-negative matrix tri-factorization (NMTF) is robust to the labeled target domain data and has shown remarkable performance in cross-domain text classification. However, the existing NMTF-based models ignore the incompatible relationship of sentiment polarities and the relatedness among emotions. Besides, their applications on large-scale datasets are limited by the high computation complexity. To address these issues, we propose a semi-supervised NMTF framework for sentiment classification and emotion distribution learning across domains. Based on a many-to-many mapping between document clusters and sentiment polarities (or emotions), we first incorporate the prior information of label dependency to improve the model performance. Then, we develop a parallel algorithm based on message passing interface (MPI) to further enhance the model scalability. Extensive experiments on real-world datasets validate the effectiveness of our method.

Topic Correlation Analysis for Cross-Domain Text Classification

Cross-domain text classification aims to automatically train a precise text classifier for a target domain by using labeled text data from a related source domain. To this end, the distribution gap between different domains has to be reduced. In previous works, a certain number of shared latent features (e.g., latent topics, principal components, etc.) are extracted to represent documents from different domains, and thus reduce the distribution gap. However, only relying the shared latent features as the domain bridge may limit the amount of knowledge transferred. This limitation is more serious when the distribution gap is so large that only a small number of latent features can be shared between domains. In this paper, we propose a novel approach named Topic Correlation Analysis (TCA), which extracts both the shared and the domain-specific latent features to facilitate effective knowledge transfer. In TCA, all word features are first grouped into the shared and the domain-specific topics using a joint mixture model. Then the correlations between the two kinds of topics are inferred and used to induce a mapping between the domain-specific topics from different domains. Finally, both the shared and the mapped domain-specific topics are utilized to span a new shared feature space where the supervised knowledge can be effectively transferred. The experimental results on two real-world data sets justify the superiority of the proposed method over the stat-of-the-art baselines.

Multi-Group Transfer Learning on Multiple Latent Spaces for Text Classification

Transfer learning aims to leverage valuable information in one domain to promote the learning tasks in the other domain. Some recent studies indicated that the latent information, which has a close relationship with the high-level concepts, are more suitable for cross-domain text classification than learning raw features. To obtain more latent information existing in the latent feature space, some previous methods constructed multiple latent feature spaces. However, those methods ignored that the latent information of different latent spaces may lack the relevance for promoting the adaptability of transfer learning models, even may lead to negative knowledge transfer when there exists a glaring discrepancy among the different latent spaces. Additionally, since those methods learn the latent space distributions using a strategy of direct-promotion, their computational complexity increases exponentially as the number of latent spaces increases. To tackle this challenge, this paper proposes a Multiple Groups Transfer Learning (MGTL) method. MGTL first constructs multiple different latent feature spaces and then integrates the adjacent ones that have a similar latent feature dimension into one latent space group. Along this way, multiple latent space groups can be obtained. To enhance the relevance among these latent space groups, MGTL makes the adjacent groups contain one same latent space at least. Then, different groups will have more relevance than raw latent spaces. Second, MGTL utilizes an indirect-promotion strategy to connect different latent space groups. The computational complexity of MGTL increases linearly as the number of latent space groups increases and is superior to those multiple latent space methods based on direct-promotion. In addition, an iterative algorithm is proposed to solve the optimization problem. Finally, a set of systematic experiments demonstrate that MGTL outperforms all the compared existing methods.

Investigating the transferring capability of capsule networks for text classification

Text classification has been attracting increasing attention with the growth of textual data created on the Internet. Great progress has been made by deep neural networks for domains where a large amount of labeled training data is available. However, providing sufficient data is time-consuming and labor-intensive, establishing substantial obstacles for expanding the learned models to new domains or new tasks. In this paper, we investigate the transferring capability of capsule networks for text classification. Capsule networks are able to capture the intrinsic spatial part-whole relationship constituting domain invariant knowledge that bridges the knowledge gap between the source and target domains (or tasks). We propose an iterative adaptation strategy for cross-domain text classification, which adapts the source domain to the target domain. A fast training method with capsule compression and class-guided routing is designed to make the capsule network more efficient in computation for cross-domain text classification. We first conduct experiments to evaluate the performance of the capsule network on six benchmark datasets for generic text classification. The capsule networks outperform the compared models on 4 out of 6 datasets, suggesting the effectiveness of the capsule networks for text classification. More importantly, we demonstrate the transferring capability of the proposed cross-domain capsule network (TL-Capsule) by applying it to two transfer learning applications: single-label to multi-label text classification and cross-domain sentiment classification. The experimental results show that capsule networks consistently and substantially outperform the compared methods for both tasks. To the best of our knowledge, this is the first work that empirically investigates the transferring capability of capsule networks for text modeling.

Heterogeneous Domain Adaptation via Nonlinear Matrix Factorization.

Heterogeneous domain adaptation (HDA) aims to solve the learning problems where the source- and the target-domain data are represented by heterogeneous types of features. The existing HDA approaches based on matrix completion or matrix factorization have proven to be effective to capture shareable information between heterogeneous domains. However, there are two limitations in the existing methods. First, a large number of corresponding data instances between the source domain and the target domain are required to bridge the gap between different domains for performing matrix completion. These corresponding data instances may be difficult to collect in real-world applications due to the limited size of data in the target domain. Second, most existing methods can only capture linear correlations between features and data instances while performing matrix completion for HDA. In this paper, we address these two issues by proposing a new matrix-factorization-based HDA method in a semisupervised manner, where only a few labeled data are required in the target domain without requiring any corresponding data instances between domains. Such labeled data are more practical to obtain compared with cross-domain corresponding data instances. Our proposed algorithm is based on matrix factorization in an approximated reproducing kernel Hilbert space (RKHS), where nonlinear correlations between features and data instances can be exploited to learn heterogeneous features for both the source and the target domains. Extensive experiments are conducted on cross-domain text classification and object recognition, and experimental results demonstrate the superiority of our proposed method compared with the state-of-the-art HDA approaches.

Softly Associative Transfer Learning for Cross-Domain Classification.

The main challenge of cross-domain text classification is to train a classifier in a source domain while applying it to a different target domain. Many transfer learning-based algorithms, for example, dual transfer learning, triplex transfer learning, etc., have been proposed for cross-domain classification, by detecting a shared low-dimensional feature representation for both source and target domains. These methods, however, often assume that the word clusters matrix or the clusters association matrix as knowledge transferring bridges are exactly the same across different domains, which is actually unrealistic in real-world applications and, therefore, could degrade classification performance. In light of this, in this paper, we propose a softly associative transfer learning algorithm for cross-domain text classification. Specifically, we integrate two non-negative matrix tri-factorizations into a joint optimization framework, with approximate constraints on both word clusters matrices and clusters association matrices so as to allow proper diversity in knowledge transfer, and with another approximate constraint on class labels in source domains in order to handle noisy labels. An iterative algorithm is then proposed to solve the above problem, with its convergence verified theoretically and empirically. Extensive experimental results on various text datasets demonstrate the effectiveness of our algorithm, even with the presence of abundant state-of-the-art competitors.

Lifelong Learning Maxent for Suggestion Classification

Suggestion classification for opinion data is defined as identifying a given utterance by suggestion or non-suggestion class. In this paper, we introduce a method called LLMaxent which is the solution for the cross-domain suggestion classification. LLMaxent is a lifelong machine learning approach using maximum entropy (Maxent). In the course of lifelong learning, the drawn knowledge from the past tasks is retained and supported for the future learning. From that, we build a classifier by using labelled data in existed domains for suggestion classification in a new domain. The experimental results show that the proposed novel model can improve the performance of cross-domain suggestion classification. This is one of the preliminary research in lifelong machine learning using Maxent. Its effect is not only for suggestion classification but also for cross-domain text classification in general.

Co-Clustering based Classification Algorithm with Latent Semantic Relationship for Cross-Domain Text Classification throughWikipedia

Co-Clustering based Classification Algorithm with Latent Semantic Relationship for Cross-Domain Text Classification throughWikipedia

Erratum to "Mining Distinction and Commonality across Multiple Domains Using Generative Model for Text Classification"

The distribution difference among multiple domains has been exploited for cross-domain text categorization in recent years. Along this line, we show two new observations in this study. First, the data distribution difference is often due to the fact that different domains use different index words to express the same concept. Second, the association between the conceptual feature and the document class can be stable across domains. These two observations actually indicate the distinction and commonality across domains. Inspired by the above observations, we propose a generative statistical model, named Collaborative Dual-PLSA (CD-PLSA), to simultaneously capture both the domain distinction and commonality among multiple domains. Different from Probabilistic Latent Semantic Analysis (PLSA) with only one latent variable, the proposed model has two latent factors y and z, corresponding to word concept and document class, respectively. The shared commonality intertwines with the distinctions over multiple domains, and is also used as the bridge for knowledge transformation. An Expectation Maximization (EM) algorithm is developed to solve the CD-PLSA model, and further its distributed version is exploited to avoid uploading all the raw data to a centralized location and help to mitigate privacy concerns. After the training phase with all the data from multiple domains we propose to refine the immediate outputs using only the corresponding local data. In summary, we propose a two-phase method for cross-domain text classification, the first phase for collaborative training with all the data, and the second step for local refinement. Finally, we conduct extensive experiments over hundreds of classification tasks with multiple source domains and multiple target domains to validate the superiority of the proposed method over existing state-of-the-art methods of supervised and transfer learning. It is noted to mention that as shown by the experimental results CD-PLSA for the collaborative training is more tolerant of distribution differences, and the local refinement also gains significant improvement in terms of classification accuracy.

Mining Distinction and Commonality across Multiple Domains Using Generative Model for Text Classification

The distribution difference among multiple domains has been exploited for cross-domain text categorization in recent years. Along this line, we show two new observations in this study. First, the data distribution difference is often due to the fact that different domains use different index words to express the same concept. Second, the association between the conceptual feature and the document class can be stable across domains. These two observations actually indicate the distinction and commonality across domains. Inspired by the above observations, we propose a generative statistical model, named Collaborative Dual-PLSA (CD-PLSA), to simultaneously capture both the domain distinction and commonality among multiple domains. Different from Probabilistic Latent Semantic Analysis (PLSA) with only one latent variable, the proposed model has two latent factors y and z, corresponding to word concept and document class, respectively. The shared commonality intertwines with the distinctions over multiple domains, and is also used as the bridge for knowledge transformation. An Expectation Maximization (EM) algorithm is developed to solve the CD-PLSA model, and further its distributed version is exploited to avoid uploading all the raw data to a centralized location and help to mitigate privacy concerns. After the training phase with all the data from multiple domains we propose to refine the immediate outputs using only the corresponding local data. In summary, we propose a two-phase method for cross-domain text classification, the first phase for collaborative training with all the data, and the second step for local refinement. Finally, we conduct extensive experiments over hundreds of classification tasks with multiple source domains and multiple target domains to validate the superiority of the proposed method over existing state-of-the-art methods of supervised and transfer learning. It is noted to mention that as shown by the experimental results CD-PLSA for the collaborative training is more tolerant of distribution differences, and the local refinement also gains significant improvement in terms of classification accuracy.

Domain Adaptation via Transfer Component Analysis

Domain adaptation allows knowledge from a source domain to be transferred to a different but related target domain. Intuitively, discovering a good feature representation across domains is crucial. In this paper, we first propose to find such a representation through a new learning method, transfer component analysis (TCA), for domain adaptation. TCA tries to learn some transfer components across domains in a reproducing kernel Hilbert space using maximum mean miscrepancy. In the subspace spanned by these transfer components, data properties are preserved and data distributions in different domains are close to each other. As a result, with the new representations in this subspace, we can apply standard machine learning methods to train classifiers or regression models in the source domain for use in the target domain. Furthermore, in order to uncover the knowledge hidden in the relations between the data labels from the source and target domains, we extend TCA in a semisupervised learning setting, which encodes label information into transfer components learning. We call this extension semisupervised TCA. The main contribution of our work is that we propose a novel dimensionality reduction framework for reducing the distance between domains in a latent space for domain adaptation. We propose both unsupervised and semisupervised feature extraction approaches, which can dramatically reduce the distance between domain distributions by projecting data onto the learned transfer components. Finally, our approach can handle large datasets and naturally lead to out-of-sample generalization. The effectiveness and efficiency of our approach are verified by experiments on five toy datasets and two real-world applications: cross-domain indoor WiFi localization and cross-domain text classification.

Bridging Domains Using World Wide Knowledge for Transfer Learning

A major problem of classification learning is the lack of ground-truth labeled data. It is usually expensive to label new data instances for training a model. To solve this problem, domain adaptation in transfer learning has been proposed to classify target domain data by using some other source domain data, even when the data may have different distributions. However, domain adaptation may not work well when the differences between the source and target domains are large. In this paper, we design a novel transfer learning approach, called BIG (Bridging Information Gap), to effectively extract useful knowledge in a worldwide knowledge base, which is then used to link the source and target domains for improving the classification performance. BIG works when the source and target domains share the same feature space but different underlying data distributions. Using the auxiliary source data, we can extract a ¿bridge¿ that allows cross-domain text classification problems to be solved using standard semisupervised learning algorithms. A major contribution of our work is that with BIG, a large amount of worldwide knowledge can be easily adapted and used for learning in the target domain. We conduct experiments on several real-world cross-domain text classification tasks and demonstrate that our proposed approach can outperform several existing domain adaptation approaches significantly.