Multi-source Adaptation Research Articles

Multi-source transfer learning for facial emotion recognition using multivariate correlation analysis.

Deep learning techniques have proven to be effective in solving the facial emotion recognition (FER) problem. However, it demands a significant amount of supervision data which is often unavailable due to privacy and ethical concerns. In this paper, we present a novel approach for addressing the FER problem using multi-source transfer learning. The proposed method leverages the knowledge from multiple data sources of similar domains to inform the model on a related task. The approach involves the optimization of aggregate multivariate correlation among the source tasks trained on the source dataset, thus controlling the transfer of information to the target task. The hypothesis is validated on benchmark datasets for facial emotion recognition and image classification tasks, and the results demonstrate the effectiveness of the proposed method in capturing the group correlation among features, as well as being robust to negative transfer and performing well in few-shot multi-source adaptation. With respect to the state-of-the-art methods MCW and DECISION, our approach shows an improvement of 7% and [Formula: see text]15% respectively.

ESG information extraction with cross-sectoral and multi-source adaptation based on domain-tuned language models

Along with future sustainability, factors such as environmental protection, co-prosperity, and ethical management are also gaining attention, resulting in the increased significance of the non-financial performance of companies. Accordingly, many institutions are evaluating the non-financial activities of companies using environmental, social, and governance (ESG) as evaluation indicators. However, the use of the current ESG rating as a reliable indicator has a limitation as the criteria are different for each evaluation institution and the ESG rating for the same company may differ when provided by different evaluation institutions. Accordingly, it is important to directly extract non-financial information generated by various media to provide objective ESG information, but the studies that focus on the need for automatic and direct extraction and classification of ESG information are limited. Therefore, this paper proposes a classifier that can discriminate ESG information. To train the ESG classifier, a dataset was constructed by manually labeling ESG data, and the ESG classifier recorded a classification accuracy of 86.66% for the 4-class classification problem of the constructed dataset. In addition, three application experiments were conducted to verify the usability of the proposed model. First, the results confirmed that the ESG classifier showed a significant performance of 83.96% accuracy for sectors not included in the training data. Second, the results qualitatively confirmed that the proposed model properly extracts ESG-related information from multi-source text data. Finally, the ESG dataset was additionally constructed through the pseudo-labeling technique, and the performance improvement of the data augmentation and classifier was verified. The ESG classifier proposed in this paper will provide appropriate ESG information to stakeholders. In future studies, we intend to extract and classify ESG-related information from more diverse types of data through an advanced ESG classifier. Further, we will use the information for investigating methods for reflecting the non-financial performance of companies in ESG ratings.

Multisource-Domain Adaptation-Based Damage Detection Method for Air-Conditioning Condenser Using Laser Ultrasonic Guided Wave

Air conditioning condenser (ACC) is a key component of air-conditioning equipment. ACC tube leakage damage will cause huge economic losses and energy waste. The laser ultrasonic guided wave detection method is suitable for ACC damage detection due to its high sensitivity and non-contact advantages. However, the ACC tube size is small and the weld joint is not standardized, which makes it difficult for the laser to aim at the tube. And the detection signals generated by laser excitation at different laser incident angles are different. In this paper, the Unsupervised Multi-source Adaptation Adversarial Network (UMAAN) is proposed to improve the generalization ability of the ACC detection model. Firstly, the feature diagrams of the signal are obtained by time-frequency analysis, and the diagrams are set to different source and target domains according to the different incident angles of laser excitation. Then, the shared feature extractor is used to obtain features of all samples. And the model parameters are optimized by adversarial network to make the features distance of the target and source domains closer. Among them, the adaptive weight coefficients are designed to adjust the loss value weights of the different source domains. And the label error weight factor is designed to improve the damaged signals detection accuracy by adjusting the loss weight of damaged samples and normal samples. Finally, the damage detection experiments of the ACC are carried out. The experimental results illustrate that the proposed UMAAN can achieve high-precision detection of ACC damages.

Dual contrastive universal adaptation network for multi-source visual recognition

This paper explores recently proposed Universal Multi-source Domain Adaptation (UniMDA) task. UniMDA task is different from existing domain adaptation (DA) tasks, e.g., Multi-source DA, Closed set DA or Universal DA. UniMDA not only handles multi-source issue, but also needs no prior knowledge about the overlap between the target and source label sets. UniMDA task has three challenges: (i) Domain shift issue among the multiple source domains. (ii) Domain shift issue between target and source domains. (iii) Category shift between the target and each source. Towards tackling the challenges, we formulate a universal multi-source adaptation network termed as Multi-Source Dual Contrastive Network (MSDCN), including a transferability rule and contrastive module. In addition, to handle multi-source scenario, we propose (1) pairwise similarities maximization over the examples from multiple source domains, and (2) alternative optimization strategy for training the ensemble of multiple source classifiers end-to-end. The proposed method can handle UniMDA scenario generally, where the label set in each source domain may be different from the target domain, while maintaining the complexity of the method in different DA scenarios. Experiments are conducted on several real-world multi-source benchmarks. The results show that MSDCN could work stably, and exceed the state-of-the-art performance against existing domain adaptation algorithms.

Robust Latent Multi-Source Adaptation for Encephalogram-Based Emotion Recognition.

In practical encephalogram (EEG)-based machine learning, different subjects can be represented by many different EEG patterns, which would, in some extent, degrade the performance of extant subject-independent classifiers obtained from cross-subjects datasets. To this end, in this paper, we present a robust Latent Multi-source Adaptation (LMA) framework for cross-subject/dataset emotion recognition with EEG signals by uncovering multiple domain-invariant latent subspaces. Specifically, by jointly aligning the statistical and semantic distribution discrepancies between each source and target pair, multiple domain-invariant classifiers can be trained collaboratively in a unified framework. This framework can fully utilize the correlated knowledge among multiple sources with a novel low-rank regularization term. Comprehensive experiments on DEAP and SEED datasets demonstrate the superior or comparable performance of LMA with the state of the art in the EEG-based emotion recognition.

PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains

Abstract Natural Language Processing algorithms have made incredible progress, but they still struggle when applied to out-of-distribution examples. We address a challenging and underexplored version of this domain adaptation problem, where an algorithm is trained on several source domains, and then applied to examples from unseen domains that are unknown at training time. Particularly, no examples, labeled or unlabeled, or any other knowledge about the target domain are available to the algorithm at training time. We present PADA: An example-based autoregressive Prompt learning algorithm for on-the-fly Any-Domain Adaptation, based on the T5 language model. Given a test example, PADA first generates a unique prompt for it and then, conditioned on this prompt, labels the example with respect to the NLP prediction task. PADA is trained to generate a prompt that is a token sequence of unrestricted length, consisting of Domain Related Features (DRFs) that characterize each of the source domains. Intuitively, the generated prompt is a unique signature that maps the test example to a semantic space spanned by the source domains. In experiments with 3 tasks (text classification and sequence tagging), for a total of 14 multi-source adaptation scenarios, PADA substantially outperforms strong baselines.1

Multi-Source and Multi-Representation Adaptation for Cross-Domain Electroencephalography Emotion Recognition.

Due to the non-invasiveness and high precision of electroencephalography (EEG), the combination of EEG and artificial intelligence (AI) is often used for emotion recognition. However, the internal differences in EEG data have become an obstacle to classification accuracy. To solve this problem, considering labeled data from similar nature but different domains, domain adaptation usually provides an attractive option. Most of the existing researches aggregate the EEG data from different subjects and sessions as a source domain, which ignores the assumption that the source has a certain marginal distribution. Moreover, existing methods often only align the representation distributions extracted from a single structure, and may only contain partial information. Therefore, we propose the multi-source and multi-representation adaptation (MSMRA) for cross-domain EEG emotion recognition, which divides the EEG data from different subjects and sessions into multiple domains and aligns the distribution of multiple representations extracted from a hybrid structure. Two datasets, i.e., SEED and SEED IV, are used to validate the proposed method in cross-session and cross-subject transfer scenarios, experimental results demonstrate the superior performance of our model to state-of-the-art models in most settings.

Robust multi-source co-adaptation with adaptive loss minimization

While multi-source adaptation has recently received much attention due to its effectiveness, limited effort has been made on improving its performance by leveraging some shared knowledge among sources and features. To this end, we propose in this paper a robust Multi-source co-Adaptation framework by mining diverse Correlation Information (MACI) among domains as well as features via joint adaptive σ-norm and correlation metric regularization. Concretely, multiple domain-invariant classification functions for different source adaptation objects are co-learned in a joint framework by minimizing both statistical and semantic distribution discrepancy between source and target domains, which enables MACI to utilize the correlated knowledge among multiple sources by exploiting the developed correlation metric function. Under this framework, MACI can preserve the original geometrical structure information as well as be robust to some noises or outliers existed in domains by employing local graph Laplacian and adaptive regression scheme. Furthermore, we also extend MACI to domain generalization scenario when the target domain of interest is unavailable in the training stage. An efficient iterative algorithm is proposed to optimize MACI (also applied to its domain generalization version), whose convergence is theoretically guaranteed. Comprehensive experimental evidence on a large number of visual datasets verifies the effectiveness of the proposed framework.

Universal multi-Source domain adaptation for image classification

Unsupervised domain adaptation (DA) enables intelligent models to learn transferable knowledge from a labeled source domain and adapt to a similar but unlabeled target domain. Studies showed that knowledge could be transferred from one source domain to another unknown target domain, called Universal DA (UDA). However, there is often more than one source domain in the real-world application to be exploited for DA. In this paper, we formally propose a more general domain adaptation setting for image classification, universal multi-source DA (UMDA), where the label sets of multiple source domains can be different, and the label set of the target domain is completely unknown. The main challenge in UMDA is to identify the common label set among each source and target domain and keep the model scalable as the number of source domains increases. In the face of this challenge, we propose a universal multi-source adaptation network (UMAN) to solve the DA problem without increasing the complexity of the model in various UMDA settings. In UMAN, the reliability of each known class belonging to the common label set is estimated via a novel pseudo-margin vector and its weighted form, which helps adversarial training better align the distributions of multiple source domains and target domain. Moreover, the theoretical guarantee for UMAN is also provided. Massive experimental results show that existing UDA and multi-source DA (MDA) methods cannot be directly deployed to UMDA, and the proposed UMAN achieves the state-of-the-art performance in various UMDA settings.

A deep multi-source adaptation transfer network for cross-subject electroencephalogram emotion recognition

In real-world application of affective brain–computer interface (aBCI), individual differences across subjects and non-stationary characteristics of electroencephalogram (EEG) signals can cause data bias. Moreover, for new specific subject, the size of sample data is very small compared to that of existing subjects, which easily leads to overfitting in deep neural network training and reduces generalization performance of the network. In this paper, the deep multi-source adaptation transfer network (DMATN) is proposed for the new subjects in aBCI. In DMATN, the multi-source selection is employed to obtain the portion of existing EEG data mostly correlated with new subject and to decrease by two-fifth source data. To explore domain-invariant structures, deep adaptation network is used to map correlated source domain and the target domain (new subject) into reproducing kernel Hilbert space (RKHS) optimized by the multiple kernel variant of maximum mean discrepancies (MK-MMD). To more precisely predict the emotional state of the new subject, domain discriminator is applied in DMATN to make the data distribution of the two domains more similar. Finally, across-subject experiments on SEED dataset are conducted to evaluate the proposed method. The experimental results show that DMATN model can achieve the state-of-the-art performance of 84.46%, 83.32% and 84.90% in three sessions, respectively. It also shows great time efficiency in applications of aBCI.

Deep Transfer Learning based Multisource Adaptation Fault Diagnosis Network for Industrial Processes

Abstract In industrial processes, there are generally multiple data sources generated from different working conditions, which can provide different fault diagnosis knowledge to the target application. In this paper, a multisource adaptation diagnosis network (MADN) method is proposed to transfer the diagnostic knowledge existed in multiple sources to the target. First, a stacked-autoencoder based feature generator is pretrained to extract feature representations from the process data acquired from diverse working conditions. Then, domain discriminators are developed to reduce the distribution discrepancy between the target domain and each of the sources in an adversarial way. The domain discrimination ability, on the other hand, also reveals the different importance of the source domains. Thus, the fault classifiers can be assembled to identify the fault types of the unlabeled target data. The superiority of the proposed method is verified using a real-world process.

Duplex adversarial networks for multiple-source domain adaptation

Domain adaptation is a powerful tool for transferring the knowledge of the source domain with sufficient annotations for target tasks. However, most existing domain adaptation methods focus on the single-source–single-target adaptation setting, which ignore that existing different but related multiple source domains are available for target tasks. In this paper, we propose a novel theoretical generalization bound for multiple source domain adaptation, which averages the multiple source hypothesis risks by using HΔH-distance with an assumption that the target hypothesis is the combination of all the source hypotheses. Following this theoretical result, we propose a new algorithm called Duplex Adversarial Networks for Multi-source Adaptation (DAN_MA): one adversarial network is utilized for feature generator to learn domain invariant representations between source and target domains with a domain classifier for each pair of source and target; the other is utilized to eliminate the domain characteristics among different sources by making target samples close to each source task-specific decision boundaries. The shared feature generator aims to learn target representations which are invariant to the multiple domain shifts while still close to the source decision boundaries. Extensive experiments show that our method outperforms other domain adaptation methods on three public datasets.

Multiple-source domain adaptation with generative adversarial nets

Current unsupervised domain adaptation (UDA) methods based on GAN (Generative Adversarial Network) architectures assume that source samples arise from a single distribution. These methods have shown compelling results by finding the transformation between source and target domains to reduce the distribution divergence. However, the one-to-one assumption renders the existing GAN-based UDA methods ineffective in a more realistic scenario that source samples are typically collected from diverse sources. In this paper, we present a novel GAN-enabled framework, which we call Multi-Source Adaptation Network (MSAN), for multiple-source domain adaptation (MDA) to mitigate the domain shifts between multiple source domains and the target domain. The proposed framework consists of multiple GAN architectures to learn bidirectional transformations between the source domains and the target domain efficiently and simultaneously. Technically, we introduce a joint feature space to guide the multi-level consistency constraints across all the transformations, in order to preserve the domain-invariant pattern and endow the discriminative power for the unlabeled target samples simultaneously during the adaptation. Moreover, the proposed model can naturally be used to enlarge the target dataset by utilizing the synthetic target images (with ground-truth labels from different source domains) and the pseudo-labeled target images, thereby allowing constructing the target-specific classifier in an unsupervised manner. Experiments demonstrate that our models exceed state-of-the-art results for MDA tasks on several benchmark datasets.

Discriminative multi-source adaptation multi-feature co-regression for visual classification

Learning an effective visual classifier from few labeled samples is a challenging problem, which has motivated the multi-source adaptation scheme in machine learning. While the advantages of multi-source adaptation have been widely recognized, there still exit three major limitations in extant methods. Firstly, how to effectively select the discriminative sources is yet an unresolved issue. Secondly, multiple different visual features on hand cannot be effectively exploited to represent a target object for boosting the adaptation performance. Last but not least, they mainly focus on either visual understanding or feature learning independently, which may lead to the so-called semantic gap between the low-level features and the high-level semantics. To overcome these defects, we propose a novel Multi-source Adaptation Multi-Feature (MAMF) co-regression framework by jointly exploring multi-feature co-regression, multiple latent spaces learning, and discriminative sources selection. Concretely, MAMF conducts the multi-feature representation co-regression with feature learning by simultaneously uncovering multiple optimal latent spaces and taking into account correlations among multiple feature representations. Moreover, to discriminatively leverage multi-source knowledge for each target feature representation, MAMF automatically selects the discriminative source models trained on source datasets by formulating it as a row-sparsity pursuit problem. Different from the state-of-the-arts, our method is able to adapt knowledge from multiple sources even if the features of each source and the target are partially different but overlapping. Experimental results on three challenging visual domain adaptation tasks consistently demonstrate the superiority of our method in comparison with the related state-of-the-arts.

Robust multi-model adaptation regression with local feature space representation

Learning visual models of object categories generally requires a large number of training examples. We show in this paper that it is possible to uncover much discriminative information about a visual category from a small number of examples with just fewlabeled data. The key insight is that, rather than learning from scratch with the original feature representation, we learn an optimal target model with the local feature space representations by leveraging some prior models pre-trained on other related datasets, no matter how different they might be. This target model can be obtained by learning a robust regression scheme with a multi-source adaptation regularization term, which is formally formulated as a Robust Multi-model Adaptation Regression (RMAR) framework with local feature space representations of the target data. This framework is composed of three core components: (1) local consistency Laplacian regularization by constructing a self-adaptive affinity graph based on local feature space representation (LFSR), which aims to augment the discriminative space of the target domain; (2) robust regression learning scheme with feature selection based on l2,1-norm minimization by explicitly considering the regression noise/outlier as well as the regression residual; and (3) scatter constrained multi-model adaptation regularization for generalizing the established robust regression framework to exploit the prior models, under the assumption that multiple auxiliary discriminative models can help the semi-supervised learning from few labeled samples. Lastly, we further propose an effective extension to RMAR, i.e., ensemble LFSR-graph Laplacians regularization framework. The optimization algorithms for RMAR and its extension can be efficiently solved by the alternating iterative strategy, and the iteration convergence can be theoretically guaranteed. Experiments over several real datasets show the promising performance of our methods compared with several representative state-of-the-art works.

Latent multi-feature co-regression for visual recognition by discriminatively leveraging multi-source models

Learning a visual category with few labeled samples is a challenging problem in machine learning, which has motivated the multi-source adaptation learning technique, which exploits to transfer multiple prior discriminative models to target domain. Under this paradigm, however, different visual features at hand cannot be effectively exploited to represent a target object with versatility for boosting the adaptation performance. Besides, existing multi-source adaptation schemes mostly focus on either visual understanding or feature learning, independently. This may lead to the so-called semantic gap between the low-level features and the high-level semantics. Last but not the least, how to discriminatively select the prior models is yet another unresolved issue. To address these issues, we propose a novel co-regression framework with Multi-Source adaptation Multi-Feature Representation (MSMFR) for visual recognition, which jointly explores robust multi-feature co-regression, latent space learning, and representative sources selection, by integrating them into a unified framework for joint visual understanding and feature learning. Specifically, MSMFR conducts the multi-feature co-regression by simultaneously uncovering multiple latent spaces and minimizing the co-regression residual by taking correlations among multiple feature representations into account. Furthermore, MSMFR also automatically selects the representative (or discriminative) source models for each target feature representation via formulating a row-sparsity pursuit problem. The validity of our method is examined by three challenging visual domain adaptation tasks on several benchmark datasets, which demonstrate the superiority of our method in comparison with several state-of-the-arts.

Sparse feature space representation: A unified framework for semi-supervised and domain adaptation learning

In a semi-supervised domain adaptation (DA) task, one has access to only few labeled target examples. In this case, the success of DA needs the effective utilization of a large number of unlabeled target data to extract more discriminative information that is useful for generalization. To this end, we exploit in this paper the feature space embeddings of the target data as well as multi-source prior models to augment the discrimination space for the target function learning. Therefore, we propose a novel multi-source adaptation learning framework based on Sparse Feature Space Representation (SFSR), or called SFSR-MSAL for short. Specifically, the SFSR algorithm is first presented for the further construction of robust graph, on which the discriminative information can be smoothly propagated into the unlabeled target data by additionally incorporating the geometric structure of the target data. Considering the robustness in the semi-supervised DA, we replace the traditional l2-norm based least squares regression with the l2, 1-norm sparse regression, and then construct the SFSR-graph based semi-supervised DA framework with multi-source adaptation constraints. Our framework is universal and can be easily degraded into semi-supervised learning by just tuning the regularization parameter. Moreover, to select the discriminative SFSR-graph Laplacians, we also introduce the ensemble SFSR-graph Laplacians regularization into SFSR-MSAL, thus further improving the performance of SFSR-MSAL. The validity of our methods including semi-supervised and DA learning are examined by several visual recognition tasks on some benchmark datasets, which demonstrate the superiority of our methods in comparison with other related state-of-the-art algorithms.

Multi-source adaptation embedding with feature selection by exploiting correlation information

While feature selection has recently received much research attention, less or limited effort has been made on improving the performance of feature selection by leveraging the shared knowledge from other related domains. Besides, multi-source adaptation embedding by exploiting the correlation information among domain features and distributions has long been largely unaddressed. To this end, we propose in this paper a robust Multi-source Adaptation Embedding framework with Feature Selection (MAEFS) by exploiting the correlation information via joint l2, 1-norm and trace-norm regularization, and apply it to cross-domain visual recognition. Specifically, to uncover cross-domain invariant subspaces by minimizing the distribution discrepancy between source and target domains, instead of evaluating the importance of each feature individually, MAEFS selects features in a collaborated mode for considering the correlation information among features. Furthermore, multiple feature selection functions for different source adaptation objects are simultaneously learned in a joint framework, which enables MAEFS to utilize the correlated knowledge among multiple source domains via trace-norm regularization, thus facilitating domain invariant embedding. Besides, by employing graph embedding and sparse regression scheme via l2, 1-norm minimization, MAEFS can preserve the original geometrical structure information as well as be robust to some noises or outliers existed in domains. Finally, an efficient iterative algorithm is proposed to optimize MAEFS, whose convergence is theoretically guaranteed. Comprehensive experimental evidence on a large number of visual datasets verifies the effectiveness of the proposed framework.

Coupled local–global adaptation for multi-source transfer learning

This paper presents a novel unsupervised multi-source domain adaptation approach, named as coupled local–global adaptation (CLGA). At the global level, in order to maximize the adaptation ability, CLGA regards multiple domains as a unity, and jointly mitigates the gaps of both marginal and conditional distributions between source and target dataset. At the local level, with the intention of maximizing the discriminative ability, CLGA investigates the relationship among distinctive domains, and exploits both class and domain manifold structures embedded in data samples. We formulate both local and global adaptation in a concise optimization problem, and further derive an analytic solution for the objective function. Extensive evaluations verify that CLGA performs better than several existing methods not only in multi-source adaptation tasks but also in single source scenarios.

Robust multi-source adaptation visual classification using supervised low-rank representation

How to guarantee the robustness of multi-source adaptation visual classification is an important challenge in current visual learning community. To this end, we address in this paper the problem of robust visual classification with few labeled samples from the target domain of interest by leveraging multiple prior source models. Motivated by the recent success of low rank representation, we formulate this problem as a robust multi-source adaptation visual classification (RMAVC) model with supervised low rank representation by combining the strength of discriminative information from the target domain and the prior models from multiple source domains. Specifically, we propose a joint supervised low rank representation and multi-source adaptation visual classification framework, which achieves dual goals of finding the most discriminative low rank representation and multi-source adaptation classifier parameters for the target domain. While it is showed in this paper that the proposed RMAVC framework is effective and can produce high accuracy on several tasks of multi-source adaptation visual classification, this framework fails to consider the local geometrical structure of the target data and the heterogeneousness among multiple source domains. Hence, under this framework, we further present two effective extensions or variants, i.e., RMAVCK and RMAVC_FM, by exploiting multiple kernel trick and flexible manifold regularization, respectively. The proposed framework and its variants are robust for classifying visual objects accurately and the experimental results demonstrate the effectiveness of our methods on several types of image and video datasets.