Negative Transfer Problem Research Articles

Cross-domain recommender system with embedding- and mapping-based knowledge correlation

A knowledge transfer-based cross-domain recommender system is currently a research hotspot. Existing research has reached a high level of maturity in mining potential knowledge and establishing transfer mechanisms. However, most of them ignore the impact of the dissimilarity of potential knowledge on the transfer performance. Herein, a cross-domain recommender system based on knowledge correlation-induced the embedding and mapping approach is proposed, denoted by KCEM-CDRS. First, we propose a knowledge correlation measure, which captures the consistency of knowledge between the target and source domains to build the bridge for knowledge transfer. Second, we construct a joint matrix triple factorization model to solve the data sparsity in the target domain while introducing graph regularization to solve the problem of negative knowledge transfer. Results of extensive experiments on real Amazon metadata indicate that compared with three existing cross-domain recommendation methods, KCEM-CDRS shows performance improvements of 0.05–9.55 % and 0.02–2.63 % on mean absolute error and root mean square error, respectively. Additionally, the results of the ablation experiments indicate that consideration of the knowledge correlation between domains is beneficial for knowledge transfer when the density of the source domain is rich.

Intelligent fault diagnosis of multi-source cross-machine bearings based on center-weighted optimal transport and class-level alignment domain adaptation

Abstract Most of the current domain adaptation research primarily focuses on the single-source or multi-source domain transfer constructed under different working conditions of the same machine. However, when faced with cross-machine tasks with significant domain discrepancies, forcing the direct feature alignment between source and target domain samples may lead to negative transfer, thereby reducing the model’s diagnostic performance. To overcome the above limitations, this paper proposes a multi-source deep transfer model based on center-weighted optimal transport (CWOT) and class-level alignment domain adaptation. Firstly, to enhance the representation capability of deep features, a multi-structure feature representation network is constructed to enrich the information capacity embedded within the deep features, thereby achieving better domain adaptation capabilities. Then, the local maximum mean discrepancy is introduced to fully exploit fine-grained information and discriminative features among different source domains, minimizing the distribution differences among the source domains to the greatest extent, thus capturing reliable and highly generalized multi-source domain invariant features. On this basis, a CWOT strategy is designed, which comprehensively considers the transport cost of intra-domain uncertainty and inter-domain correlation among samples, establishing a more effective transport between source and target domains, alleviating the problem of sample negative transfer, and improving the model’s cross-machine diagnostic performance. Finally, instance studies are conducted through multiple cross-machine transfer diagnostic tasks, demonstrating that the proposed method outperforms existing domain adaptation methods in terms of diagnostic accuracy and fault transfer capability. This research provides a reliable fault diagnosis method for detecting the health status of rotating machinery equipment, promoting the application of domain adaptation technology in practical industry.

A new indirect transfer fault diagnosis method based on feature separation

The existing fault diagnosis methods based on deep transfer learning achieve domain adaptation by matching shared features between the source and target domains. However, there are different private characteristics between the source domain and the target domain, and traditional cross-domain fault diagnosis algorithms ignore the impact of private attributes in the target domain on classification tasks. Meanwhile, the existence of private features causes a negative transfer problem for domain adaptation methods in domains with large aligned domain gaps. In practical applications, indirect transfer between the source domain and target domain can reduce the limitations of inherent attributes and enhance the guidance of private features in the target domain for classification tasks, which can enrich the transfer of fault knowledge and improve the effectiveness of cross-domain fault diagnosis. Therefore, an indirect transfer fault diagnosis method based on feature separation (ITFS) is proposed. Firstly, an attention-enhanced feature separation framework is proposed, which constructs a set of joint attention convolutional autoencoders (CAEs) to separate features from two domains, avoiding the influence of domain shift in private features. Secondly, to better adapt the two domains, an indirect domain adaptation method based on Gaussian distribution is proposed, which combines Gaussian indirect transfer learning with pseudo-labeling enhancement to reduce the domain gap in the two domains. Six transfer tasks on the PU datasets demonstrate that compared to MCD, the average accuracy of the proposed method is improved by 14.09 %, and it is superior to other popular existing methods. Experiments on three publicly available datasets have shown that this method outperforms other popular existing methods.

Cross-domain continual learning via CLAMP

Artificial neural networks, celebrated for their human-like cognitive learning abilities, often encounter the well-known catastrophic forgetting (CF) problem, where the neural networks lose the proficiency in previously acquired knowledge. Despite numerous efforts to mitigate CF, it remains the significant challenge particularly in complex changing environments. This challenge is even more pronounced in cross-domain adaptation following the continual learning (CL) setting, which is a more challenging and realistic scenario that is under-explored. To this end, this article proposes a cross-domain CL approach making possible to deploy a single model in such environments without additional labelling costs. Our approach, namely continual learning approach for many processes (CLAMP), integrates a class-aware adversarial domain adaptation strategy to align a source domain and a target domain. An assessor-guided learning process is put forward to navigate the learning process of a base model assigning a set of weights to every sample controlling the influence of every sample and the interactions of each loss function in such a way to balance the stability and plasticity dilemma thus preventing the CF problem. The first assessor focuses on the negative transfer problem rejecting irrelevant samples of the source domain while the second assessor prevents noisy pseudo labels of the target domain. Both assessors are trained in the meta-learning approach using random transformation techniques and similar samples of the source domain. Theoretical analysis and extensive numerical validations demonstrate that CLAMP significantly outperforms established baseline algorithms across all experiments by at least 10% margin.

Remaining useful life prediction and cycle life test optimization for multiple-formula battery: A method based on multi-source transfer learning

Achieving highly accurate predictions based on less data with multiple formulations has become a significant challenge. Unlike the traditional prediction model that ignores the similarities and differences between multi-formula batteries, the proposed multi-source transfer RUL prediction is an effective way to jointly transfer several samples and merge degradation information from multi-source domains, thereby taking full advantage of the information. Aiming at the problem of negative transfer and insufficient transfer when multi-source domains are available, dynamic time warping is employed to adaptively select transferable samples, ignoring the local individual differences. The least absolute shrinkage and selection operator are utilized to generate an optimal subset and allocate personalized weights. Then, an optimization strategy for the cycle life tests is proposed by automatically giving a reasonable threshold of stopping the test. The proposed method is verified based on an actual dataset with multiple formulations collected under different test conditions from a battery manufacturer. Experimental results show that the proposed method has high accuracy based on even a small amount of data. The systematic optimization strategy can significantly reduce the time and cost of the test.

Multi-task prediction of hot rolled steel mechanical properties based on CA-MMCE

The accurate prediction of mechanical properties of multi-task hot rolling steel is of great significance to the development of manufacturing, construction, and other industries, and can effectively improve the efficiency of continuous quality control of the hot rolling process. Aiming at the problem that the existing deep models lack interpretability in the field of steel rolling, the hot rolling process was deeply analyzed, and the temperature fluctuation was integrated into the input features as a process variable through feature augmentation to obtain the correlation between features. In order to solve the problem that the conventional Attention mechanism only focuses on the overall global information, a Convolutional Attention mechanism (CA) is proposed to focus on local feature information and improve the ability of the network to extract local key features. At the same time, so as to solve the negative transfer problem of the multi-task model, a Multi-gate Mixture of Convolutional Experts (MMCE) is proposed, which uses multi-gate to adjust the influence of each convolutional expert network on different tasks to capture the correlation between tasks. Thus, it can ensure that multiple tasks can be accurately predicted. Experimental results show that, compared with the existing models, the proposed model achieves excellent prediction performance in multi-task prediction and is advanced.

The Influence of Negative Transfer of Mother Tongue on Oral English in Senior High Schools and Its Countermeasures

This study discusses the phenomenon of negative transfer of mother tongue, which is common in oral English learning in senior high schools. The research topic is divided into two parts: the influence of negative transfer of the mother tongue and its countermeasures, covering vocabulary misuse, phonetic influence and corresponding teaching strategies. Through literature analysis and theoretical comparison, the phenomenon of rigid mother tongue order and mechanical Chinglish is deeply discussed, and some countermeasures are put forward to solve the problem of negative transfer of mother tongue, such as contextualization practice, in-depth understanding of language characteristics and structure, application of cooperation and correction strategies, peer discussion and cooperative activities. The results show that these strategies can effectively reduce the negative impact of the mother tongue on oral English learning. The research conclusion emphasizes that in order to overcome the challenge of negative transfer of mother tongue, diversified and practical teaching methods should be adopted, emphasizing practical communication and cultural integration, aiming at cultivating students' all-round English application ability and cross-cultural communication ability.

MGFKD: A semi-supervised multi-source domain adaptation algorithm for cross-subject EEG emotion recognition

Currently, most models rarely consider the negative transfer problem in the research field of cross-subject EEG emotion recognition. To solve this problem, this paper proposes a semi-supervised domain adaptive algorithm based on few labeled samples of target subject, which called multi-domain geodesic flow kernel dynamic distribution alignment (MGFKD). It consists of three modules: 1) GFK common feature extractor: projects the feature distribution of source and target subjects to the Grassmann manifold space, and obtains the latent common features of the two feature distributions through GFK method. 2) Source domain selector: obtains pseudo-labels of the target subject through weak classifier, finds "golden source subjects" by using few known labels of target subjects. 3) Label corrector: uses a dynamic distribution balance strategy to correct the pseudo-labels of the target subject. We conducted comparison experiments on the SEED and SEED-IV datasets, and the results show that MGFKD outperforms unsupervised and semi-supervised domain adaptation algorithms, achieving an average accuracy of 87.51±7.68% and 68.79±8.25% on the SEED and SEED-IV datasets with only one labeled sample per video for target subject. Especially when the number of source domains is set as 6 and the number of known labels is set as 5, the accuracy increase to 90.20±7.57% and 69.99±7.38%, respectively. The above results prove that our proposed algorithm can efficiently and quickly improve the cross-subject EEG emotion classification performance. Since it only need a small number of labeled samples of new subjects, making it has strong application value in future EEG-based emotion recognition applications.

PAMK: Prototype Augmented Multi-Teacher Knowledge Transfer Network for Continual Zero-Shot Learning.

Continual zero-shot learning (CZSL) aims to develop a model that accumulates historical knowledge to recognize unseen tasks, while eliminating catastrophic forgetting for seen tasks when learning new tasks. However, existing CZSL methods, while mitigating catastrophic forgetting for old tasks, often lead to negative transfer problem for new tasks by over-focusing on accumulating old knowledge and neglecting the plasticity of the model for learning new tasks. To tackle these problems, we propose PAMK, a prototype augmented multi-teacher knowledge transfer network that strikes a trade-off between recognition stability for old tasks and generalization plasticity for new tasks. PAMK consists of a prototype augmented contrastive generation (PACG) module and a multi-teacher knowledge transfer (MKT) module. To reduce the cumulative semantic decay of the class representation embedding and mitigate catastrophic forgetting, we propose a continual prototype augmentation strategy based on relevance scores in PACG. Furthermore, by introducing the prototype augmented semantic-visual contrastive loss, PACG promotes intra-class compactness for all classes across all tasks. MKT effectively accumulates semantic knowledge learned from old tasks to recognize new tasks via the proposed multi-teacher knowledge transfer, eliminating the negative transfer problem. Extensive experiments on various CZSL settings demonstrate the superior performance of PAMK compared to state-of-the-art methods. In particular, in the practical task-free CZSL setting, PAMK achieves impressive gains of 3.28%, 3.09% and 3.71% in mean harmonic accuracy on the CUB, AWA1, and AWA2 datasets, respectively.

Dynamics simulation-driven fault diagnosis of rolling bearings using security transfer support matrix machine

Transfer fault diagnosis holds paramount significances in safeguarding the reliability and safety of rolling bearings. However, the current studies require massive experiment or field data in the source domain. Besides, sparse data (only single or several samples for each fault) and strong-noise data in the target domain are prone to cause the negative transfer problem. Importantly, most transfer fault diagnosis methods conduct fault classification in vector space, so the learning ability is weak for matrix-form fault features such as 2D time-frequency images. To address these issues, this paper proposes a novel dynamics simulation-driven fault diagnosis framework with security transfer support matrix machine (STSMM). In this framework, a high-fidelity bearing dynamic model is designed to generate sufficient source-domain data, which greatly reduces the acquisition cost of real data. A new matrix-form transfer learning model, i.e., STSMM, is proposed to effectively utilize the structural information contained in matrix data and achieve the simulation-to-real transfer of fault knowledge. Besides, a security transfer strategy is embedded in STSMM to improve the cross-domain diagnosis performance and theoretically avoid the negative transfer problem caused by sparse data and strong-noise data in the target domain. Experiment results demonstrate the effectiveness and superiority of the proposed framework.

Multi-representation transferable attention network for remaining useful life prediction of rolling bearings under multiple working conditions

Remaining useful life (RUL) prediction improves equipment reliability by avoiding unexpected breakdowns. However, different operating conditions may cause the performance degradation of the prediction model due to the domain-shift problem. Therefore, a novel method based on the multiple representation transferable attention network (MRTAN) is proposed in this paper for RUL prediction under multiple working conditions. In the MRTAN-based method, multiple convolutional modules are first used to learn the degradation features. Next, the multi-representation adaptation module is utilized to further mine the multiple domain-invariant representations. Then, during the training process, the transferable attention module can activate the representation with high transferability dynamically. Finally, the prognostic model is optimized by multiple optimization objectives and the back-propagation algorithm. Besides, multiple cross-domain RUL prediction tasks are employed to validate the effectiveness of the MRTAN-based model. Experiments demonstrate that the proposed method can provide better prognostic performance and avoid the negative transfer problem.

Selective transfer subspace learning for small-footprint end-to-end cross-domain keyword spotting

In small-footprint end-to-end keyword spotting, it is often expensive and time-consuming to acquire sufficient labels in various speech scenarios. To overcome this problem, transfer learning leverages the rich knowledge of the auxiliary domain to annotate the unlabeled target data. However, most existing transfer learning methods typically learn a domain-invariant feature representation while ignoring the negative transfer problem. In this paper, we propose a new and general cross-domain keyword spotting framework called selective transfer subspace learning (STSL) that avoid negative transfer and dramatically improve the accuracy for cross-domain keyword spotting by actively selecting appropriate source samples. Specifically, STSL first aligns geometrical relationship and weighted distribution discrepancy to learn a domain-invariant projection subspace. Then, it actively selects appropriate source samples that are similar to the target domain for transfer learning to avoid negative transfer. Finally, we formulate a minimization problem that alternately optimizes the projection subspace and source active selection, giving an effective optimization. Experimental results on 10 groups of cross-domain keyword spotting tasks show that our STSL outperforms some state-of-the-art transfer learning methods and no transfer learning methods.

Transfer Learning Algorithm With Knowledge Division Level.

One of the major challenges of transfer learning algorithms is the domain drifting problem where the knowledge of source scene is inappropriate for the task of target scene. To solve this problem, a transfer learning algorithm with knowledge division level (KDTL) is proposed to subdivide knowledge of source scene and leverage them with different drifting degrees. The main properties of KDTL are three folds. First, a comparative evaluation mechanism is developed to detect and subdivide the knowledge into three kinds-the ineffective knowledge, the usable knowledge, and the efficient knowledge. Then, the ineffective and usable knowledge can be found to avoid the negative transfer problem. Second, an integrated framework is designed to prune the ineffective knowledge in the elastic layer, reconstruct the usable knowledge in the refined layer, and learn the efficient knowledge in the leveraged layer. Then, the efficient knowledge can be acquired to improve the learning performance. Third, the theoretical analysis of the proposed KDTL is analyzed in different phases. Then, the convergence property, error bound, and computational complexity of KDTL are provided for the successful applications. Finally, the proposed KDTL is tested by several benchmark problems and some real problems. The experimental results demonstrate that this proposed KDTL can achieve significant improvement over some state-of-the-art algorithms.

Dynamic Semi-Supervised Federated Learning Fault Diagnosis Method Based on an Attention Mechanism.

In cases where a client suffers from completely unlabeled data, unsupervised learning has difficulty achieving an accurate fault diagnosis. Semi-supervised federated learning with the ability for interaction between a labeled client and an unlabeled client has been developed to overcome this difficulty. However, the existing semi-supervised federated learning methods may lead to a negative transfer problem since they fail to filter out unreliable model information from the unlabeled client. Therefore, in this study, a dynamic semi-supervised federated learning fault diagnosis method with an attention mechanism (SSFL-ATT) is proposed to prevent the federation model from experiencing negative transfer. A federation strategy driven by an attention mechanism was designed to filter out the unreliable information hidden in the local model. SSFL-ATT can ensure the federation model's performance as well as render the unlabeled client capable of fault classification. In cases where there is an unlabeled client, compared to the existing semi-supervised federated learning methods, SSFL-ATT can achieve increments of 9.06% and 12.53% in fault diagnosis accuracy when datasets provided by Case Western Reserve University and Shanghai Maritime University, respectively, are used for verification.

The Negative Transfer Effect on the Neural Machine Translation of Egyptian Arabic Adjuncts into English: The Case of Google Translate

Parallel corpora for low-resource Arabic dialects and English are limited and small-scale, and most neural machine translation models, including Google Translate, rely mainly on parallel corpora of standard Arabic and English to train for dialectal Arabic translation. A model well trained to translate to and from standard Arabic is believed to efficiently translate dialectal Arabic, given their similarities. This study demonstrates the impact of not using large-scale, dialect-specific parallel corpora by quantitatively and qualitatively analyzing the performance of Google Translate in translating Egyptian Arabic adjuncts. Compared to human reference translation, Google Translate achieved a low BLEU score of 14.69. Qualitative analysis showed that reliance on standard Arabic parallel corpora caused a negative transfer problem manifested in the literal translation of idiomatic adjuncts, the misinterpretation of dialectal adjuncts as main clause constituents, the translation of dialectal adjuncts after orthographically similar standard Arabic words, and the use of standard Arabic common lexical meanings to translate dialect-specific adjuncts. This study’s findings will be relevant for researchers interested in dialectal Arabic neural machine translation and has implications for investment in the development of large-scale, dialect-specific corpora to better process the peculiarities of Arabic dialects and reduce the effect of negative transfer from standard Arabic.

Multi-scale fusion for few-shot remote sensing image classification

ABSTRACT Few-shot image classification is currently valuable research in the field of remote sensing image applications. The difficulty in obtaining training data for a large number of labelled remote sensing images leads to difficulties in applying traditional deep learning-based remote sensing image classification methods. Existing few-shot remote sensing image classification methods usually use a large number of base datasets to train network models in the pre-training phase and perform few-shot classification tasks after model fine-tuning on new datasets in the meta-testing phase. However, an insufficient number of remote sensing images lead to inaccurate features being extracted by the model, and the large amount of new class data for fine-tuning is also not easily accessible. In addition, using a pre-trained model to extract features from the new dataset will create a”negative transfer” problem. In this paper, we aim to address the above challenges in two ways. First, we use a subset of ImageNet, a readily available few-shot natural image dataset, for model pre-training, and eliminate the fine-tuning operation to simulate a real-world application scenario where remote sensing data is extremely scarce. Second, due to the significant differences in scale and style between the ImageNet dataset and the remote sensing dataset, which lead to a serious negative transfer problem in the model, we design a multi-scale feature fusion module to comprehensively decide the labels of query samples considering all scales to compensate for the scale differences and alleviate the”negative transfer” problem. We conducted experiments on four benchmark remote sensing datasets and achieved satisfactory performance.

DITN: User’s indirect side-information involved domain-invariant feature transfer network for cross-domain recommendation

The cross-domain recommendation aims to make more accurate personalized recommendations by transferring user preferences from a relatively rich interactive information domain to a sparse data domain. Traditional methods mostly only use direct side-information to improve the generalization of user and item representations which is usually limited or introduce indirect side-information without fully considering the problems of noise reduction and balanced fusion. For these two considerations, we propose a User’s Indirect Side-information Involved Domain-invariant Feature Transfer Network for Cross-Domain Recommendation (DITN) to mutually suppress and balanced fuse direct and indirect side-information. To effectively suppress the noise caused by adding side-information, we define a group of global property representations to guide the user’s feature extraction of direct and indirect side-information and adopt the co-attention mechanism to balanced fuse two kinds of side-information features, which improves the quality of the user’s preferences learning. Then we propose a novel user’s feature disentangle module based on the VAE framework to extract the invariant and the specific features between domains and enhance feature disentangling by utilizing mutual information to reduce the negative transfer problem. We have conducted sufficient experiments on 6 domains and 3 scenarios which include 2,408,007 interactive information from the Amazon dataset to demonstrate the significant performance of our proposed model. The results show that DITN effectively enhances the rating prediction accuracy and outperforms state-of-the-art recommendation methods.

Cross-Domain Sentiment Analysis Based on Feature Projection and Multi-Source Attention in IoT.

Social media is a real-time social sensor to sense and collect diverse information, which can be combined with sentiment analysis to help IoT sensors provide user-demanded favorable data in smart systems. In the case of insufficient data labels, cross-domain sentiment analysis aims to transfer knowledge from the source domain with rich labels to the target domain that lacks labels. Most domain adaptation sentiment analysis methods achieve transfer learning by reducing the domain differences between the source and target domains, but little attention is paid to the negative transfer problem caused by invalid source domains. To address these problems, this paper proposes a cross-domain sentiment analysis method based on feature projection and multi-source attention (FPMA), which not only alleviates the effect of negative transfer through a multi-source selection strategy but also improves the classification performance in terms of feature representation. Specifically, two feature extractors and a domain discriminator are employed to extract shared and private features through adversarial training. The extracted features are optimized by orthogonal projection to help train the classification in multi-source domains. Finally, each text in the target domain is fed into the trained module. The sentiment tendency is predicted in the weighted form of the attention mechanism based on the classification results from the multi-source domains. The experimental results on two commonly used datasets showed that FPMA outperformed baseline models.

Deep Multiadversarial Conditional Domain Adaptation Networks for Fault Diagnostics of Industrial Equipment

Deep learning methods of fault diagnostics require the availability of a large amount of labeled data for training, i.e., signal values corresponding to known degradation and fault states. Furthermore, the distribution of the training data should be similar to that of the (test) data collected in the field. Since these conditions are typically not satisfied in most industrial applications, this work develops a deep multi-adversarial conditional domain adaptation network. The main original contribution lies in a novel method to align, class by class, the weighted marginal data distributions using multiple domain discriminators. The network allows overtaking the classification underperformance caused by the problem of negative transfer, which is typically encountered when only few training data of some of the classes are available. The proposed method is shown to outperform other state-of-the-art methods on two cross-domain fault diagnostic case studies, verified by applying Friedman and Holm post-hoc tests.

Prediction of common labels for universal domain adaptation

Universal domain adaptation (UniDA) is an unsupervised domain adaptation that selectively transfers the knowledge between different domains containing different label sets. However, the existing methods do not predict the common labels of different domains and manually set a threshold to discriminate private samples, so they rely on the target domain to finely select the threshold and ignore the problem of negative transfer. In this paper, to address the above problems, we propose a novel classification model named Prediction of Common Labels (PCL) for UniDA, in which the common labels are predicted by Category Separation via Clustering (CSC). It is noted that we devise a new evaluation metric called category separation accuracy to measure the performance of category separation. To weaken negative transfer, we select source samples by the predicted common labels to fine-tune model for better domain alignment. In the test process, the target samples are discriminated by the predicted common labels and the results of clustering. Experimental results on three widely used benchmark datasets indicate the effectiveness of the proposed method.