Identifying Solitary Granulomatous Nodules from Solid Lung Adenocarcinoma: Exploring Robust Image Features with Cross-Domain Transfer Learning.

The focus of Source-free Unsupervised Domain Adaptation (SFUDA) is to effectively transfer a well-trained model from the source domain to an unlabelled target domain. During the target domain adaptation, the source domain data is no longer accessible. Prevalent methodologies attempt to synchronize the data distributions between the source and target domains, utilizing pseudo-labels to impart categorical information, which has made some progress in improving the model’s performance. However, performance impairments persist due to the introduction of learning bias from the source model and the impact of noisy pseudo-labels generated for the target domain. In this research, we reveal that the central cause for feature misalignment during domain transition is the learning bias, which is generated by the discrepancy of information between source and target domain data. The source domain data may contain distinguishable features that do not appear on the target domain, which causes the pre-trained source model to fail to work during domain adaptation. To overcome the information discrepancy, we propose a Prototypical Feature Compensation (PFC) Network. The network extracts representative feature maps of the source domain. Then use them to minimize the discrepancy information in the target domain feature maps. This mechanism facilitates feature alignment across different domains, allowing the model to generate more accurate categorical data through pseudo-labelling. The experimental results and ablation studies demonstrate exceptional performance on three SFUDA datasets and provide evidence of the proposed PFC method’s ability to adjust the feature distribution of both source and target domain data, ensuring their overlap in the latent space.

Reducing the effect of sample bias for small data sets with double‐weighted support vector transfer regression

Source-Free Domain Adaptation (SFDA) aims to address the challenge of effectively transferring a source domain model to a target domain when the target domain data is unlabeled and the source domain data is unavailable. Traditional Unsupervised Domain Adaptation (UDA) methods rely on simultaneous access to both source and target domain data. However, in many practical scenarios, such as medical data privacy protection or resource-constrained devices, direct access to source domain data is not feasible. SFDA leverages only a pre-trained source domain model and unlabeled target domain data to update the model, avoiding the direct use of source domain data and thereby meeting privacy and security requirements. This paper provides a systematic classification and review of SFDA research methods, categorizing them into three main types: data-related methods, model-related methods, and loss-related methods. Data-related methods replace missing source data by extracting data or feature augmentation information from pre-trained models; model-related methods reduce domain discrepancies by optimizing feature representations or utilizing information in the feature space; and loss-related methods enhance the model's generalization ability through specific loss functions. This paper aims to offer a clear research roadmap for researchers in the field by systematically classifying and analyzing existing SFDA methods, facilitating the selection of appropriate methods or the development of new strategies to address specific problems.

In industrial scenarios, the source domain (SD) data typically encompasses condition monitoring (CM) data from all machines within a workshop or factory setting, while the target domain (TD) data may only include CM data from one or a small number of machines. The intelligent diagnostic method based on partial domain adaptation (PDA) represents a powerful tool for aligning features between SD and TD data within partial categories. However, existing PDA techniques can only align either the marginal or conditional distributions between SD and TD data within the shared label space, but not both simultaneously. To overcome this limitation, our study introduces a dual structural consistent PDA network. This network leverages the vision transformer as its foundation, ensuring effective extraction of distinguishable features from both SD and TD data. A weight balance mechanism is integrated into the partial adversarial training process, facilitating marginal distribution alignment between SD and TD data within the shared label space. Additionally, a knowledge distillation based approach is employed for conditional distribution alignment across the two structural consistent networks, ensuring consistency in predictions for TD data. The effectiveness of our proposed method is demonstrated through its application on two sets of experimental faulty data, confirming its ability to provide a feature distribution that is not affected by domain changes but is discriminative for different classes when dealing with PDA tasks.

A novel classification method for GPR B-scan images based on weak-shot learning

Recent domain adaptation work tends to obtain a uniformed representation in an adversarial manner through joint learning of the domain discriminator and feature generator. However, this domain adversarial approach could render sub-optimal performances due to two potential reasons: First, it might fail to consider the task at hand when matching the distributions between the domains. Second, it generally treats the source and target domain data in the same way. In our opinion, the source domain data which serves the feature adaption purpose should be supplementary, whereas the target domain data mainly needs to consider the task-specific classifier. Motivated by this, we propose a dual adversarial network for domain adaptation, where two adversarial learning processes are conducted iteratively, in correspondence with the feature adaptation and the classification task respectively. The efficacy of the proposed method is first demonstrated on Visual Domain Adaptation Challenge (VisDA) 2017 challenge, and then on two newly proposed Ground/Satellite-to-Aerial Scene adaptation tasks. For the proposed tasks, the data for the same scene is collected not only by the traditional camera on the ground, but also by satellite from the out space and unmanned aerial vehicle (UAV) at the high-altitude. Since the semantic gap between the ground/satellite scene and the aerial scene is much larger than that between ground scenes, the newly proposed tasks are more challenging than traditional domain adaptation tasks. The datasets/codes can be found at https://github.com/jianzhelin/DuAN.

Sentiment Analysis is a fast growing sub area of Natural Language Processing which extracts user's opinion and classify it according to its polarity into positive, negative or neutral classes. This task of classification is required for many purposes like opinion mining, opinion summarization, contextual advertising and market analysis but it is domain dependent. The words used to convey sentiments in one domain is different from the words used to express sentiments in other domain and it is a costly task to annotate the corpora in every possible domain of interest before training the classifier for the classification. We are making an attempt to solve this problem by creating a sentiment aware dictionary using multiple domain data. The source domain data is labeled into positive and negative classes at the document level and the target domain data is unlabeled. The dictionary is created using both source and target domain data. The words used to express positive or negative sentiments in labeled data has relatedness weights assigned to it which signifies its co-occurrence frequency with the words expressing the similar sentiments in target domain. This work is carried out in Hindi, the official language of India. The web pages in Hindi language is booming very quickly after the introduction of UTF-8 encoding style. The dictionary can be used to classify the unlabeled data in the target domain by training a classifier.

A complex process fault diagnosis method based on manifold distribution adaptation

In recent years, the research of dependency parsing focuses on improving the accuracy of in-domain data and has made remarkable progress. However, the real world is different from a single scenario dataset, filled with countless scenarios that are not covered by the dataset, namely, out-of-domain. As a result, parsers that perform well on the in-domain data often suffer significant performance degradation on the out-of-domain data. Therefore, in order to adapt the existing in-domain parsers with substantial performance to the new domain scenario, cross-domain transfer learning techniques are essential to solve the domain problem in parsing. In this paper, we examine two scenarios for cross-domain transfer learning: semi-supervised and unsupervised cross-domain transfer learning. Specifically, we adopt a pretrained language model BERT for training on the source domain (in-domain) data at subword level and introduce two tri-training variant methods for the two scenarios so as to achieve the goal of cross-domain transfer learning. The system based on this paper participated in NLPCC-2019-shared-task on cross-domain dependency parsing and won the first place on the “subtask3-un-open” and “subtask4-semi-open” subtasks, indicating the effectiveness of the approaches adopted.

Contrastive domain-invariant generalization for remaining useful life prediction under diverse conditions and fault modes

In recent years, the research on dependency parsing focuses on improving the accuracy of the domain-specific (in-domain) test datasets and has made remarkable progress. However, there are innumerable scenarios in the real world that are not covered by the dataset, namely, the out-of-domain dataset. As a result, parsers that perform well on the in-domain data usually suffer from significant performance degradation on the out-of-domain data. Therefore, to adapt the existing in-domain parsers with high performance to a new domain scenario, cross-domain transfer learning methods are essential to solve the domain problem in parsing. This paper examines two scenarios for cross-domain transfer learning: semi-supervised and unsupervised cross-domain transfer learning. Specifically, we adopt a pre-trained language model BERT for training on the source domain (in-domain) data at the subword level and introduce self-training methods varied from tri-training for these two scenarios. The evaluation results on the NLPCC-2019 shared task and universal dependency parsing task indicate the effectiveness of the adopted approaches on cross-domain transfer learning and show the potential of self-learning to cross-lingual transfer learning.

Unsupervised open-set domain adaptation (UODA) is a realistic problem where unlabeled target data contain unknown classes. Prior methods rely on the coexistence of both source and target domain data to perform domain alignment, which greatly limits their applications when source domain data are restricted due to privacy concerns. In this paper we address the challenging hypothesis transfer setting for UODA, where data from source domain are no longer available during adaptation on target domain. Specifically, we propose to use pseudo-labels and a novel consistency regularization on target data, where using conventional formulations fails in this open-set setting. Firstly, our method discovers confident predictions on target domain and performs classification with pseudo-labels. Then we enforce the model to output consistent and definite predictions on semantically similar transformed inputs, discovering all latent class semantics. As a result, unlabeled data can be classified into discriminative classes coincided with either source classes or unknown classes. We theoretically prove that under perfect semantic transformation, the proposed objective that enforces consistency can recover the information of true labels in prediction. Experimental results show that our model outperforms state-of-the-art methods on UODA benchmarks.

Preoperative diagnosis of gastric cancer and primary gastric lymphoma is challenging and has important clinical significance. Inspired by the inductive reasoning learning of the human brain, transfer learning can improve diagnosis performance of target task by utilizing the knowledge learned from the other domains (source domain). However, most studies focus on single-source transfer learning and may lead to model performance degradation when a large domain shift exists between the single-source domain and target domain. By simulating the multi-modal information learning and transfer mechanism of human brain, this study designed a multisource transfer learning feature extraction and classification framework, which can enhance the prediction performance of the target model by using multisource medical data (domain). First, this manuscript designs a feature extraction network that takes the maximum mean difference based on the Wasserstein distance as an adaptive measure of probability distribution and extracts the domain-specific invariant representations between source and target domain data. Then, aiming at the random generation of parameters bringing uncertainties to prediction accuracy and generalization ability of extreme learning machine network, the 1-norm regularization is used to implement sparse constraints of the output weight matrix and improve the robustness of the model. Finally, some experiments are carried out on the data of two medical centers. The experimental results show that the area under curves (AUCs) of the method are 0.958 and 0.929 in the two validation cohorts, respectively. The method in this manuscript can provide doctors with a better diagnostic reference, which has certain practical significance.

Transfer learning aims to enhance performance in a target domain by exploiting useful information from auxiliary or source domains when the labeled data in the target domain are insufficient or difficult to acquire. In some real-world applications, the data of source domain are provided in advance, but the data of target domain may arrive in a stream fashion. This kind of problem is known as online transfer learning. In practice, there can be several source domains that are related to the target domain. The performance of online transfer learning is highly associated with selected source domains, and simply combining the source domains may lead to unsatisfactory performance. In this paper, we seek to promote classification performance in a target domain by leveraging labeled data from multiple source domains in online setting. To achieve this, we propose a new online transfer learning algorithm that merges and leverages the classifiers of the source and target domain with an ensemble method. The mistake bound of the proposed algorithm is analyzed, and the comprehensive experiments on three real-world data sets illustrate that our algorithm outperforms the compared baseline algorithms.

Labeling remote sensing data for classification is labor-intensive and time-consuming. Transfer learning (TL), under such context, is attracting increasing attention as it aims to harness information from data set of other regions where labels are readily available. The central topic of concern is to homogenize the large disparities of feature distribution of different data set through domain adaptation (DA). This article proposes a novel DA method for unsupervised TL, namely, multikernel jointly domain matching (MKJDM), which by definition considers multiple kernels as opposed to the currently popular single-kernel methods for measuring the distances between distributions. The single-kernel methods minimize the distances of feature distribution between the source domain (data set with training labels) and the target domain (data set to be classified) through, for example, maximum mean discrepancy (MMD) metric, formed under a kernel function mapping, while the multikernel version (MK-MMD) uses different kernel functions to encapsulate multiple aspects of distribution discrepancies, and is, therefore, more capable of distance minimization. Our MKJDM implementation also considers simultaneously aligning marginal and class conditional distributions and reweight for each instance, which further improves the performance. Two experiments performed on remote sensing images and multi-modal data sets (i.e., Orthophoto and Digital Surface Models), with regions of different countries with distinctly different land patterns serving as source and target domain data, show that the overall accuracies are improved by 37.28% and 46.62% after applications of our MKJDM method. An additional comparative experiment with five state-of-the-art DA methods also demonstrates that our method achieves the best performance.