Transductive Transfer Learning Research Articles

Enhancing indoor thermal comfort prediction in tropical regions: A transfer learning strategy in West Bengal

Ensuring energy-efficient building management and sustainability necessitates precise thermal comfort prediction, a task particularly critical in tropical regions with significant energy-saving potential. This study confronts the complexities introduced by humid air conditions and escalating internal and external heat levels. The objective is to devise a sophisticated strategy for predicting indoor personal thermal comfort that enhances prediction accuracy. The proposed method extends beyond previous techniques by incorporating a comprehensive array of thermal comfort factors and their potential interdependencies, while also leveraging a transductive transfer learning strategy to circumvent data sparsity in the target domain. The experimental results indicate the presence of localized feature coupling within thermal comfort datasets, which the proposed CNN-ConvTrans model effectively processes, thereby enhancing predictive efficacy. The primary innovation of this work lies in the development of a novel transfer learning strategy that considers localized feature coupling, integrated with a deep learning model, effectively addressing issues such as class imbalance and data scarcity. This study furnishes an innovative and valuable framework for thermal comfort prediction across diverse environmental settings.

Contrastive Learning and Cycle Consistency-Based Transductive Transfer Learning for Target Annotation

Contrastive Learning and Cycle Consistency-Based Transductive Transfer Learning for Target Annotation

Genetic Programming for Document Classification: A Transductive Transfer Learning System.

Document classification is a challenging task to the data being high-dimensional and sparse. Many transfer learning methods have been investigated for improving the classification performance by effectively transferring knowledge from a source domain to a target domain, which is similar to but different from the source domain. However, most of the existing methods cannot handle the case that the training data of the target domain does not have labels. In this study, we propose a transductive transfer learning system, utilizing solutions evolved by genetic programming (GP) on a source domain to automatically pseudolabel the training data in the target domain in order to train classifiers. Different from many other transfer learning techniques, the proposed system pseudolabels target-domain training data to retrains classifiers using all target-domain features. The proposed method is examined on nine transfer learning tasks, and the results show that the proposed transductive GP system has better prediction accuracy on the test data in the target domain than existing transfer learning approaches including subspace alignment-domain adaptation methods, feature-level-domain adaptation methods, and one latest pseudolabeling strategy-based method.

Resource efficient PV power forecasting: Transductive transfer learning based hybrid deep learning model for smart grid in Industry 5.0

This paper presents an innovative approach for enhancing power output forecasting of Photovoltaic (PV) power plants in dynamic environmental conditions using a Hybrid Deep Learning Model (DLM). The hybrid DLM employs a synergy of Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM) network, and Bidirectional LSTM (Bi-LSTM), effectively capturing spatial and temporal dependencies within weather data crucial for accurate predictions. To optimize the DLM’s performance efficiently, a unique Kepler Optimization Algorithm (KOA) is introduced for hyperparameter tuning, drawing inspiration from Kepler’s laws of planetary motion. By leveraging KOA, the DLM attains optimal hyperparameter configurations, elevating power output prediction precision. Additionally, this study integrates Transductive Transfer Learning (TTL) with the deep learning models to enhance resource efficiency. By leveraging knowledge gained from previously learned tasks, TTL enables the DLM to improve its forecasting capabilities while minimizing resource utilization. Datasets encompassing environmental parameters and PV plant-generated power across diverse sites are employed for DLM training and testing. Three hybrid models, amalgamating KOA, CNN, LSTM, and Bi-LSTM techniques, are introduced and evaluated. Comparative assessment of these models across distinct PV sites yields insightful observations. Performance evaluation, focused on short-term PV power forecasting, underscores the hybrid DLM’s superiority over individual CNN and LSTM models. This hybrid approach achieves remarkable accuracy and resilience in predicting power output under varying weather conditions, showcasing its potential for efficient PV power plant management.

UNSUPERVISED TRANSUDATIVE TL FEATURE LEARNING FOR IMAGE FEATURE EXTRACTION AND REPRESENTATION

In this study, we address the problem of unsupervised transductive transfer learning for image feature extraction and representation. While transfer learning has shown promising results in various domains, its application to image feature extraction in an unsupervised transductive setting remains relatively unexplored. The research gap lies in the scarcity of methods that can effectively learn meaningful image representations without access to labeled data in the target domain, hindering the broader applicability of transfer learning in computer vision. Our research seeks to bridge this gap by proposing a novel framework that leverages unsupervised feature learning to enhance the adaptability of models across different image domains, thus contributing to the advancement of transfer learning in the field of computer vision. Experimental results demonstrate the effectiveness of our method in addressing this critical research gap and its potential for real-world applications.

Transfer learning of clinical outcomes from preclinical molecular data, principles and perspectives.

Accurate transfer learning of clinical outcomes from one cellular context to another, between cell types, developmental stages, omics modalities or species, is considered tremendously useful. When transferring a prediction task from a source domain to a target domain, what counts is the high quality of the predictions in the target domain, requiring states or processes common to both the source and the target that can be learned by the predictor reflected by shared denominators. These may form a compendium of knowledge that is learned in the source to enable predictions in the target, usually with few, if any, labeled target training samples to learn from. Transductive transfer learning refers to the learning of the predictor in the source domain, transferring its outcome label calculations to the target domain, considering the same task. Inductive transfer learning considers cases where the target predictor is performing a different yet related task as compared with the source predictor. Often, there is also a need to first map the variables in the input/feature spaces and/or the variables in the output/outcome spaces. We here discuss and juxtapose various recently published transfer learning approaches, specifically designed (or at least adaptable) to predict clinical (human in vivo) outcomes based on preclinical (mostly animal-based) molecular data, towards finding the right tool for a given task, and paving the way for a comprehensive and systematic comparison of the suitability and accuracy of transfer learning of clinical outcomes.

Developing High-Resolution Crop Maps for Major Crops in the European Union Based on Transductive Transfer Learning and Limited Ground Data

Precise and timely information on crop spatial distribution over large areas is paramount to agricultural monitoring, food security, and policy development. Currently, automatically classifying crop types at a large scale is challenging due to the scarcity of ground data. Although previous studies have indicated that transductive transfer learning (TTL) is a promising method to address this problem, it performs poorly within regions where crop compositions and phenology differ largely. Here we transferred random forest classifiers trained in limited regions with diversified growing conditions and land covers to the rest of the study area where ground data are scarce, with more than 130,000 Sentinel-2 images processed using the Google Earth Engine (GEE) platform. We established the 10 m crop maps for four major crops (i.e., maize, rapeseed, winter, and spring Triticeae crops) across 10 European Union (EU) countries from 2018 to 2019. The final crop maps had a high accuracy with overall accuracy generally greater than 0.89, with user’s accuracy and producer’s accuracy ranging from 0.72 to 0.98. Moreover, the resulting maps were consistent with the NUTS-2 level official statistics, with R2 consistently greater than 0.9. We further analyzed the crop rotation patterns and found that the rotation intervals across these EU countries were generally at least one year. Maize was dominantly rotated with winter Triticeae crops or converted to other land covers in the following year. Rapeseed was generally grown in rotation with winter Triticeae crops, whereas the rotation patterns of winter and spring Triticeae crops were more diversified. Red Edge Position (REP) and Normalized Difference Yellow Index (NDYI) played significant roles in crop classification across the EU. This study highlights the potential of the developed TTL method for crop classification over large spatial extents where labeled data are limited and the differences in crop compositions and phenology are relatively large.

Improving Neural Machine Translation for Low Resource Algerian Dialect by Transductive Transfer Learning Strategy.

This study is the first work on a transductive transfer learning approach for low-resource neural machine translation applied to the Algerian Arabic dialect. The transductive approach is based on a fine-tuning transfer learning strategy that transfers knowledge from the parent model to the child model. This strategy helps to solve the learning problem using limited parallel corpora. We tested the approach on a sequence-to-sequence model with and without the Attention mechanism. We first trained the models on a parallel multi-dialects Arabic corpus and then switch them to a low-resource of the Algerian dialect. Transductive transfer learning raises the BLEU score for the Seq2Seq model from 0.3 to more than 34, and for the Attentional-Seq2Seq model from less than 17 to more than 35. The obtained results prove the validity of this approach.

Transfer learning: a friendly introduction

Infinite numbers of real-world applications use Machine Learning (ML) techniques to develop potentially the best data available for the users. Transfer learning (TL), one of the categories under ML, has received much attention from the research communities in the past few years. Traditional ML algorithms perform under the assumption that a model uses limited data distribution to train and test samples. These conventional methods predict target tasks undemanding and are applied to small data distribution. However, this issue conceivably is resolved using TL. TL is acknowledged for its connectivity among the additional testing and training samples resulting in faster output with efficient results. This paper contributes to the domain and scope of TL, citing situational use based on their periods and a few of its applications. The paper provides an in-depth focus on the techniques; Inductive TL, Transductive TL, Unsupervised TL, which consists of sample selection, and domain adaptation, followed by contributions and future directions.

Lost in Transduction: Transductive Transfer Learning in Text Classification

Obtaining high-quality labelled data for training a classifier in a new application domain is often costly. Transfer Learning (a.k.a. “Inductive Transfer”) tries to alleviate these costs by transferring, to the “target” domain of interest, knowledge available from a different “source” domain. In transfer learning the lack of labelled information from the target domain is compensated by the availability at training time of a set of unlabelled examples from the target distribution. Transductive Transfer Learning denotes the transfer learning setting in which the only set of target documents that we are interested in classifying is known and available at training time. Although this definition is indeed in line with Vapnik’s original definition of “transduction”, current terminology in the field is confused. In this article, we discuss how the term “transduction” has been misused in the transfer learning literature, and propose a clarification consistent with the original characterization of this term given by Vapnik. We go on to observe that the above terminology misuse has brought about misleading experimental comparisons, with inductive transfer learning methods that have been incorrectly compared with transductive transfer learning methods. We then, give empirical evidence that the difference in performance between the inductive version and the transductive version of a transfer learning method can indeed be statistically significant (i.e., that knowing at training time the only data one needs to classify indeed gives an advantage). Our clarification allows a reassessment of the field, and of the relative merits of the major, state-of-the-art algorithms for transfer learning in text classification.

Transductive Transfer Learning for Domain Adaptation in Brain Magnetic Resonance Image Segmentation.

Segmentation of brain images from Magnetic Resonance Images (MRI) is an indispensable step in clinical practice. Morphological changes of sub-cortical brain structures and quantification of brain lesions are considered biomarkers of neurological and neurodegenerative disorders and used for diagnosis, treatment planning, and monitoring disease progression. In recent years, deep learning methods showed an outstanding performance in medical image segmentation. However, these methods suffer from generalisability problem due to inter-centre and inter-scanner variabilities of the MRI images. The main objective of the study is to develop an automated deep learning segmentation approach that is accurate and robust to the variabilities in scanner and acquisition protocols. In this paper, we propose a transductive transfer learning approach for domain adaptation to reduce the domain-shift effect in brain MRI segmentation. The transductive scenario assumes that there are sets of images from two different domains: (1) source—images with manually annotated labels; and (2) target—images without expert annotations. Then, the network is jointly optimised integrating both source and target images into the transductive training process to segment the regions of interest and to minimise the domain-shift effect. We proposed to use a histogram loss in the feature level to carry out the latter optimisation problem. In order to demonstrate the benefit of the proposed approach, the method has been tested in two different brain MRI image segmentation problems using multi-centre and multi-scanner databases for: (1) sub-cortical brain structure segmentation; and (2) white matter hyperintensities segmentation. The experiments showed that the segmentation performance of a pre-trained model could be significantly improved by up to 10%. For the first segmentation problem it was possible to achieve a maximum improvement from 0.680 to 0.799 in average Dice Similarity Coefficient (DSC) metric and for the second problem the average DSC improved from 0.504 to 0.602. Moreover, the improvements after domain adaptation were on par or showed better performance compared to the commonly used traditional unsupervised segmentation methods (FIRST and LST), also achieving faster execution time. Taking this into account, this work presents one more step toward the practical implementation of deep learning algorithms into the clinical routine.

Classification of Prostate Transitional Zone Cancer and Hyperplasia Using Deep Transfer Learning From Disease-Related Images.

PurposeThe diagnosis of prostate transition zone cancer (PTZC) remains a clinical challenge due to their similarity to benign prostatic hyperplasia (BPH) on MRI. The Deep Convolutional Neural Networks (DCNNs) showed high efficacy in diagnosing PTZC on medical imaging but was limited by the small data size. A transfer learning (TL) method was combined with deep learning to overcome this challenge.Materials and methodsA retrospective investigation was conducted on 217 patients enrolled from our hospital database (208 patients) and The Cancer Imaging Archive (nine patients). Using T2-weighted images (T2WIs) and apparent diffusion coefficient (ADC) maps, DCNN models were trained and compared between different TL databases (ImageNet vs. disease-related images) and protocols (from scratch, fine-tuning, or transductive transferring).ResultsPTZC and BPH can be classified through traditional DCNN. The efficacy of TL from natural images was limited but improved by transferring knowledge from the disease-related images. Furthermore, transductive TL from disease-related images had comparable efficacy to the fine-tuning method. Limitations include retrospective design and a relatively small sample size.ConclusionDeep TL from disease-related images is a powerful tool for an automated PTZC diagnostic system. In developing regions where only conventional MR scans are available, the accurate diagnosis of PTZC can be achieved via transductive deep TL from disease-related images.

Probabilistic Feature Extraction Techniques for Electrocardiogram Signal-A Review

Extraction of features for an ECG signal plays a vital role in making the diagnosis of most of the diseases associated with cardiac muscle as well as the different states of arrhythmias. In this study, a literature review over the comprehensive manner has been done for the probabilistic feature extraction technique of an electrocardiogram signal in making the analysis of various classification methods of ECG arrhythmia signals that has been proposed over the past years of research. The ECG arrhythmia classification methods includes few digital signal processing techniques, Fuzzy Logic techniques, Hidden Markov Model, Support Vector Machines, Genetic Algorithm, Particle Swarm Optimization, Artificial Neural Networks, Transductive Transfer Learning etc. with which each individual approach exhibits its own disadvantages and advantages. For diagnosing the heart’s clinical condition, Echocardiogram is considered to be an essential tool but it consumes more time for processing and analyzing the ECG rhythms which possess huge number of heart beats. Therefore, it is in the urge that the system of diagnosis has to be made automated for classifying the clinical states of heart rhythms and heart beats in making the diagnosis very accurately and in the précised manner. For analyzing the electrocardiogram signal subsequently, the fundamental characteristics in terms of features like peak amplitudes, time intervals are very much required in determining the heart functions.

Transductive transfer learning based Genetic Programming for balanced and unbalanced document classification using different types of features

Document classification is one of the predominant tasks in Natural Language Processing. However, some document classification tasks do not have ground truth while other similar datasets may have ground truth. Transfer learning can utilize similar datasets with ground truth to train effective classifiers on the dataset without ground truth. This paper introduces a transductive transfer learning method for document classification using two different text feature representations—the term frequency (TF) and the semantic feature doc2vec. It has three main contributions. First, it enables the sharing knowledge in a dataset using TF and a dataset using doc2vec in transductive transfer learning for performance improvement. Second, it demonstrates that the partially learned programs from TFs and from doc2vecs can be alternatively used to “label then learn” and they improve each other. Lastly, it addresses the unbalanced dataset problem by considering the unbalanced distributions on categories for evolving proper Genetic Programming (GP) programs on the target domains. Our experimental results on two popular document datasets show that the proposed technique effectively transfers knowledge from the GP programs evolved from the source domains to the new GP programs on the target domains using TF or doc2vec. There are obviously more than 10 percentages improvement achieved by the GP programs evolved by the proposed method over the GP programs directly evolved from the source domains. Also, the proposed technique effectively utilizes GP programs evolved from unbalanced datasets (on the source and target domains) to evolve new GP programs on the target domains, which balances predictions on different categories.

Activity recognition in wearables using adversarial multi-source domain adaptation

Human activity recognition (HAR) from wearable sensors data has become ubiquitous due to the widespread proliferation of IoT and wearable devices. However, recognizing human activity in heterogeneous environments, for example, with sensors of different models and make, across different persons and their on-body sensor placements introduces wide range discrepancies in the data distributions, and therefore, leads to an increased error margin. Transductive transfer learning techniques such as domain adaptation have been quite successful in mitigating the domain discrepancies between the source and target domain distributions without the costly target domain data annotations. However, little exploration has been done when multiple distinct source domains are present, and the optimum mapping to the target domain from each source is not apparent. In this paper, we propose a deep Multi-Source Adversarial Domain Adaptation (MSADA) framework that opportunistically helps select the most relevant feature representations from multiple source domains and establish such mappings to the target domain by learning the perplexity scores. We showcase that the learned mappings can actually reflect our prior knowledge on the semantic relationships between the domains, indicating that MSADA can be employed as a powerful tool for exploratory activity data analysis. We empirically demonstrate that our proposed multi-source domain adaptation approach achieves 2% improvement with OPPORTUNITY dataset (cross-person heterogeneity, 4 ADLs), whereas 13% improvement on DSADS dataset (cross-position heterogeneity, 10 ADLs and sports activities).

A transductive transfer learning approach for image classification

Among machine learning paradigms, unsupervised transductive transfer learning is useful when no labeled data from the target domain are available at training time, but there is accessible unlabeled target data during training phase instead. The current paper proposes a novel unsupervised transductive transfer learning method to find the specific and shared features across the source and the target domains. The proposed learning method then maps both domains into the respective subspaces with minimum marginal and conditional distribution divergences. It is shown that the discriminative learning across domains leads to boost the model performance. Hence, the proposed method discriminates the classes of both domains via maximizing the distance between each sample-pairs with different labels and via minimizing the distance between each instance-pairs of the same classes. We verified our approach using standard visual benchmarks, with the average accuracy of 46 experiments as 76.5%, which rates rather high in comparison with other state-of-the-art transfer learning methods through various cross-domain tasks.

Inter-subject pattern analysis for multivariate group analysis of functional neuroimaging. A unifying formalization

Background and objectiveIn medical imaging, population studies have to overcome the differences that exist between individuals to identify invariant image features that can be used for diagnosis purposes. In functional neuroimaging, an appealing solution to identify neural coding principles that hold at the population level is inter-subject pattern analysis, i.e. to learn a predictive model on data from multiple subjects and evaluate its generalization performance on new subjects. Although it has gained popularity in recent years, its widespread adoption is still hampered by the blatant lack of a formal definition in the literature. In this paper, we precisely introduce the first principled formalization of inter-subject pattern analysis targeted at multivariate group analysis of functional neuroimaging. MethodsWe propose to frame inter-subject pattern analysis as a multi-source transductive transfer question, thus grounding it within several well defined machine learning settings and broadening the spectrum of usable algorithms. We describe two sets of inter-subject brain decoding experiments that use several open datasets: a magneto-encephalography study with 16 subjects and a functional magnetic resonance imaging paradigm with 100 subjects. We assess the relevance of our framework by performing model comparisons, where one brain decoding model exploits our formalization while others do not. ResultsThe first set of experiments demonstrates the superiority of a brain decoder that uses subject-by-subject standardization compared to state of the art models that use other standardization schemes, making the case for the interest of the transductive and the multi-source components of our formalization The second set of experiments quantitatively shows that, even after such transformation, it is more difficult for a brain decoder to generalize to new participants rather than to new data from participants available in the training phase, thus highlighting the transfer gap that needs to be overcome. ConclusionThis paper describes the first formalization of inter-subject pattern analysis as a multi-source transductive transfer learning problem. We demonstrate the added value of this formalization using proof-of-concept experiments on several complementary functional neuroimaging datasets. This work should contribute to popularize inter-subject pattern analysis for functional neuroimaging population studies and pave the road for future methodological innovations.

Multi-source domain adaptation for image classification

In recent years, domain adaptation and transfer learning are known as promising techniques with admirable performance to deal with problems with distribution difference between the training (source domain) and test (target domain) data. In this paper, a novel unsupervised multi-source transductive transfer learning approach, referred to as multi-source domain adaptation for image classification (MDA), is proposed, to transfer knowledge across the selected samples of multiple-source domains and samples of target domain into a shared low-dimensional subspace with maximum decision regions. MDA extends maximum mean discrepancy criteria across multiple-source domains to find an optimal projection subspace and constructs embedded condensed domain-invariant clusters. Furthermore, MDA minimizes empirical risk and maximizes the rate of consistency between manifold and prediction function via learning an optimal classification. Extensive evaluations on two types of visual benchmark datasets under different difficulties illustrate that MDA significantly outperforms other baseline and state-of-the-art methods in both multiple- and single-source tasks. Our source code is available at https://github.com/jtahmores/MDA .

A Suppression Method of Concentration Background Noise by Transductive Transfer Learning for a Metal Oxide Semiconductor-Based Electronic Nose.

Signal drift caused by sensors or environmental changes, which can be regarded as data distribution changes over time, is related to transductive transfer learning, and the data in the target domain is not labeled. We propose a method that learns a subspace with maximum independence of the concentration features (MICF) according to the Hilbert-Schmidt Independence Criterion (HSIC), which reduces the inter-concentration discrepancy of distributions. Then, we use Iterative Fisher Linear Discriminant (IFLD) to extract the signal features by reducing the divergence within classes and increasing the divergence among classes, which helps to prevent inconsistent ratios of different types of samples among the domains. The effectiveness of MICF and IFLD was verified by three sets of experiments using sensors in real world conditions, along with experiments conducted in the authors’ laboratory. The proposed method achieved an accuracy of 76.17%, which was better than any of the existing methods that publish their data on a publicly available dataset (the Gas Sensor Drift Dataset). It was found that the MICF-IFLD was simple and effective, reduced interferences, and deftly managed tasks of transfer classification.

Deep transductive transfer learning framework for zero-day attack detection

Zero-day attack detection in Intrusion Detection Systems is challenging due to the lack of labeled instances. This paper applies manifold alignment approach of TL that transforms the source and target domains into a common latent space to evade the problem of different feature spaces and different marginal probability distributions among the domains. On the transformed space, a method is proposed for generating target soft labels to compensate for the lack of labeled target instances by applying the cluster correspondence procedures. On top of this, DNN is applied to build a framework for the detection of zero-day attacks. Authors have conducted several experiments using NSL-KDD and CIDD datasets to evaluate the performance of the proposed framework. From the experimental results it is evident that the proposed framework could successfully detect zero-day attacks on unseen data.