Transfer Learning Framework Research Articles

META-EEG: Meta-learning-based class-relevant EEG representation learning for zero-calibration brain–computer interfaces

Transfer learning for motor imagery-based brain–computer interfaces (MI-BCIs) struggles with inter-subject variability, hindering its generalization to new users. This paper proposes an advanced implicit transfer learning framework, META-EEG, designed to overcome the challenge arising from inter-subject variability. By incorporating gradient-based meta-learning with an intermittent freezing strategy, META-EEG ensures efficient feature representation learning, providing a robust zero-calibration solution.A comparative analysis reveals that META-EEG significantly outperforms all the baseline methods and competing methods on three different public datasets. Moreover, we demonstrate the efficiency of the proposed model through a neurophysiological and feature-representational analysis. With its robustness and superior performance on challenging datasets, META-EEG provides an effective solution for calibration-free MI-EEG classification, facilitating broader usability.

A novel deep transfer learning framework with adversarial domain adaptation: application to financial time-series forecasting

A novel deep transfer learning framework with adversarial domain adaptation: application to financial time-series forecasting

Adaptive and Simultaneous Trajectory Prediction for Heterogeneous Agents via Transferable Hierarchical Transformer Network

Simultaneously and accurately predicting trajectories of multiple heterogeneous agents is crucial for intelligent transportation systems (ITS) applications, e.g., connected and autonomous vehicles. Existing model-based and data-driven methods can achieve good prediction accuracy, but most of them neglect the domain shift issue and prevalent imperfect data problems, i.e., few-shot learning and zero-shot learning issues. To address these issues, we propose a multi-source transfer learning (TL) framework, transferable hierarchical Siamese Transformer network (T-HSTN), for trajectory prediction of multiple heterogeneous agents, e.g., vehicles, bicycles, and pedestrians, at urban unsignalized intersections under small data conditions. Specifically, by extending the self-attention mechanism and exploring feature representations of traffic scenes, a Transformer-based network that hierarchically extracts temporal/spatial features and map features is introduced as the basic prediction model. Moreover, a TL framework with adaptive learning and feature alignment modules is built to explore the feature representations of unfixed traffic scenes and align both statistical and deep features to learn domain-invariant knowledge. More challenging trajectory prediction experiments are designed, corresponding to newly-built or badly-instrumented intersections under real-world scenarios. Experimental results verify the proposed method’s high accuracy, transferability, and generability. Our work fills the gap in solutions and benchmarks for TL tasks in trajectory prediction for heterogeneous agents. The conducted TL experiments provide a more practical setting of considering imperfect data problems in trajectory prediction.

Smart manufacturing under limited and heterogeneous data: a sim-to-real transfer learning with convolutional variational autoencoder in thermoforming

ABSTRACT Data in advanced manufacturing are often sparse and collected from various sensory devices in a heterogeneous and multi-modal fashion. Thus, for such intricate input spaces, learning robust and reliable predictive models for product quality assessments entails implementing complex nonlinear models such as deep learning. However, these ‘data-greedy’ models require massive datasets for training, and they tend to exhibit poor generalization performance otherwise. To address the data paucity and the data heterogeneity in smart manufacturing applications, this paper introduces a sim-to-real transfer-learning framework. Specifically, using a unified wide-and-deep learning approach, the model pre-processes structured sensory data (wide) as well as high-dimensional thermal images (deep) separately, and then passes the respective concatenated features to a regressor for predicting product quality metrics. Convolutional variational autoencoder (ConvVAE) is utilized to learn concise representations of thermal images in an unsupervised fashion. ConvVAE is trained via a sim-to-real transfer learning approach, backed by theory-based heat transfer simulations. The proposed metamodeling framework was evaluated in an industrial thermoforming process case study. The results suggested that ConvVAE outperforms conventional dimensionality reduction methods despite limited data. A model explainability analysis was conducted and the resulting SHAP values demonstrated the agreement between the model’s predictions, theoretical expectations, and data correlation statistics.

A feature decomposition-based deep transfer learning framework for concrete dam deformation prediction with observational insufficiency

Most current dam deformation prediction models (DDPM) are mainly based on long-term sequential data, but observational insufficiency is still a universal challenge in dam deformation forecasting for old dams with unreliable measurements or newly-built dams after short operation. This study proposes a feature decomposition-based deep transfer learning framework (FD-DTL) with short-term observation for concrete dam deformation prediction. Specifically, the variational mode decomposition (VMD), dispersion entropy (DE), and maximal information coefficient (MIC) are combined to decompose a single sequence into trend, periodic and random components. Then, dynamic time warping (DTW) is applied to match similar feature components from other dams and the target components. Based on transfer learning, convolutional neural network (CNN) and long short-term memory (LSTM), a combined prediction model (TL-CNN-LSTM) is designed for each pair of similar feature components, and it transfers the generic knowledge learned from similar feature components to the target ones. Eventually, the predicted displacement is obtained by superimposing all feature components. The engineering application verifies that the proposed framework can effectively process and predict dam displacement under observational insufficiency with considerable generalization ability and computational accuracy, which provides a novel approach for establishment of DDPM with observational insufficiency.

Automatic tuning of robust model predictive control in iterative tasks using efficient Bayesian optimization

Robust model predictive control (RMPC) is an effective technology for controlling uncertain systems while robustly handling constraints, and its closed-loop performance heavily relies on the selection of objective functions. However, the objective functions are typically chosen to be close to the real control objectives, despite an objective function that leads to less conservative constraints often provides better closed-loop performance. In this paper, we propose an automatic tuning framework for RMPC in iterative tasks. In particular, we parameterize RMPC and develop a Bayesian optimization (BO) method to tune it by solving a black-box optimization problem. We then introduce an efficient transfer learning framework within BO, which speeds up the searching process and enhances the controller performance. The effectiveness of the proposed tuning framework is illustrated on numerical examples.

Transfer Learning with Kernel Methods

Transfer learning refers to the process of adapting a model trained on a source task to a target task. While kernel methods are conceptually and computationally simple models that are competitive on a variety of tasks, it has been unclear how to develop scalable kernel-based transfer learning methods across general source and target tasks with possibly differing label dimensions. In this work, we propose a transfer learning framework for kernel methods by projecting and translating the source model to the target task. We demonstrate the effectiveness of our framework in applications to image classification and virtual drug screening. For both applications, we identify simple scaling laws that characterize the performance of transfer-learned kernels as a function of the number of target examples. We explain this phenomenon in a simplified linear setting, where we are able to derive the exact scaling laws.

Transferring Vision-Language Models for Visual Recognition: A Classifier Perspective

Transferring knowledge from pre-trained deep models for downstream tasks, particularly with limited labeled samples, is a fundamental problem in computer vision research. Recent advances in large-scale, task-agnostic vision-language pre-trained models, which are learned with billions of samples, have shed new light on this problem. In this study, we investigate how to efficiently transfer aligned visual and textual knowledge for downstream visual recognition tasks. We first revisit the role of the linear classifier in the vanilla transfer learning framework, and then propose a new paradigm where the parameters of the classifier are initialized with semantic targets from the textual encoder and remain fixed during optimization. To provide a comparison, we also initialize the classifier with knowledge from various resources. In the empirical study, we demonstrate that our paradigm improves the performance and training speed of transfer learning tasks. With only minor modifications, our approach proves effective across 17 visual datasets that span three different data domains: image, video, and 3D point cloud.

Fusion with Joint Distribution and Adversarial Networks: A New Transfer Learning Approach for Intelligent Fault Diagnosis

Bearings and gears are important components in rotating machinery, and the diagnosis of faults in bearings and gears has always been an important topic. Currently, data-driven fault diagnosis is a better method. However, under actual working conditions, domain shift can easily occur due to different operating conditions, leading to difficulties in transfer learning and significantly reducing the diagnostic performance of the model. Re-labeling the fault types of the model is time-consuming and costly. To overcome these difficulties, a new unsupervised transfer learning framework based on the fusion of joint distribution and adversarial networks has been introduced for the fault diagnosis of bearings and gears in rotating machinery. The joint adaptation network learns the transfer network by aligning the joint distribution of multiple specific domain layers across domains, based on Joint Maximum Mean Discrepancy (JMMD) to achieve domain alignment. At the same time, the domain classifier in the adversarial network is used to minimize the domain classification loss as domain distribution difference to minimize domain shift. The fusion of these two methods achieves domain alignment, reduces model training time, and improves the accuracy and stability of the model. The experimental results demonstrate that the proposed model framework exhibits excellent performance in detecting and classifying different types of faults. The new model framework also demonstrates outstanding performance across various fault detection and classification tasks.

Feature augmentation and semi-supervised conditional transfer learning for early detection of sepsis

Early detection of Sepsis is crucial for improving patient outcomes, as it is a significant public health concern that results in substantial morbidity and mortality. However, despite the widespread use of the Sequential Organ Failure Assessment (SOFA) in clinical settings to identify sepsis, obtaining sufficient physiological data before onset remains challenging, limiting early detection of sepsis. To address this challenge, we propose an interpretable machine learning model, ITFG (Interpretable Tree-based Feature Generation), that leverages potential correlations between features based on existing knowledge to identify sepsis within six hours of onset using valuable and continuous physiological measures. Furthermore, we introduce a Semi-supervised Attention-based Conditional Transfer Learning (SAC-TL) framework to enhance the model’s generality and enable it to be used for early warning of sepsis in the target domain with less information from the source domain. Our proposed approaches effectively address the problem of systematic feature sparsity and missing data, while also being practical for different degrees of generalizability. We evaluated our proposed approaches on open datasets, MIMIC and PhysioNet, obtaining AUC of 97.98% and 86.21%, respectively, demonstrating their effectiveness in different data environments and achieving the best early detection results.

Fuzzy-Membership-Based Framework for Task Transfer Learning Between Fault Diagnosis and RUL Prediction

Fault classification and remaining useful life (RUL) prediction are two important but different issues in data-driven prognostics and health management. Generally, they are resolved separately in present deep-learning-based methods, with one network orientated for one individual task, bringing low training efficiency and waste of computing resources. Although the output labels for these two tasks are entirely different, one is discrete and the other is continuous, they both reflect the equipment's health status and, thus, are related. To address these two tasks altogether in one deep network, this article proposes a new framework for task transfer learning. Long short-term memory plus two dense layers is built as the source model, and fuzzy membership is first adopted to construct new labels for source model training, with the target to bridge the gap between classical discrete and continuous output labels. Furthermore, three kinds of fuzzy membership functions (triangular, trapezoidal, and Gaussian) are designed for labeling and comparison. To verify the proposed framework, experimental validation is conducted on the C-MAPSS turbo-engine dataset. The degradation phase classification and RUL prediction are designed as discrete classification and continuous regression tasks, respectively. Results confirm that the fuzzy-membership-based labeling can effectively improve the source model performance in task transfer learning. In addition, the trapezoidal performs better than the triangular and Gaussian among the three membership functions.

A Deep Transfer Learning Framework for the Multi-Class Classification of Vector Mosquito Species

A Deep Transfer Learning Framework for the Multi-Class Classification of Vector Mosquito Species

Compositional Zero-Shot Domain Transfer with Text-to-Text Models

Abstract Label scarcity is a bottleneck for improving task performance in specialized domains. We propose a novel compositional transfer learning framework (DoT51) for zero-shot domain transfer. Without access to in-domain labels, DoT5 jointly learns domain knowledge (from masked language modelling of unlabelled in-domain free text) and task knowledge (from task training on more readily available general-domain data) in a multi-task manner. To improve the transferability of task training, we design a strategy named NLGU: We simultaneously train natural language generation (NLG) for in-domain label-to-data generation, which enables data augmentation for self-finetuning and natural language understanding (NLU) for label prediction. We evaluate DoT5 on the biomedical domain and the resource-lean subdomain of radiology, focusing on natural language inference, text summarization, and embedding learning. DoT5 demonstrates the effectiveness of compositional transfer learning through multi-task learning. In particular, DoT5 outperforms the current state-of-the-art in zero-shot transfer by over 7 absolute points in accuracy on RadNLI. We validate DoT5 with ablations and a case study demonstrating its ability to solve challenging NLI examples requiring in-domain expertise.

FedPos: A Federated Transfer Learning Framework for CSI-Based Wi-Fi Indoor Positioning

This article proposes FedPos, a federated transfer learning framework together with a novel position estimation method for Wi-Fi indoor positioning. Compared with traditional machine learning with privacy leakage problems and the cloud model trained through federated learning (FL) fails in personalization, the FedPos framework aggregates nonclassification layer parameters of models trained from different environments to build a robust and versatile encoder on the cloud server while preserving user privacy. The global cloud encoder can aggregate different classifiers and then construct personalized models for new users through fine-tuning. The proposed framework can be updated incrementally and is highly extensible. Specifically, we exploit channel state information (CSI) as the positioning feature and assess the transferability of a lightweight convolutional neural network (CNN) in unfamiliar environments. We evaluate the performance of our proposed framework and position estimation method in different indoor environments. Our experimental results indicate that the proposed framework can achieve a mean localization error of 42.18 cm in a 64-position living room. They also confirm that FedPos can achieve a 5.22% average localization performance boost and reduce the average model training time by about 34.78% when compared with normal training. By reusing part of the feature extractor layers that are trained from other environments, at least 65% of training data can be saved to achieve a localization performance that is similar to the base model. Overall, the proposed position estimation method can effectively improve localization accuracy as compared with seven other existing CSI-based methods.

AutoUnmix: an autoencoder-based spectral unmixing method for multi-color fluorescence microscopy imaging.

Multiplexed fluorescence microscopy imaging is widely used in biomedical applications. However, simultaneous imaging of multiple fluorophores can result in spectral leaks and overlapping, which greatly degrades image quality and subsequent analysis. Existing popular spectral unmixing methods are mainly based on computational intensive linear models, and the performance is heavily dependent on the reference spectra, which may greatly preclude its further applications. In this paper, we propose a deep learning-based blindly spectral unmixing method, termed AutoUnmix, to imitate the physical spectral mixing process. A transfer learning framework is further devised to allow our AutoUnmix to adapt to a variety of imaging systems without retraining the network. Our proposed method has demonstrated real-time unmixing capabilities, surpassing existing methods by up to 100-fold in terms of unmixing speed. We further validate the reconstruction performance on both synthetic datasets and biological samples. The unmixing results of AutoUnmix achieve the highest SSIM of 0.99 in both three- and four-color imaging, with nearly up to 20% higher than other popular unmixing methods. For experiments where spectral profiles and morphology are akin to simulated data, our method realizes the quantitative performance demonstrated above. Due to the desirable property of data independency and superior blind unmixing performance, we believe AutoUnmix is a powerful tool for studying the interaction process of different organelles labeled by multiple fluorophores.

Adaptive resizer-based transfer learning framework for the diagnosis of breast cancer using histopathology images

Adaptive resizer-based transfer learning framework for the diagnosis of breast cancer using histopathology images

Multi-sensor and multi-platform retrieval of water chlorophyll a concentration in karst wetlands using transfer learning frameworks with ASD, UAV, and Planet CubeSate reflectance data

China has one of the widest distributions of carbonate rocks in the world. Karst wetland is a special and important ecosystem of carbonate rock regions. Chlorophyll-a (Chla) concentration is a key indicator of eutrophication, and could quantitatively evaluate water quality status of karst wetland. However, the spectral reflectance characteristics of the water bodies of karst wetland are not yet clear, resulting in remote sensing retrieval of Chla with great challenges. This study is a pioneer in utilizing field-based full-spectrum hyperspectral data to reveal the spectral response characteristics of karst wetland water body and determine the sensitive spectral bands of Chla. We further evaluated the Chla retrieval performance of multi-platform spectral data between Analytical Spectral Device (ASD), Unmanned aerial vehicle (UAV), and PlanetScope (Planet). We proposed two multi-sensor weighted integration strategies and two transfer learning frameworks for estimating water Chla from the largest karst wetland in China by combing a partial least square with adaptive ensemble algorithms. The results showed that: (1) In the range of 500–850 nm, the spectral reflectance of water bodies in the karst wetland was overall 0.001–0.105 higher than the inland water bodies, and the sensitive spectral ranges of water Chla focus on 603–778 nm; (2) UAV images outperformed ASD and Planet data, and produced the highest inversion accuracy (R2 = 0.670) for water Chla in karst wetland; (3) Multi-sensor weighted integration retrieval methods improved the Chla estimation accuracy (R2 = 0.716). Integration retrieval methods with the different weights produced the better Chla estimation accuracy than that of methods with the equal weights; (4) The transfer learning methods from ASD to UAV platform provided the better retrieval performance (the average R2 = 0.669) than that of methods from UAV to Planet platform. The transfer learning methods obtained the highest estimation accuracy of Chla (R2 = 0.814) when the ratio of the training and test data in the target domain was 7:3. The transfer learning methods produced the higher estimation accuracies with the distribution of the absolute residuals between predicted and measured values <20.957 mg/m3 compared to the multi-sensor weighted integration retrieval methods, which demonstrated that transfer learning is more suitable for estimating Chla in karst wetland water bodies using multi-platform and multi-sensor data. The results provide a scientific basis for the protection and sustainable development of karst wetlands.

Development of VGG-16 transfer learning framework for geographical landmark recognition

Computer vision mandates the development of landmark recognition paradigms for efficient Image Recognition. In this article, the concept of Visual Geometry Group Network (VGG-16) transfer learning is used to develop an AI model over a geographical landmarks’ dataset. The dataset is a small part of Google Landmarks dataset V2 which originally consists of over 4M images. A VGG-16 model trained on ImageNet dataset is used to transfer knowledge. A positive transfer of knowledge is seen and it was observed that the trained model was a highly generalized model for our dataset. Not only a training accuracy of more than 0.85 is observed but equivalent validation accuracy suggests that the developed model is highly robust with minimal overfitting. A comparison of our proposed approach was made with classical machine learning techniques like KNN (K Nearest Neighbor), Decision Trees, Random Forest, SVM (Support Vector Machines) and a 3 Layered CNN. The results clearly depict that the proposed approach outperforms all other machine learning classifiers in consideration.

Classical-to-quantum convolutional neural network transfer learning

Machine learning using quantum convolutional neural networks (QCNNs) has demonstrated success in both quantum and classical data classification. In previous studies, QCNNs attained a higher classification accuracy than their classical counterparts under the same training conditions in the few-parameter regime. However, the general performance of large-scale quantum models is difficult to examine because of the limited size of quantum circuits, which can be reliably implemented in the near future. We propose transfer learning as an effective strategy for utilizing small QCNNs in the noisy intermediate-scale quantum era to the full extent. In the classical-to-quantum transfer learning framework, a QCNN can solve complex classification problems without requiring a large-scale quantum circuit by utilizing a pre-trained classical convolutional neural network (CNN). We perform numerical simulations of QCNN models with various sets of quantum convolution and pooling operations for MNIST data classification under transfer learning, in which a classical CNN is trained with Fashion-MNIST data. The results show that transfer learning from classical to quantum CNN performs considerably better than purely classical transfer learning models under similar training conditions.

Health status prediction of lithium ion batteries based on zero-shot learning

In this paper, we propose an adaptive transfer learning method for predicting battery health status. We construct a multi-task transfer learning framework to address the problem of transferring battery health status predictions across different usage scenarios. To overcome the challenge of determining the optimal weight for different task loss functions, we introduce an adaptive optimization method that automatically assigns the optimal weight to the multi-task objective function. Our experiments on lithium-ion battery data from Huazhong University of Science and Technology show that our proposed method outperforms other typical prediction methods.