A Physics-Guided Transfer Learning Framework with Consistency Verification for Cross-Domain Bearing Fault Diagnosis

Transfer learning model to predict flow boiling heat transfer coefficient in mini channels with micro pin fins

We propose a novel unsupervised transfer learning framework that utilises unlabelled auxiliary data to quantify and select the most relevant transferrable knowledge for recognising a target object class from the background given very limited training target samples. Unlike existing transfer learning techniques, our method does not assume that auxiliary data are labelled, nor the relationships between target and auxiliary classes are known a priori. Our unsupervised transfer learning is formulated by a novel kernel adaptation transfer (KAT) learning framework, which aims to (a) extract general knowledge about how more structured objects are visually distinctive from cluttered background regardless object class, and (b) more importantly, perform selective transfer of knowledge extracted from the auxiliary data to minimise negative knowledge transfer suffered by existing methods. The effectiveness and efficiency of the proposed approach is demonstrated by performing one-class object recognition (object vs. background) task using the Caltech256 dataset.

High-definition metrology (HDM) has gained significant attention for surface quality inspection since it can reveal spatial surface variations in detail. Due to its cost and durability, such HDM measurements are occasionally implemented. The limitation creates a new research opportunity to improve surface variation characterization by fusing the insights gained from limited HDM data with widely available low-resolution surface data during quality inspections. A useful insight from state-of-the-art research using HDM is the revealed relationship and positive correlation between surface height and certain measurable covariates, such as material removal rate (MRR). Such a relationship was assumed spatially constant and integrated with surface measurements to improve surface quality modeling. However, this method encounters challenges when the covariates have nonstationary relationships with the surface height over different surface areas, i.e., the covariate-surface height relationship is spatially varying. Additionally, the nonstationary relationship can only be captured by HDM, adding to the challenge of surface modeling when most training data are measured at low resolution. This paper proposes a transfer learning (TL) framework to deal with these challenges by which the common information from a spatial model of an HDM-measured surface is transferred to a new surface where only low-resolution data are available. Under this framework, the paper develops and compares three surface models to characterize the nonstationary relationship including two varying coefficient-based spatial models and an inference rule-based spatial model. Real-world case studies were conducted to demonstrate the proposed methods for improving surface modeling.

Deep learning has provided numerous breakthroughs in natural imaging tasks. However, its successful application to medical images is severely handicapped with the limited amount of annotated training data. Transfer learning is commonly adopted for the medical imaging tasks. However, a large covariant shift between the source domain of natural images and target domain of medical images results in poor transfer learning. Moreover, scarcity of annotated data for the medical imaging tasks causes further problems for effective transfer learning. To address these problems, we develop an augmented ensemble transfer learning technique that leads to significant performance gain over the conventional transfer learning. Our technique uses an ensemble of deep learning models, where the architecture of each network is modified with extra layers to account for dimensionality change between the images of source and target data domains. Moreover, the model is hierarchically tuned to the target domain with augmented training data. Along with the network ensemble, we also utilize an ensemble of dictionaries that are based on features extracted from the augmented models. The dictionary ensemble provides an additional performance boost to our method. We first establish the effectiveness of our technique with the challenging ChestXray-14 radiography data set. Our experimental results show more than 50% reduction in the error rate with our method as compared to the baseline transfer learning technique. We then apply our technique to a recent COVID-19 data set for binary and multi-class classification tasks. Our technique achieves 99.49% accuracy for the binary classification, and 99.24% for multi-class classification.

Incremental transfer learning for spatial autoregressive model with linear constraints

Timely identification about health disorders has been made possible by photographic technologies, therefore photographic evidence provides a helpful technique towards illness identification. In addition to being labor-intensive, mechanical imagery processing techniques are prone to inter- and intra-observer inconsistency. Those restrictions may be addressed by computerized diagnostic imagery investigation methods. Transfer Learning (TL) frameworks using computerized health-related imagery processing have been looked at within the paper. It is has found how transfer learning can be used for many different healthcare scanning responsibilities, including recognizing objects, diseases classification, separation, and sensitivity scoring, etc. In contrast to conventional deep learning techniques, it is demonstrated that transfer learning offers superior selection assistance and uses reduced experimental input. A total of 100 peer-approved English language publications using the archives, IEEE Xplore and PubMed, where obtained till April 2025. A total of 53 research papers are considered suitable to the subject matter of this study after the PRISMA procedures for article screening was applied. Transfer learning techniques, such as pattern extraction, pattern extraction mix, fine tuning, and fine tuning from the start, are examined by works that concentrated on choosing core algorithms. Most research conducted experimental evaluations about several algorithms, then examined shallower depth approaches. Depth framework that has been used frequently within the academic research is Inception. In order to determine the best arrangement for the Transfer Learning, most research quantitatively compared several methods. just one methodology was used throughout the remaining experiments, then the two highly popular methods included pattern extraction and fine-tuning from start. Some research used pretrained algorithms for fine tuning and extraction of features hybrids. According to this study, the best popular Transfer Learning algorithms in analyzing medical images are AlexNet, ResNet, VGGNet, and GoogleNet. Such TL algorithms have been shown to be capable of comprehending healthcare photos, plus their capability to do so is improved by customisation, rendering them valuable instruments in studying photo scans.

BackgroundHistopathology images of tumor biopsies present unique challenges for applying machine learning to the diagnosis and treatment of cancer. The pathology slides are high resolution, often exceeding 1GB, have non-uniform dimensions, and often contain multiple tissue slices of varying sizes surrounded by large empty regions. The locations of abnormal or cancerous cells, which may constitute a small portion of any given tissue sample, are not annotated. Cancer image datasets are also extremely imbalanced, with most slides being associated with relatively common cancers. Since deep representations trained on natural photographs are unlikely to be optimal for classifying pathology slide images, which have different spectral ranges and spatial structure, we here describe an approach for learning features and inferring representations of cancer pathology slides based on sparse coding.ResultsWe show that conventional transfer learning using a state-of-the-art deep learning architecture pre-trained on ImageNet (RESNET) and fine tuned for a binary tumor/no-tumor classification task achieved between 85% and 86% accuracy. However, when all layers up to the last convolutional layer in RESNET are replaced with a single feature map inferred via a sparse coding using a dictionary optimized for sparse reconstruction of unlabeled pathology slides, classification performance improves to over 93%, corresponding to a 54% error reduction.ConclusionsWe conclude that a feature dictionary optimized for biomedical imagery may in general support better classification performance than does conventional transfer learning using a dictionary pre-trained on natural images.

<p>Soil image classification is a critical task within the realms of agriculture and environmental applications. In recent years, the integration of deep learning has sparked significant interest in image-based soil classification. Transfer learning, a well-established technique in image classification, involves finetuning a pre-trained model on a specific dataset. However, conventional transfer learning methods typically focus solely on fine-tuning the final layer of the pre-trained model, which may not suffice to attain high performance on a new task. HybridTransferNet, a unique hybrid transfer learning approach designed for soil classification based on images is proposed in this paper. HybridTransferNet goes beyond the conventional approach by finetuning not only the final layer but also a select number of earlier layers in a pre-trained ResNet50 model. This extension results in substantially enhanced ability to classify when compared to standard transfer learning methods. Our evaluation of HybridTransferNet, conducted on a soil classification dataset, encompasses the reporting of various performance indicators, such as the F1 score, recall, accuracy, and precision. Our findings from experiments highlight HybridTransferNet's advantages over conventional transfer learning strategies, establishing it as a state-of-the-art solution in the domain of soil classification.</p>

Feature space transformation of user-clicks and deep transfer learning framework for fraudulent publisher detection in online advertising

A general multi-source ensemble transfer learning framework integrate of LSTM-DANN and similarity metric for building energy prediction

There has been an amplified focus on and benefit from the adoption of artificial intelligence (AI) in medical imaging applications. However, deep learning approaches involve training with massive amounts of annotated data in order to guarantee generalization and achieve high accuracies. Gathering and annotating large sets of training images require expertise which is both expensive and time-consuming, especially in the medical field. Furthermore, in health care systems where mistakes can have catastrophic consequences, there is a general mistrust in the black-box aspect of AI models. In this work, we focus on improving the performance of medical imaging applications when limited data is available while focusing on the interpretability aspect of the proposed AI model. This is achieved by employing a novel transfer learning framework, progressive transfer learning, an automated annotation technique and a correlation analysis experiment on the learned representations. Progressive transfer learning helps jump-start the training of deep neural networks while improving the performance by gradually transferring knowledge from two source tasks into the target task. It is empirically tested on the wrist fracture detection application by first training a general radiology network RadiNet and using its weights to initialize RadiNetwrist , that is trained on wrist images to detect fractures. Experiments show that RadiNetwrist achieves an accuracy of 87% and an AUC ROC of 94% as opposed to 83% and 92% when it is pre-trained on the ImageNet dataset. This improvement in performance is investigated within an explainable AI framework. More concretely, the learned deep representations of RadiNetwrist are compared to those learned by the baseline model by conducting a correlation analysis experiment. The results show that, when transfer learning is gradually applied, some features are learned earlier in the network. Moreover, the deep layers in the progressive transfer learning framework are shown to encode features that are not encountered when traditional transfer learning techniques are applied. In addition to the empirical results, a clinical study is conducted and the performance of RadiNetwrist is compared to that of an expert radiologist. We found that RadiNetwrist exhibited similar performance to that of radiologists with more than 20 years of experience. This motivates follow-up research to train on more data to feasibly surpass radiologists’ performance, and investigate the interpretability of AI models in the healthcare domain where the decision-making process needs to be credible and transparent.

The purpose of transfer learning is to utilize the knowledge gained from the existing (source) domain to enhance the performance on a distinct but related (target) domain. Existing works on transfer learning are not capable of optimizing different quality measures (components) such as minimizing the marginal distribution, minimizing the conditional distribution, maximizing the target domain variance, modeling the manifold by utilizing the common geometric properties in the source as well as the target domain at the same time. Moreover, existing transfer learning methods use conventional approaches to determine the appropriate values of their parameters, which is very hectic and time-consuming. Therefore, in order to overcome the drawbacks of existing approaches, we propose a Particle Swarm Optimization based Parameter Selection Approach for Unsupervised Discriminant Analysis (UDATL-PSO) in transfer learning framework. In UDATL-PSO, all the quality measures are considered at the same time, as well as the PSO approach has been used to select the best values of their parameters. Extensive experiments on various transfer learning tasks show that the proposed method has a significant influence on state-of-the-art methods.

Annotating medical images for disease detection is often tedious and expensive. Moreover, the available training samples for a given task are generally scarce and imbalanced. These conditions are not conducive for learning effective deep neural models. Hence, it is common to ‘transfer’ neural networks trained on natural images to the medical image domain. However, this paradigm lacks in performance due to the large domain gap between the natural and medical image data. To address that, we propose a novel concept of Pre-text Representation Transfer (PRT). In contrast to the conventional transfer learning, which fine-tunes a source model after replacing its classification layers, PRT retains the original classification layers and updates the representation layers through an unsupervised pre-text task. The task is performed with (original, not synthetic) medical images, without utilizing any annotations. This enables representation transfer with a large amount of training data. This high-fidelity representation transfer allows us to use the resulting model as a more effective feature extractor. Moreover, we can also subsequently perform the traditional transfer learning with this model. We devise a collaborative representation based classification layer for the case when we leverage the model as a feature extractor. We fuse the output of this layer with the predictions of a model induced with the traditional transfer learning performed over our pre-text transferred model. The utility of our technique for limited and imbalanced data classification problem is demonstrated with an extensive five-fold evaluation for three large-scale models, tested for five different class-imbalance ratios for CT based COVID-19 detection. Our results show a consistent gain over the conventional transfer learning with the proposed method.KeywordsTransfer learningImbalanced dataCOVID-19

Existing frameworks for transfer learning are incomplete from a systems theoretic perspective. They place emphasis on notions of domain and task, and neglect notions of structure and behavior. In doing so, they limit the extent to which formalism can be carried through into the elaboration of their frameworks. Herein, we use the Mesarovician systems theory to define transfer learning as a relation on sets, and subsequently, characterize the general nature of transfer learning as a mathematical construct. We interpret existing frameworks in terms of ours and go beyond existing frameworks to define notions of transferability, transfer roughness, and transfer distance. Importantly, despite its formalism, our framework avoids the detailed mathematics of the learning theory or machine learning solution methods without excluding their consideration. As such, we provide a formal, general systems framework for modeling transfer learning that offers a rigorous foundation for system design and analysis.

This work explores better adaptation methods to low-resource languages using an external language model (LM) under the framework of transfer learning. We first build a language-independent ASR system in a unified sequence-to-sequence (S2S) architecture with a shared vocabulary among all languages. During adaptation, we perform LM fusion transfer, where an external LM is integrated into the decoder network of the attention-based S2S model in the whole adaptation stage, to effectively incorporate linguistic context of the target language. We also investigate various seed models for transfer learning. Experimental evaluations using the IARPA BABEL data set show that LM fusion transfer improves performances on all target five languages compared with simple transfer learning when the external text data is available. Our final system drastically reduces the performance gap from the hybrid systems.